Visual stimuli that are uncomfortable to look at evoke a large neural response suggesting altered processing. While there is some evidence linking uncomfortable achromatic stimuli to impaired visual processing, the effect of uncomfortable chromatic patterns on visual cognition has yet to be explored. Large differences in chromaticity separation (e.g. red and blue) elicit visual discomfort, larger metabolic responses, larger visual evoked potentials, and greater alpha suppression compared to small chromaticity separations (e.g. pink and purple). We investigated the impact of stimuli that varied in their chromaticity separation (calculated in perceptual color space) on a visual task and their effect on neural responses across the cortex. Thirty participants completed a continuous pairs task (letters changed at 3 Hz) while grating patterns that differed in their chromaticity separation alternated with a grey screen at 5 Hz. The different temporal frequencies allowed for steady-state visual evoked potentials (SSVEPs) to the two stimulus-types to be measured simultaneously using electroencephalography (EEG). A subset of participants rated the gratings on a 9-point scale of discomfort. We observed greater ratings of discomfort and increased power at 5 Hz with the larger chromaticity separations. The increase in 5 Hz power with greater chromaticity separation was evident across the cortex. However, there was no significant effect of chromaticity separation on power at 3 Hz, or on reaction times, and no consistent effect on behavioral accuracy. Despite eliciting heightened neural responses across the cortex, short term exposure to uncomfortable chromatic stimuli does not adversely impact visual task performance.