Contrast thresholds for discriminating orientation and direction of a drifting, oriented grating are usually similar to contrast detection thresholds, which suggest that the most sensitive detectors are labeled for both orientation and direction (Watson and Robson, 1981). This was found to be true in noiseless condition, but Arena et al. (2013) recently found that this was not true in localized noise (i.e., noise having the same spatiotemporal window as the target) as thresholds for discriminating direction were higher than for discriminating orientation. They suggested that this could be explained by the fact that there are more neurons selective to orientation than direction. Another possible interpretation is that, unlike contrast thresholds in absence of noise, the most sensitive detectors in localized noise were labeled for orientation, but not for direction. This hypothesis is supported by recent findings showing different processes operating in localized and extended noise (i.e., full-screen, continuously displayed noise, Allard and Cavanagh, 2011). In the current study, we evaluated contrast thresholds for orientation and direction discrimination tasks in noiseless conditions, and in noise that was either spatially localized or extended, and temporally localized or extended. We found similar orientation and direction thresholds in absence of noise and in temporally extended noise, but greater direction thresholds in temporally localized noise. This suggests that in noiseless and temporally extended noise the most sensitive detectors were labeled for both orientation and direction (e.g., direction-selective complex cells), whereas in temporally localized noise the most sensitive detectors were labeled for orientation but not direction (e.g., simple cells). We conclude that to avoid violating the noise-invariant processing assumption, external noise paradigms investigating motion processing should use noise that is temporally extended, not localized.