Mesoscale oceanic eddies have a visible signature on Sea Surface Temperature (SST) satellite images, portraying diverse patterns of coherent vortices, temperature gradients and swirling filaments. However, learning the regularities of such signatures defines a challenging pattern recognition task, due to their complex structure but also to the cloud coverage which can corrupt a large fraction of the image. We introduce a novel Deep Learning approach to classify sea temperature eddy signatures, even if they are corrupted by strong cloud coverage. A large dataset of SST image patches is automatically retained and used to train a CNN-based classifier. Classification is performed with very high accuracy on coherent eddy signatures and is robust to a high level of cloud coverage, surpassing human expert efficiency on this task. This methodology can serve to validate and correct detections on satellite altimetry, the standard method used until now to track mesoscale eddies.