Laser Doppler holography is a planar blood flow imaging technique recently introduced in ophthalmology to image human retinal and choroidal blood flow non-invasively. Here we present a digital method based on the Doppler spectrum asymmetry that reveals the local direction of blood flow with respect to the optical axis in out-of-plane vessels. This directional information is overlaid on standard grayscale blood flow images to depict flow moving towards the camera in red and flow moving away from the camera in blue, as in ultrasound color Doppler imaging. We show that thanks to the strong contribution of backscattering to the Doppler spectrum in out-of-plane vessels, the local axial direction of blood flow can be revealed with a high temporal resolution, which enables us to evidence pathological blood flow reversals. We also demonstrate the use of optical Doppler spectrograms to quantitatively monitor retinal blood flow reversals.