Localized surface plasmon resonance (LSPR) of metal nanoparticles are widely used to develop plasmonic sensors, which permit to detect minute amounts of molecular compounds. Two methods of measurements are commonly used. The first most common mode, which investigates the shift in wavelength of the LSPR induced by the analyte, requires the use of high-resolution monochromators. The second mode, based on self-reference or perfect absorbing systems, measures the changes of the signal intensity, and requires the use of sophisticated samples elaborated by lithography techniques. In this article, we adapt the very sensitive transmittance anisotropy spectroscopy technique with anisotropic plasmonic gold films, formed of slightly elongated nanoparticles easily elaborated by grazing deposition on microscopy glass slides. These films display two different LSPR, caused by their morphological anisotropy, as a function of the light polarization. Working at a single wavelength, we experimentally demonstrate an ultrahigh sensitivity to bulk refractive index sensing, with a resolution below delta_n=10-4 . The factor of merit (FoM*), defined for intensity-based plasmonic sensors, reaches the outstanding value of 23 000, which is due to the null optical anisotropy signal initially measured, thus granting an enhanced optical contrast when changing the RI. Furthermore, sensitivity down to four avidin molecules per square micron of the sample is theoretically shown, which offers a promising route towards easily made ultra-sensitive miniaturized plasmonic sensors, compatible with microfluidic systems, without the need for a monochromator.