The impact of gravity waves (GW) on diurnal tides and the global circulation in the middle/upper atmosphere of Mars is investigated using a general circulation model (GCM). We have implemented a stochastic parameterization of non‐orographic GW into the Laboratoire de Météorologie Dynamique (LMD) Mars GCM (LMD‐MGCM) following an innovative approach. The source is assumed to be located above typical convective cells ( urn:x-wiley:jgre:media:jgre21298:jgre21298-math-0001250 Pa), and the effect of GW on the circulation and predicted thermal structure above 1 Pa ( urn:x-wiley:jgre:media:jgre21298:jgre21298-math-000250 km) is analyzed. We focus on the comparison between model simulations and observations by the Mars Climate Sounder (MCS) on board Mars Reconnaissance Orbiter during Martian Year 29. MCS data provide the only systematic measurements of the Martian mesosphere up to 80 km to date. The primary effect of GW is to damp the thermal tides by reducing the diurnal oscillation of the meridional and zonal winds. The GW drag reaches magnitudes of the order of 1 m/s/sol above 10 urn:x-wiley:jgre:media:jgre21298:jgre21298-math-0003 Pa in the northern hemisphere winter solstice and produces major changes in the zonal wind field (from tens to hundreds of m/s), while the impact on the temperature field is relatively moderate (10–20 K). It suggests that GW‐induced alteration of the meridional flow is the main responsible for the simulated temperature variation. The results also show that with the GW scheme included, the maximum day‐night temperature difference due to the diurnal tide is around 10 K, and the peak of the tide is shifted toward lower altitudes, in better agreement with MCS observations.