Studies suggest a decreasing trend in the consumption of meat products and a growing interest in vegetarian diets. Medical support may be relevant, especially when switching to a vegan diet. Our objective was to describe the beliefs and attitudes of primary care physicians toward vegetarian diets. A cross-sectional survey was conducted among general practitioners and pediatricians thorough a questionnaire including socio-demographic characteristics, specific care to vegetarians, and the risks and benefits of vegetarian diets according to physicians. Out of the 177 participating physicians, 104 (59%) have seen at least one vegetarian patient in consultation in the last three months. Half of the physicians declared that they would dissuade their patients from switching to a vegan diet (n = 88, 51%) and 14% (n = 24) from switching to an ovo-lacto-vegetarian (OLV) diet. Most physicians (n = 141, 88%) did not feel informed enough about these diets. Physicians thought that the most frequent deficiencies for OLV and vegan diets were iron (76% and 84%, respectively) and protein (45% and 79%, respectively). These results highlight the fact that French primary care physicians feel concerned by this subject and need more information on these diets. Specific recommendations would be useful to support their practice and relationship with vegetarians.