Background: Despite improved health, unemployment has increased among people living with HIV (PlwHIV) over the last decade. However, since the economic recession of 2008, unemployment also increased in the French general population. This paper aimed to determine if the increase in the unemployment rate in the HIV population was higher than that in the French general population. Methods: We used data from the ANRS-Vespa study, a repeated cross-sectional survey among two national representative samples of PlwHIV followed at hospitals in France in 2003 and 2011. We compared employment and unemployment rates between HIV-infected people (overall and according to period of HIV diagnosis) and the French general population in 2003 and 2011, using multivariate Poisson regressions adjusted for individual sociodemographic characteristics. Results: The employment rate among PlwHIV was consistently lower than that in the general population in 2003 and 2011. In contrast, there was a trend of an increasing unemployment rate difference between PlwHIV and the general population: PlwHIV’s unemployment rate was 1.48 (95% confidence interval [CI]: 1.16–1.90) times higher than that of the general population in 2003, versus 1.62 (95% CI: 1.34–1.96) times higher in 2011. This unemployment rate difference was the highest for PlwHIV diagnosed in or after 2008 (adjusted prevalence rate ratio: 2.06; 95% CI: 1.59–2.67). Conclusions: These results suggest that in time of economic recession, an increasing proportion of PlwHIV may be excluded from the labor market although they are willing to re-enter it. This constitutes a major issue relative to social consequences of chronic disease