Semiconductor quantum dots are very efficient sources of single and highly indistinguishable photons. These properties rely on the possibility to coherently control the system at the single spin level. At this ultimate level of control, the quantum dot becomes a very sensitive probe of its solid-state environment and any interaction turns into a dephasing process that alters its coherence properties. In this topical review, we give an overview of the issue of charge noise which remains one of the main dephasing mechanisms to overcome. This phenomenon which strongly depends on sample preparation, originates from a fluctuating electrostatic landscape around the quantum dots and renders a unified description quite awkward. We present the common characteristic features induced by charge noise that have been observed in the resonant fluorescence experiments of single quantum dots and discuss the different approaches that have been proposed in the literature to circumvent this problem.