Inorganic nanocrystals, such as gold, iron oxide and semiconductor quantum dots, offer promising prospects for cancer diagnostics, imaging and therapy, due to their specific plasmonic, magnetic or fluorescent properties. The organic coating, or surface ligands, of these nanoparticles ensures their colloidal stability in complex biological fluids and enables their functionalization with targeting functions. It also controls the interactions of the nanoparticle with biomolecules in their environment. It therefore plays a crucial role in determining nanoparticle biodistribution and, ultimately, the imaging or therapeutic efficiency. This review summarizes the various strategies used to develop optimal surface chemistries for the in vivo preclinical and clinical application of inorganic nanocrystals. It discusses the current understanding of the influence of the nanoparticle surface chemistry on its colloidal stability, interaction with proteins, biodistribution and tumor uptake, and the requirements to develop an optimal surface chemistry.