The translocator protein (TSPO), an 18-kDa transmembrane protein primarily found in the outer mitochondrial membrane, is evolutionarily conserved and widely distributed across species. In mammals, TSPO has been described as a key member of a multiprotein complex involved in many putative functions and, over the years, several classes of ligand have been developed to modulate these functions. In this review, we consider the currently available atomic structures of mouse and bacterial TSPO and propose a rationale for the development of new ligands for the protein. We provide a review of TSPO monomeric and oligomeric states and their conformational flexibility, together with ligand-binding site and interaction mechanisms. These data are expected to help considerably the development of high-affinity ligands for TSPO-based therapies or diagnostics.