— Surgeons can hardly perform manually complex tasks under ultrasound (US) imaging. Visual servoing appears as a possible solution to assist the surgeon in such an operation. The idea of the proposed system is to allow the surgeon to select a desired position on a US image and then program a robot to automatically move the instrument towards the location indicated by the surgeon. This approach requires robust tracking of the instrument in the US image, modeling of the overall system and implementation of a visual servoing loop. This paper presents geometrical and kinematic models of the system, as well as the control loop design, which is validated through both numerical simulations, and results of in vitro experiments.