— In recent years, there has been an increasing interest in developing systems that couple a robotic device with an ultrasound imager. Applications range from automatic probe positioning to ultrasound image-based guidance of instruments. One issue in these systems is to determine, prior to the intervention, the localization of the probe with respect to the robot. Literature suggests using external localizers, but they add to the complexity of the system, and the resulting precision is usually not good due to the addition of errors in the kinematic chain. In this paper, we study the hand-eye calibration problem without using any additional localizers. A generic system consisting of a fixed probe observing an instrument manipulated by a robot is used. We first derive a simplified model for mapping the image of the instrument with its 3D location w.r.t. the probe and then propose a calibration procedure based on a minimization algorithm. Results show that although very simple models were used for the imaging device, the localization is quite precise, as it results in errors of less than 2mm, which is enough of a number of ultrasound guided interventions.