Visual neglect is a frequent and disabling consequence of right hemisphere damage. Previous work demonstrated a probable role of posterior callosal dysfunction in the chronic persistence of neglect signs. Prism adaptation is a non-invasive and convenient technique to rehabilitate chronic visual neglect, but it is not effective in all patients. Here we aimed to assess the hypothesis that prism adaptation improves left neglect by facilitating compensation through the contribution of the left, undamaged hemisphere. We assessed the relationship between prism adaptation effects, cortical thickness and white matter integrity in a group of 14 patients with unilateral right-hemisphere strokes and chronic visual neglect. Results showed that patients who benefitted from prism adaptation had thicker cortex in temporo-parietal, prefrontal and cingulate areas of the left, undamaged hemisphere. Additionally, these patients had a higher fractional anisotropy value in the body and genu of the corpus callosum. Results from normal controls show that these callosal regions connect temporo-parietal, sensorimotor and prefrontal areas. Finally, shorter time intervals from the stroke tended to improve patients' response to prism adaptation. We concluded that prism adaptation may improve left visual neglect by promoting the contribution of the left hemisphere to neglect compensation. These results support current hypotheses on the role of the healthy hemisphere in the compensation for stroke-induced, chronic neuropsychological deficits, and suggest that prism adaptation can foster this role by exploiting sensorimotor/prefrontal circuits, especially when applied at early stages post-stroke.