Background: Genetic polymorphisms play a significant role in determining brain morphology, including white matter structure and may thus influence the development of brain functions. The main objective of this study was to examine the effect of Val158Met (rs4680) polymorphism of Catechol-O-Methyltransferase (COMT) gene on white matter connectivity in healthy adults. Methods: We used a whole-brain diffusion-weighted imaging method with Tract-Based Spatial Statistics (TBSS) analysis to examine white matter structural integrity in intrinsic brain networks on a sample of healthy subjects (N = 82). Results: Results revealed a sex-specific effect of COMT on corpus callosum (CC): in males only, Val homozygotes had significantly higher fractional anisotropy (FA) compared to Met-carriers. Volume-of-interest analysis showed a genotype by sex interaction on FA in genu and rostral midbody of CC, whereby Val males demonstrated higher FA than Met females. Conclusions: These results demonstrate the key effect of genes by sex interaction, rather than their individual contribution, on the corpus callosum anatomy.