Viscoelasticity is an essential property of bone related to fragility, which is altered in aging and bone disease. Bone viscoelastic behavior is attributed to several mechanisms involving collagen and mineral properties, porosities, and bone hierarchical tissue organization. We aimed to assess the relationships between cortical bone viscoelastic damping measured with Resonant Ultrasound Spectroscopy (RUS), microstructural and compositional characteristics. We measured 52 bone specimens from the femur of 26 elderly human donors. RUS provided a shear damping coefficient at a frequency of the order of 150 kHz. The characteristics of the structure of the vascular pore network and tissue mineral density were measured using synchrotron radiation high-resolution computed tomography (SR-μCT). Fourier transformed infrared microspectroscopy (FTIRM) was used to quantify mineral-to-organic phase ratio, mineral maturity, crystallinity, and collagen maturity. Cross-links were quantified from biochemistry. Viscoelastic damping was found to increase with vascular porosity (r=0.68), to decrease with the degree of mineralization of the extravascular matrix (r=-0.68), and was marginally affected by collagen. We built a multilinear model suggesting that when porosity is controlled, the variation of mineral content explains a small additional part of the variability of damping. The work supports the consideration of viscoelasticity measurement as a potential biomarker of fragility and provides a documentation of bone viscoelastic behavior and its determinants in a frequency range rarely investigated.