The mineralization level is heterogeneous in cortical bone extracellular matrix as a consequence of remodeling. Models of the effective elastic properties at the millimeter scale have been developed based on idealizations of the vascular pore network and matrix properties. Some popular models do not take into account the heterogeneity of the matrix. However, the errors on the predicted elasticity when the difference in elastic properties between osteonal and interstitial tissues is not modeled have not been quantified. This work provides an estimation of the maximum error. We compare the effective elasticity of a representative volume element (RVE) assuming (1) different elastic properties in osteonal and interstitial tissues vs. (2) average matrix properties. In order to account for the variability of bone microstructure, we use a collection of high resolution images of the pore network to build RVEs. In each RVE we assumed a constant osteonal wall thickness and we artificially varied this thickness between 35 and 140 µm to create RVEs with different amounts of osteonal tissue. The homogenization problem was solved with a fast Fourier transform (FFT) based numerical scheme. We found that the error depends on pore volume fraction and varies on average from 1% to 7% depending on the assumed diameter of the osteons. The results suggest that matrix heterogeneity may be disregarded in cortical bone models in most practical cases.