Studying the internal structure of complex samples with light is an important task but a difficult challenge due to light scattering. While the complex optical distortions induced by scattering can be effectively undone if the medium's scattering-matrix is known, this matrix generally cannot be retrieved without the presence of an invasive detector or guide-star at the target points of interest. To overcome this limitation, the current state-of-the-art approaches utilize focused ultrasound for generating acousto-optic guide-stars, in a variety of different techniques. Here, we introduce the acousto-optic transmission matrix (AOTM), which is an ultrasonically-encoded, spatially-resolved, optical scattering-matrix. The AOTM provides both a generalized framework to describe any acousto-optic based technique, and a tool for light control and focusing beyond the acoustic diffraction-limit inside complex samples. We experimentally demonstrate complex light control using the AOTM singular vectors, and utilize the AOTM framework to analyze the resolution limitation of acousto-optic guided focusing approaches.