Modeling the stochastic gravitational wave background from various astrophysical sources is a key objective in view of upcoming observations with ground- and space-based gravitational wave observatories such as Advanced LIGO, VIRGO, eLISA, and the pulsar timing array. We develop a synthetic model framework that follows the evolution of single and binary compact objects in an astrophysical context. We describe the formation and merger rates of binaries, the evolution of their orbital parameters with time, and the spectrum of emitted gravitational waves at different stages of binary evolution. Our approach is modular and allows us to test and constrain different ingredients of the model, including stellar evolution, black hole formation scenarios, and the properties of binary systems. We use this framework in the context of a particularly well-motivated astrophysical setup to calculate the gravitational wave background from several types of sources, including inspiraling stellar-mass binary black holes that have not merged during a Hubble time. We find that this signal, albeit weak, has a characteristic shape that can help constrain the properties of binary black holes in a way complementary to observations of the background from merger events. We discuss possible applications of our framework in the context of other gravitational wave sources, such as supermassive black holes.