Dispersing solid hard particles in an elasto-plastic material leads to important shear-history dependenceof the behavior, namely strain hardening and Bauschinger effect. Strain hardening is observed as theprogressive strengthening of a material during its plastic deformation and is usually associated withductility, a property often sought after in composite materials to postpone fractures and failure. Inaddition, anisotropic mechanical properties are developed, the material resistance being larger in thedirection of the imposed flow, which is referred to as the Bauschinger effect. We show that this isrelated here to shear-history-dependent particle-pair distribution functions. Roughness and interparticlecontacts likely play a major role, as replacing hard particles by non-deformable bubbles modifies thesuspension microstructure and suppresses strain hardening. Beyond suspensions, our study providesnew insight in the understanding and control of strain hardening and Bauschinger effect in compositematerials.