Context. H2 is the simplest and the most abundant molecule in the interstellar medium (ISM), and its formation precedes the formation of other molecules. Aims. Understanding the dynamical influence of the environment and the interplay between the thermal processes related to the formation and destruction of H2 and the structure of the cloud is mandatory to understand correctly the observations of H2. Methods. We performed high-resolution magnetohydrodynamical colliding-flow simulations with the adaptive mesh refinement code RAMSES in which the physics of H2 has been included. We compared the simulation results with various observations of the H2 molecule, including the column densities of excited rotational levels. Results. As a result of a combination of thermal pressure, ram pressure, and gravity, the clouds produced at the converging point of HI streams are highly inhomogeneous. H2 molecules quickly form in relatively dense clumps and spread into the diffuse interclump gas. This in particular leads to the existence of significant abundances of H2 in the diffuse and warm gas that lies in between clumps. Simulations and observations show similar trends, especially for the HI-to-H2 transition (H2 fraction vs. total hydrogen column density). Moreover, the abundances of excited rotational levels, calculated at equilibrium in the simulations, turn out to be very similar to the observed abundances inferred from FUSE results. This is a direct consequence of the presence of the H2 enriched diffuse and warm gas. Conclusions. Our simulations, which self-consistently form molecular clouds out of the diffuse atomic gas, show that H2 rapidly forms in the dense clumps and, due to the complex structure of molecular clouds, quickly spreads at lower densities. Consequently, a significant fraction of warm H2 exists in the low-density gas. This warm H2 leads to column densities of excited rotational levels close to the observed ones and probably reveals the complex intermix between the warm and cold gas in molecular clouds. This suggests that the two-phase structure of molecular clouds is an essential ingredient for fully understanding molecular hydrogen in these objects.