Metastasis is the leading cause of cancer mortality. We have investigated the tumor microenvironment at all metastatic cascade steps (early-metastasic dissemination, synchronous metastasis, metachronous metastasis) to delineate the impact of tumor and immune parameters to this process. Tumors with and without signs of early metastasis invasion (venous-emboli, lymphatic-invasion, perineural-invasion, collectively, VELIPI) had similar levels of inflammatory and immunosuppressive molecules. Cancer mutations, gene expression levels or chromosomal instability did not significantly differ in primary tumors from patients with or without metastasis. In contrast, tumors without early metastasis invasion were highly infiltrated with Th1 and memory T cells and were associated with a good outcome. A cytotoxic immune signature, Immunoscore and increased lymphatic vessels at the invasive margin of tumors, protected against the generation of distant metastases. The metastatic landscape was highly heterogeneous, each of the metastases of a patient bearing diverse tumor-cell clones and diverse immune-microenvironments. The Immunoscore within a random metastasis significantly predicted major differences in patient’s survival, and Immunoscore from the least immune-infiltrated metastasis was the most associated with patient long-term survival. We proposed an alternative theory of tumor evolution, where an immune selection model best-described tumor evolution in humans. Metachronous metastasis revealed that immunoedited tumor clones are eliminated, while the immune privileged clones progress underlines relationships between clonal seeding and immune surveillance and advances the understanding of cancer evolution. A strong intratumoral immune infiltrate and Immunoscore prevent the metastatic invasion at all its steps and it is associated with prolonged survival.