Calcified coralline algae are ecologically important in rocky habitats in the marine photic zone worldwide and there is growing concern that ocean acidification will severely impact them. Laboratory studies of these algae in simulated ocean acidification conditions have revealed wide variability in growth, photosynthesis and calcification responses, making it difficult to assess their future biodiversity, abundance and contribution to ecosystem function. Here, we apply molecular systematic tools to assess the impact of natural gradients in seawater carbonate chemistry on the biodiversity of coralline algae in the Mediterranean and the NW Pacific, link this to their evolutionary history and evaluate their potential future biodiversity and abundance. We found a decrease in the taxonomic diversity of coralline algae with increasing acidification with more than half of the species lost in high pCO2 conditions. Sporolithales is the oldest order (Lower Cretaceous) and diversified when ocean chemistry favoured low Mg calcite deposition; it is less diverse today and was the most sensitive to ocean acidification. Corallinales were also reduced in cover and diversity but several species survived at high pCO2; it is the most recent order of coralline algae and originated when ocean chemistry favoured aragonite and high Mg calcite deposition. The sharp decline in cover and thickness of coralline algal carbonate deposits at high pCO2 highlighted their lower fitness in response to ocean acidification. Reductions in CO2 emissions are needed to limit the risk of losing coralline algal diversity.