The young planetary system β Pictoris is surrounded by a debris disk of dust and gas. The gas source of this disk could be exocomets (or “falling and evaporating bodies”, FEBs), which produce refractory elements (Mg, Ca, Fe) through sublimation of dust grains at several tens of stellar radii. Nearly 1700 high resolution spectra of β Pictoris were obtained between 2003 and 2017 using the HARPS spectrograph. In Paper I, we showed that a high signal to noise ratio allows the detection of many weak Fe I lines in more than ten excited levels, and we derived the physical characteristics of the iron gas in the disk. The measured temperature of the gas (~1300 K) suggests that it is produced by evaporation of grains at about 0.3 au (38 R⋆) from the star. Here, we describe the yearly variations of the column densities of all Fe I components (from both ground and excited levels). The drop in the Fe I ground level column density after 2011 coincides with a drop in Fe I excited levels column density, as well as in the Ca II doublet and a ground level Ca I line at the same epoch. All drops are compatible with photoionisation-recombination equilibrium and β Pic like relative abundances, in a medium at 1300 K and at 0.3 au from β Pictoris. Interestingly, this warm medium does not correlate with the numerous exocomets in the circumstellar environnement of this young star.