Background: Plasma neurofilament light (NFL) and total Tau (t-Tau) proteins are candidate biomarkers for early stages of Alzheimer's disease (AD). The impact of biological factors on their plasma concentrations in individuals with subjective memory complaints (SMC) has been poorly explored. We longitudinally investigate the effect of sex, age, APOE ε4 allele, comorbidities, brain amyloid-β (Aβ) burden, and cognitive scores on plasma NFL and t-Tau concentrations in cognitively healthy individuals with SMC, a condition associated with AD development. Methods: Three hundred sixteen and 79 individuals, respectively, have baseline and three-time point assessments (at baseline, 1-year, and 3-year follow-up) of the two biomarkers. Plasma biomarkers were measured with an ultrasensitive assay in a mono-center cohort (INSIGHT-preAD study). Results: We show an effect of age on plasma NFL, with women having a higher increase of plasma t-Tau concentrations compared to men, over time. The APOE ε4 allele does not affect the biomarker concentrations while plasma vitamin B12 deficiency is associated with higher plasma t-Tau concentrations. Both biomarkers are correlated and increase over time. Baseline NFL is related to the rate of Aβ deposition at 2-year follow-up in the left-posterior cingulate and the inferior parietal gyri. Baseline plasma NFL and the rate of change of plasma t-Tau are inversely associated with cognitive score. Conclusion: We find that plasma NFL and t-Tau longitudinal trajectories are affected by age and female sex, respectively, in SMC individuals. Exploring the influence of biological variables on AD biomarkers is crucial for their clinical validation in blood.