Lorentz symmetry violations can be described by an effective field theory framework that contains both General Relativity and the Standard Model of particle physics called the Standard-Model extension (SME). Recently, post-fit analysis of Gravity Probe B and binary pulsars lead to an upper limit at the $10^{-4}$ level on the time-time coefficient $\bar s^{TT}$ of the pure-gravity sector of the minimal SME. In this work, we derive the observable of Very Long Baseline Interferometry (VLBI) in SME and then we implement it into a real data analysis code of geodetic VLBI observations. Analyzing all available observations recorded since 1979, we compare estimates of $\bar s^{TT}$ and errors obtained with various analysis schemes, including global estimations over several time spans and with various Sun elongation cut-off angles, and with analysis of radio source coordinate time series. We obtain a constraint on $\bar s^{TT}=(-5\pm 8)\times 10^{-5}$, directly fitted to the observations and improving by a factor 5 previous post-fit analysis estimates.