The interface between a solid and vacuum can become electronically distinct from the bulk. This feature, encountered in the case of quantum Hall effect, has a manifestation in insulators with topologically protected metallic surface states. Non-trivial Berry curvature of the Bloch waves or periodically driven perturbation are known to generate it. Here, by studying the angle-dependent magnetoresistance in prismatic bismuth crystals of different shapes, we detect a robust surface contribution to electric conductivity when the magnetic field is aligned parallel to a two-dimensional boundary between the three-dimensional crystal and vacuum. The effect is absent in antimony, which has an identical crystal symmetry, a similar Fermi surface structure and equally ballistic carriers, but an inverted band symmetry and a topological invariant of opposite sign. Our observation confirms that the boundary interrupting the cyclotron orbits remains metallic in bismuth, which is in agreement with what was predicted by Azbel decades ago. However, the absence of the effect in antimony indicates an intimate link between band symmetry and this boundary conductance.