We analyze four- and six-derivative couplings in the low energy effective action of $D=3$ string vacua with half-maximal supersymmetry. In analogy with an earlier proposal for the $(\nabla\Phi)^4$ coupling, we propose that the $\nabla^2(\nabla\Phi)^4$ coupling is given exactly by a manifestly U-duality invariant genus-two modular integral. In the limit where a circle in the internal torus decompactifies, the $\nabla^2(\nabla\Phi)^4$ coupling reduces to the $D^2 F^4$ and $R^2 F^2$ couplings in $D=4$, along with an infinite series of corrections of order $e^{-R}$, from four-dimensional 1/4-BPS dyons whose wordline winds around the circle. Each of these contributions is weighted by a Fourier coefficient of a meromorphic Siegel modular form, explaining and extending standard results for the BPS index of 1/4-BPS dyons.