Many applications in geosciences require solving inverse problems to estimate the state of a physical system. Data assimilation provides a strong framework to do so when the system is partially observed and its underlying dynamics is known to some extent. In the variational flavor, it can be seen as an optimal control problem where initial conditions are the control parameters. Such problems being often ill-posed, regularization may be needed using explicit prior knowledge to enforce satisfying solution. In this work we propose to use a deep prior, a neural architecture that generates potential solution and acts as implicit regularization. The architecture is trained in an fully-unsupervised manner using the variational data assimilation cost so that gradients are backpropagated through the dynamical model and then through the neural network. To demonstrate its use, we set a twin experiment using a shallow-water toy model, where we test various variational assimilation algorithms on a ocean-like circulation estimation.