Approximating Lipschitz continuous functions with GroupSort neural networks - Sorbonne Université Access content directly
Conference Papers Year : 2021

Approximating Lipschitz continuous functions with GroupSort neural networks

Abstract

Recent advances in adversarial attacks and Wasserstein GANs have advocated for use of neural networks with restricted Lipschitz constants. Motivated by these observations, we study the recently introduced GroupSort neural networks, with constraints on the weights, and make a theoretical step towards a better understanding of their expressive power. We show in particular how these networks can represent any Lipschitz continuous piecewise linear functions. We also prove that they are well-suited for approximating Lipschitz continuous functions and exhibit upper bounds on both the depth and size. To conclude, the efficiency of GroupSort networks compared with more standard ReLU networks is illustrated in a set of synthetic experiments.

Dates and versions

hal-03895050 , version 1 (12-12-2022)

Identifiers

Cite

Ugo Tanielian, Maxime Sangnier, Gerard Biau. Approximating Lipschitz continuous functions with GroupSort neural networks. International Conference on Artificial Intelligence and Statistics, Apr 2021, San Diego, California, USA, France. pp.442-450. ⟨hal-03895050⟩
13 View
0 Download

Altmetric

Share

Gmail Facebook X LinkedIn More