Learning Unbiased Representations via Rényi Minimization - Sorbonne Université Access content directly
Conference Papers Year : 2021

Learning Unbiased Representations via Rényi Minimization

Oualid El Hajouji
  • Function : Author
Sylvain Lamprier
Marcin Detyniecki


In recent years, significant work has been done to include fairness constraints in the training objective of machine learning algorithms. Differently from classical prediction retreatment algorithms, we focus on learning fair representations of the inputs. The challenge is to learn representations that capture most relevant information to predict the targeted output Y, while not containing any information about a sensitive attribute S. We leverage recent work which has been done to estimate the Hirschfeld-Gebelein-Renyi (HGR) maximal correlation coefficient by learning deep neural network transformations and use it as a min-max game to penalize the intrinsic bias in a multi dimensional latent representation. Compared to other dependence measures, the HGR coefficient captures more information about the non-linear dependencies, making the algorithm more efficient in mitigating bias. After providing a theoretical analysis of the consistency of the estimator and its desirable properties for bias mitigation, we empirically study its impact at various levels of neural architectures. We show that acting at intermediate levels of neural architectures provides best expressiveness/generalization abilities for bias mitigation, and that using an HGR based loss is more efficient than more classical adversarial approaches from the literature.

Dates and versions

hal-03923320 , version 1 (04-01-2023)



Vincent Grari, Oualid El Hajouji, Sylvain Lamprier, Marcin Detyniecki. Learning Unbiased Representations via Rényi Minimization. European Conference on Machine Learning and Knowledge Discovery in Databases - ECML PKDD 2021, Sep 2021, Bilbao, Spain. pp.749-764, ⟨10.1007/978-3-030-86520-7_46⟩. ⟨hal-03923320⟩
24 View
0 Download



Gmail Mastodon Facebook X LinkedIn More