Selecting Pre-trained Models for Transfer Learning with Data-centric Meta-features - Sorbonne Université
Conference Papers Year : 2024

Selecting Pre-trained Models for Transfer Learning with Data-centric Meta-features

Matt van den Nieuwenhuijzen
  • Function : Author
Carola Doerr
Henry Gouk
  • Function : Author
  • PersonId : 1136179

Abstract

When applying a neural network to address a new learning problem, rather than training a network from scratch, it is common practice to utilise a network pre-trained on a related dataset and then fine-tune it to the data of the target task. This poses the question: which pre-trained network should be selected? This work investigates this problem in the context of three different dataset relationships: same-source, same-domain, and cross-domain. We utilise Meta-Album, which offers an extensive collection of datasets from various unrelated domains. We first split each of the 30 datasets of Meta-Album into a meta-train dataset and meta-test dataset, then create pre-trained models for each meta-train dataset, and finally compare the performances of the pre-trained models in a fine-tuning context when applied to meta-test tasks. We categorise the performances into the three dataset relationship groups and find that the same-source category performs best in terms of accuracy. Then, using meta-features calculated on the meta-train dataset and meta-test tasks, we train statistical meta-models that are employed to select the best pre-trained model for a given meta-test task. Our best meta-model identifies the best-performing model in ∼ 25% of the cases. It improves upon a baseline that selects the best average model by more than 30%.
Fichier principal
Vignette du fichier
MetaAlbum-AutoML24-Poster.pdf (1.94 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04759450 , version 1 (29-10-2024)

Identifiers

  • HAL Id : hal-04759450 , version 1

Cite

Matt van den Nieuwenhuijzen, Carola Doerr, Jan N van Rijn, Henry Gouk. Selecting Pre-trained Models for Transfer Learning with Data-centric Meta-features. AutoML Conference 2024 (Workshop Track), Sep 2024, Paris, France. ⟨hal-04759450⟩
0 View
0 Download

Share

More