Léo Meynent

Thumbnail Image
Last Name
Meynent
First name
Léo
Email
leo.meynent@unisg.ch
Phone
+41 71 224 7184

Settings

Now showing 1 - 3 of 3
  • Thumbnail Image
    Some of the metrics are blocked by your 
    consent settings
    Item type:Publication,
    A Model Zoo on Phase Transitions in Neural Networks
    (2025-09-29)
    Konstantin Schürholt  
    ;
    Léo Meynent  
    ;
    Zhou, Yefan
    ;
    Lu, Haiquan
    ;
    Yang, Yaoqing
    Using the weights of trained Neural Network (NN) models as data modality has recently gained traction as a research field-dubbed Weight Space Learning (WSL). Multiple recent works propose WSL methods to analyze models, evaluate methods, or synthesize weights. Weight space learning methods require populations of trained models as datasets for development and evaluation. However, existing collections of models-called 'model zoos'-are unstructured or follow a rudimentary definition of diversity. In parallel, work rooted in statistical physics has identified phases and phase transitions in NN models. Models are homogeneous within the same phase but qualitatively differ from one phase to another. We combine the idea of 'model zoos' with phase information to create a controlled notion of diversity in populations. We introduce 12 large-scale zoos that systematically cover known phases and vary over model architecture, size, and datasets. These datasets cover different modalities, such as computer vision, natural language processing, and scientific ML. For every model, we compute loss landscape metrics and validate full coverage of the phases. With this dataset, we provide the community with a resource with a wide range of potential applications for WSL and beyond. Evidence suggests the loss landscape phase plays a role in applications such as model training, analysis, or sparsification. We demonstrate this in an exploratory study of the downstream methods like transfer learning or model weights averaging.
    Type:journal article
    Journal:Journal of Data-centric Machine Learning Research
    Volume:2
    URL:https://openreview.net/forum?id=zJRWvNpdIr
    URI:https://www.alexandria.unisg.ch/handle/20.500.14171/125485
  • Thumbnail Image
    Some of the metrics are blocked by your 
    consent settings
    Item type:Publication,
    A Model Zoo of Vision Transformers
    (2025-04-14)
    Damian Falk  
    ;
    Léo Meynent  
    ;
    Florence Pfammatter
    ;
    Konstantin Schürholt  
    ;
    Damian Borth  
    The availability of large, structured populations of neural networks-called "model zoos"-has led to the development of a multitude of downstream tasks ranging from model analysis, to representation learning on model weights or generative modeling of neural network parameters. However, existing model zoos are limited in size and architecture and neglect the transformer, which is among the currently most successful neural network architectures. We address this gap by introducing the first model zoo of vision transformers (ViT). To better represent recent training approaches, we develop a new blueprint for model zoo generation that encompasses both pre-training and fine-tuning steps, and publish 250 unique models. They are carefully generated with a large span of generating factors, and their diversity is validated using a thorough choice of weight-space and behavioral metrics. To further motivate the utility of our proposed dataset, we suggest multiple possible applications grounded in both extensive exploratory experiments and a number of examples from the existing literature. By extending previous lines of similar work, our model zoo allows researchers to push their model populationbased methods from the small model regime to state-of-the-art architectures. We make our model zoo available at github.com/ModelZoos/ViTModelZoo.
    Type:conference contribution
    Journal:ICLR Workshop on Neural Network Weights as a New Data Modality
    DOI:10.48550/arXiv.2504.10231
    URI:https://www.alexandria.unisg.ch/handle/20.500.14171/122786
  • Thumbnail Image
    Some of the metrics are blocked by your 
    consent settings
    Item type:Publication,