Event Classification With Masked Transformer Autoencoders

Description

One of the key tasks in particle physics analyses is proper classification of particle collision events based on the parent particles and the process that produced them. To handle this task, we’re developing a flexible machine learning pipeline which can be applied to a broad range of classification tasks. We’ll leverage a mix of older and newer techniques for transformer models like masking, pretraining using autoencoder architectures, and cross attention of task-specific attention heads.

Duration

Total project length: 175/350 hours.

Task ideas

Develop a scalable transformer encoder model with task-specific attention heads combined using a cross attention mechanism
Improve existing code pipeline with features like multi-GPU parallelism and flexible preprocessing and analysis options
Deploy the developed models and pipeline on simulated physics data and analyze performance gains and changes in model understanding from the techniques used

Requirements

Significant experience in Python and Machine Learning in Pytorch. Preferably some experience with Transformers and multi-GPU parallelization or with the ROOT library developed by CERN.

Difficulty Level

Advanced

Mentors

Eric Reinhardt (University of Alabama)
Diptarko Choudhury (NISER)
Ruchi Chudasama (University of Alabama)
Emanuele Usai (University of Alabama)
Sergei Gleyzer (University of Alabama)

Please DO NOT contact mentors directly by email. Questions should instead be directed to ml4-sci@cern.ch which is forwarded to mentors. To submit your proposal, CV, and test task solutions, please use this Google form.