Joint ML Journal Club 7

Friday 14 Mar 2025, 10:00 → 11:00 Europe/Berlin

O1.435 (FTX seminar room) (Building 1d)

Do you want to get the very latest news on research in ML?
The joint journal club meets on the second Friday of each month, covering new ML research for physicists across the board. We are a diverse group from theory to experiment, high energy to astrophysics. Everyone is welcome.

 

Join Zoom Meeting: https://desy.zoom.us/j/69478568183

Meeting ID: 694 7856 8183

Passcode: MLclub 

 

This week Anna Hallin will present their paper "OmniJet-α: The first cross-task foundation model for particle physics"; https://arxiv.org/abs/2403.05618

 

Mailing list: jointml@desy.de. We will announce upcoming meetings there. To subscribe, please visit lists.desy.de/sympa/subscribe/jointml.

Please do suggest papers that interest you! Or perhaps you'd like to present one of our suggested papers; [github.com/HenryDayHall/JointML](github.com/HenryDayHall/JointML)

map.png

10:00 → 10:20 OmniJet-α: The first cross-task foundation model for particle physics

Foundation models are multi-dataset and multi-task machine learning methods that once pre-trained can be fine-tuned for a large variety of downstream applications. The successful development of such general-purpose models for physics data would be a major breakthrough as they could improve the achievable physics performance while at the same time drastically reduce the required amount of training time and data.
We report significant progress on this challenge on several fronts. First, a comprehensive set of evaluation methods is introduced to judge the quality of an encoding from physics data into a representation suitable for the autoregressive generation of particle jets with transformer architectures (the common backbone of foundation models). These measures motivate the choice of a higher-fidelity tokenization compared to previous works. Finally, we demonstrate transfer learning between an unsupervised problem (jet generation) and a classic supervised task (jet tagging) with our new OmniJet-α model. This is the first successful transfer between two different and actively studied classes of tasks and constitutes a major step in the building of foundation models for particle physics.

Speaker: Anna Hallin (UNI/EXP (Uni Hamburg, Institut fur Experimentalphysik))

OmniJet-alpha_Anna_Hallin.pdf