The idea behind this meeting is to bring together those people interested in and working on machine learning, deep learning and artificial intelligence methods on DESY Campus, to get an overview how the heterogeneous mixture of scientists makes use of the same tools.
As in the previous years this meeting shall strengthen the network of AI-experts working door to door here on campus in sometimes entirely different domains using the same tools and algorithms.
This year DESY IT and Helmholtz Imaging join forces together with the novel seminar series called Physics ❤ AI organised by Gregor Kasieczka (UHH) and Janis Kummer (CDCS) to organise the Round Table on DL.
This meeting explicitly addresses colleagues from all institutions on campus, including DESY, UHH, EMBL, CSSB, XFEL, MPSD, Hereon and beyond. Please feel invited!
We will (hopefully) again meet in person at the Flash Seminar Room on DESY campus. There will be coffee breaks and a lunch break, which gives you the opportunity to get to know and meet the other deep learning experts from the campus. Please register below, so that we have a good idea how many people we have to expect.
Call for contributions: Main part of the round table will be short talks from all the different fields of DL application, to get the best possible overview. Please consider to present how you are using deep learning to push your research domain forward. Please submit a short abstract (just one short paragraph) here in Indico before 12 Nov 2022.
Machine learning is emerging as vital tool in many sciences. In quantum physics, notable examples are neural networks for the efficient representation of quantum many-body states and reinforcement learning of preparation and read-out routines. In this talk, I will present our results on machine learning of quantum phase transitions using classification techniques. This approach works very well even on noisy experimental data both with supervised and unsupervised machine learning. Next to the practical advantages, such techniques might in the future reveal phase transitions, for which conventional order parameters are not known.
Generative machine learning models allow fast event generation, yet are so far primarily constrained to fixed data and detector geometries.
We introduce a Deep Sets-based permutation equivariant generative adversarial network (GAN) for generating point clouds with variable cardinality - a flexible data structure optimal for collider events such as jets. The generator utilizes an interpretable global latent vector and does not rely on pairwise information sharing between particles, leading to a significant speed-up over graph-based approaches
The simulation of calorimeters using standard Monte Carlo based methods is expected to create a major computational bottleneck at the next generation of collider experiments, motivating the development of alternative approaches. Surrogate simulators based on deep generative models could potentially provide significant computational speedups- however the highly granular nature of many future detectors poses a major challenge in terms of the degree of physical fidelity required. This contribution reports progress on the simulation of both electromagnetic and hadronic showers, as well as steps taken to broaden the scope of these simulators, for example when conditioning a model on multiple parameters.
Simulation in High Energy Physics places a heavy burden on the available computing resources and is expected to become a major bottleneck for the upcoming high luminosity phase of the LHC and future Higgs factories, motivating a concerted effort to develop computationally efficient solutions. Generative machine learning methods hold promise to alleviate the computational strain produced by simulation while providing the physical accuracy required of a surrogate simulator.
Normalizing flows have shown significant potential in the field of fast calorimeter simulation in simple detector geometries. We expand on this by demonstrating how a normalizing flow setup can be extended to simulate showers in a significantly more complicated highly granular calorimeter.
We present techniques to perform model-agnostic searches for new physics at the LHC, both at the trigger and analysis level. We highlight some of the recently uncovered practical challenges and discuss proposed solutions.
Particle colliders such as the LHC produce data at an unprecedented rate and volume. To overcome bandwidth constraints, event filtering systems are employed, with the first stage usually implemented in hardware using FPGAs. We present the studies on real-time event filtering using machine learning for the Level-1 Trigger of the CMS experiment.
In high energy physics, detailed and time-consuming simulations are used for particle interactions with detectors. For the upcoming High-Luminosity phase of the Large Hadron Collider (HL-LHC), the computational costs of conventional simulation tools exceeds the projected computational resources. Generative machine learning is expected to provide a fast and accurate alternative. The CMS experiment at the LHC will use a new High Granularity Calorimeter (HGCal) to cope with the high particle density. The new HGCal is an imaging calorimeter with a complex geometry and more than 3 million cells. We report on the development of a GraphGAN to simulate particle showers under these challenging conditions.
The solution of the time-dependent Schrödinger equation completely characterizes the quantum dynamics of molecular systems. Because this equation is analytically solvable only for toy problems, for real-life simulations, a variety of numerical approximations have been developed. The most accurate methodologies employ the variational approach based on the linear expansion of the solutions in a set of basis functions. Recently, neural network approaches have emerged as a strong methodology for modeling the basis functions, which prove very efficient for solving quantum chemical problems.
I will present a novel approach for solving the stationary and time-dependent Schrödinger equation based on normalizing flows and invertible neural networks acting on the problem's real-space coordinates. In this method, instead of using neural networks as a basis, a set of classical orthogonal polynomials is composed together with a neural network function. This allows us to calculate a much larger number of excited states at a much lower cost. I'll present calculations for excited electronic and vibrational states of small molecules. To solve the time-dependent problems, the current approach is to cast in the form of a recurrent neural network that describes the molecular state on a discretized time lattice. This method will be used as a workbench for simulations of the Laser Induced Electron Diffraction (LIED) experiments, a technique for high spatial and temporal resolution imaging.
The House of Computing & Data Science is a central unit of U Hamburg for the digital transformation of research; the KIEZ of Computing and Data Science is a Hamburg-wide network to connect computation-oriented people in research and development. After motivating the need of such units and networks for all research disciplines, the current level of their implementation is described.
Machine learning techniques have left the drawing board and are currently industrialized in the aerospace sector.
First an overview of the wide range of these applications at Airbus is given, followed by a more detailed description of the research on how an LSTM and AutoEncoder-based dimensionality reduction approach is used to calculate the loads on an aircraft during landing. Finally, an outlook is given on what mastering these techniques will enable in the future.
Proteins are important biomolecules of life. While many proteins can function independently, most proteins interact with other proteins to control and mediate their biological activities. Hence, studying protein-protein interactions (PPIs) is important to better understand biological functions. There are several biological factors that can influence the presence or absence of a PPI. In our work, we are specifically interested in studying how a PPI is affected by a biological phenomenon called alternative splicing (AS), where a single gene gives rise to multiple proteins. We aim to use deep learning methods to predict the likelihood of a PPI in the event of AS. In detail, we use (and later modify) an existing geometric deep learning approach to predict PPIs. Intuitively, given 3-dimensional structures of two proteins, the approach first represents the surface of each protein as a "cloud'' of points. Then, the approach computes surface curvatures and chemical characteristics as features of each point. Finally, the approach uses these features to train a neural network architecture and predict which parts of the two proteins are interacting. In our work, we further advance this existing approach to more accurately predict which parts of the two proteins interact. To do this, we use a convolutional neural network to extract advanced features for each protein and use those features to classify interacting versus non-interacting parts of a given pair of proteins. In our on-going work, we aim to use our novel approach to predict PPIs in the event of AS.
Single particle imaging (SPI) with X-ray free-electron lasers (XFELs) sources can empower visualization of structural heterogeneity in biological entities such as viruses and complex proteins. We have developed a CNN-based generative network approach for mapping and reconstruction of the 3D structure of a target object at any point of its structural variation landscape. We discuss applications to ultrafast melting of gold nanoparticles and to natural variations in a virus structure.
Serial femtosecond crystallography produces large data volumes however, only a small percentage of the data is useful for downstream analysis. In this work, we handle serial crystallography data with deep learning for various challenges including classification, Bragg peak detection, domain adaptation, explainability.
Hard X-ray nanotomography is a commonly used tool in many research areas such as material science, biology and medicine. However, the quality of the reconstructed tomogram is often obscured by noise, especially for in situ experiments when a high time resolution is required. Machine learning (ML) techniques offer a powerful alternative to conventional filtering methods. Here, we present a ML denoising approach without the need of a reference scan. This technique is applied to high-resolution nanotomography data. The ML approach proves to be a very powerful tool that outperforms conventional filters by eliminating noise without blurring relevant structural features, thus enabling efficient quantitative analysis and fast measurements well suited for in situ applications.
I am developing a data analysis workflow involving machine learning technologies like recurrent and convolutional neural networks as well as classification systems, for the rapid and reliable identification and quantification of viral proteins in mass spectrometry data from human saliva samples. The goal is to implement this workflow in high-throughput quantitative testing for respiratory viral infections, including advanced features such as parallel testing for different viruses and early detection of viral mutations.
With the increasing success of deep learning techniques in many applications, researchers also use it for segmentation of biodegradable implants in high resolution synchrotron radiation microtomograms (SRμCT). Deep learning models, however, require lots of annotated training examples in order to generalize and perform well on new data. Manual annotation of high resolution SRμCT is a very time-consuming, and expensive task. Moreover, domain expertise is required for accurate segmentation of biodegradable bone implants. We leverage deep active learning techniques in this research to alleviate the need for annotating a large dataset, yet achieve a similar performance. A small number of annotated tomograms are used to train a model. The trained model and a so-called acquisition function are used to propose new tomograms to manually annotate and include in the training set which could potentially improve the performance of the deep learning model the most. The new annotated tomograms are added to the training set and the process is repeated, evaluating the trained model on a separate test dataset after each round. The process is halted when the increase in performance on the test dataset between two successive rounds is negligible. Experiments are conducted on an annotated dataset of biodegradable implants. There are 6 samples (5591 tomograms) for validation, and 6 samples (6000 tomograms) for testing. Even though 8 samples (6878 tomograms) are allocated for training, initially, only 8 tomograms (1 from each sample) from the training set are used to train the model in the first round. After each round, 8 new tomograms are added to the training set based on the scores calculated using the acquisition function. After 10 rounds of active learning and only 80 annotated tomograms for training, the model achieves a mean intersection over union (mIoU) score of 80.51% on the test dataset. Training a model on the whole dataset, i.e., 6787 annotated tomograms, in a vanilla supervised fashion yields a mIoU score of 82.91%, only ~2% higher, on the test dataset.
The abundance of data currently stored for heterogeneous subsystems represents an excellent opportunity for data analysis. Data-driven approaches, relying solely on data can significantly benefit from having massive streams of data available. So far, frequently the modeling activities focused on model-based analyses which relied on human expertise. In this talk, we provide examples of activities of purely data-driven approaches for anomaly detection and modeling of the behavior of pulsed linacs like European XFEL and their subsystems that do not rely on human-expertise modeling.
Virtual diagnostics involves using fast computational tools that can predict the output of a diagnostic when it is unavailable. One work in this direction proposes the use of ML learning methods at the European XFEL's SASE1 beamline to predict X-ray properties such as beam pointing using undulator electron properties. Such an approach is promising for providing accurate knowledge on X-ray pulses of high-repetition rate XFELs. Another application for hard X-ray self-seeding operation uses a machine learning classifier to identify crystal reflections and determine the seeding energy.
High-fidelity particle-in-cell simulations are an essential tool for the modeling and optimization of laser-plasma accelerators. However, the high computational cost associated with them severely limits the possibility of broad parameter exploration. Here, we show that a multitask Bayesian optimization algorithm can be used to mitigate the need for high-fidelity simulations by incorporating information from inexpensive evaluations with reduced physical models. In a proof-of-principle study combining the FBPIC (high fidelity) and Wake-T (reduced model) codes, this algorithm demonstrates an order-of-magnitude speedup when the optimization is assisted by the reduced-model simulations. This opens the way for cost-effective optimization of laser-plasma accelerators in large parameter spaces.
In this work we present a model-based approach for fault detection for the superconducting cavities at the European XFEL. With the help of a classification algorithm, a special class faults, the quenches, can be distinguished from others. Application results are presented.
Laser-plasma accelerators are promising candidates for driving compact undulator radiation sources. Future applications would greatly benefit from optimization by data-driven modelling of the acceleration process, but intrinsic noise and large parameter spaces poses a problem for conventional modelling methods. At LUX beamline we use an ensemble of neural networks and bootstrap aggregation to extract a model robust to noise and outliers.
Reinforcement learning (RL) has enabled the development of intelligent controllers for complex tasks that previously required human intuition to solve. In the context of particle accelerators, there exist many such tasks and solving them with conventional methods takes away from scarce experiment time and limits the operability of accelerators. We demonstrate how to successfully apply RL to the optimisation of part of a linear particle accelerator under highly limited partial observability and without requiring expensive beam time for training on the real machine. Our method outperforms conventional optimisation algorithms in both the achieved result and time taken, and achieves close to human-level performance. In the future, RL-based controllers like ours will enable more challenging beam configurations and significantly reduce the time required to attain them, thereby increasing both quality and quantity of experimental output of accelerator facilities and consequently enable scientific advances in the research fields served by these machines.