Speaker
Description
The forthcoming deployment of the High Energy Photon Source (HEPS), a fourth-generation light source, is set to produce an unprecedented volume of data, with estimates reaching hundreds of terabytes daily. This development signifies a considerable shift in synchrotron radiation technology and experimental approaches, transitioning from the previously straightforward data transmission pipeline—a single-channel flow from detector to storage—to a complex network involving multiple data generators and applications. To maintain an optimal experimental experience, there is a critical need for effective data management and distribution mechanisms. In response to these challenges, the Mamba Data Worker (MDW) has been developed to fulfill crucial roles such as real-time detector data acquisition, managing interactions with complex dataflows, extracting metadata, reducing data, and facilitating online assembly, writing, and processing. This report details the architecture and developmental roadmap of MDW, outlines the core technologies involved, and showcases its application in settings such as the Beijing Synchrotron Radiation Facility (BSRF) and HEPS, underlining its significance in advancing experimental procedures through enhanced data handling and processing capabilities.
| I plan to submit also conference proceedings | Yes |
|---|
