Speaker
Description
High bandwidth instruments (data production rates of GB/s) have proliferated in photon science experimental facilities in the last years across the globe. Some of them are planned to be operated 24/7. Data volumes thus produced exceed both the budget of storage facilities and sometimes even the ingest capacities of hardware.
In this talk, I'd like to highlight key challenges when considering both lossless and lossy compression in photon science. I will highlight data science approaches to characterize or preprocess data. The talk will also showcase advances in finding optimal encoding parameters to achieve high data ingest bandwidths at high compression ratios. In addition, I'd like to introduce challenges for lossy compression with respect to good scientific practice and our advances to mitigate them without regressing to data quality metrics.
