26–28 May 2025
Europe/Berlin timezone

Enhancing HDF5 with Multi-Threading, Sparse Data Storage, and Encryption: Preliminary Results and Demonstrations

27 May 2025, 11:25
25m
FLASH seminar room

FLASH seminar room

FLASH Notkestrasse 85 22607 Hamburg
20-minute presentation + 5-minute Q&A

Speaker

Elena Pourmal (Lifeboat, LLC)

Description

Over the past few years, Lifeboat LLC has been focused on advancing the capabilities of HDF5 by incorporating multi-threaded support, enhancing the storage of sparse and variable-length data, and implementing robust encryption mechanisms for data stored within HDF5 files. These improvements are aimed at optimizing performance, increasing flexibility, and strengthening data security.
In our presentation, we will share the preliminary results of our work on multi-threading and sparse data storage. Specifically, we will showcase the progress made in enabling efficient multi-threaded I/O operations and managing sparse datasets more effectively. Additionally, we will demonstrate the newly introduced encryption feature for securing sensitive data within HDF5, ensuring both confidentiality and integrity.
We look forward to engaging with the community on these developments and receiving valuable feedback to further refine these features.

Key words: HDF5 multi-threading, sparse data, variable-size data, data encryption

May we record your session? Yes

Primary author

Elena Pourmal (Lifeboat, LLC)

Co-author

Mr John Mainzer (Lifeboat, LLC)

Presentation materials