26–28 May 2025
Europe/Berlin timezone

Revisiting HDF5 SWMR and file versioning features

26 May 2025, 11:25
25m
FLASH seminar room

FLASH seminar room

FLASH Notkestrasse 85 22607 Hamburg
20-minute presentation + 5-minute Q&A

Speaker

Elena Pourmal (Lifeboat, LLC)

Description

In this talk, we will present an overview and demonstration of two HDF5 features implemented via virtual file drivers (VFDs), both currently in a prototype stage: the full single-writer/multiple-reader capability (VFD SWMR) and HDF5 versioning (also known as the Onion VFD).
The VFD SWMR feature enables file modifications during writing and provides guarantees on the maximum latency before new data becomes available to readers. The Onion VFD allows tracking of modifications to an HDF5 file between open/close calls while preserving—or providing access to—the file's state prior to a given set of changes.
Although the Onion VFD has been released and is available in current maintenance versions, a more robust implementation is still in development. In this presentation, we aim to gather feedback from the HDF5 user community on both VFDs and to share our roadmap for developing production-ready versions of these features.

May we record your session? Yes

Primary author

Elena Pourmal (Lifeboat, LLC)

Co-author

Dr Scot Breitenfeld (The HDF Group)

Presentation materials