NetCDF Compression Improvements

19 Sept 2023, 13:30
30m
Main auditorium, building 5 (DESY)

Main auditorium, building 5

DESY

Notkestrasse 85, 22607 Hamburg, Germany
Submitted talk Day 1

Speaker

Mr Edward Hartnett (CIRES/NOAA)

Description

New compression features have been added to the netCDF C and Fortran libraries, including lossy compression, zstandard, and parallel I/O support.

These features will help science data producers such as NOAA, NCAR, NASA, and ESA process, store and distribute the large scientific datasets produced by higher-resolution models and instruments.

The Community Codec Repository (CCR) will be used to bring additional compression filters to the netCDF community. Continuing to improve compression technology for the Earth science community requires collaboration and consultation to select which filters to support.

Primary author

Mr Edward Hartnett (CIRES/NOAA)

Presentation materials