26–30 Aug 2024
Europe/Berlin timezone

ParSeq – a Tool for Creation of Comparative Data Analysis Pipelines

29 Aug 2024, 12:15
15m
Saal B

Saal B

Contributed talk 3. Data, Automation and the Use of AI Mikrosymposium 3/1: Data, Automation and the Use of AI

Speaker

Konstantin Klementiev (MAX IV Laboratory)

Description

Package ParSeq is a python software library for _Parallel execution of _Sequential data analysis. It implements a general analysis framework that consists of transformation nodes – intermediate stops along the analysis propagation to visualize data, display status and provide user input – and transformations that connect the nodes. It provides an adjustable data model (a collection of data objects that supports grouping, renaming, moving and drag-and-drop actions), tunable data format definitions, plotters for 1D, 2D and 3D data, cross-data analysis routines and flexible widget work space suitable for single- and multi-screen computers. It also defines a structure of Python modules to implement particular analysis pipelines as relatively lightweight Python packages.

ParSeq is intended for synchrotron based techniques, first of all spectroscopy.

In this contribution, I present the basic ideas about the analysis framework, go through the list of centralized facilities (data import/export, plotting, project saving/restoring, undo, dynamically built html help pages etc) and show examples of a few ready pipelines for XAS, XES and XRD.

The main ParSeq package and a few analysis ParSeq pipelines are available on GitHub [github.com/kklmn/ParSeq]. Its online documentation can be viewed on parseq.readthedocs.io .

Fig 1. A screenshot of ParSeq-XAS (an EXAFS analysis pipeline, github.com/kklmn/ParSeq-XAS) as an application example that builds on top of ParSeq framework. The XAS pipeline itself is a relatively lightweight package of less than 2500 lines; it includes all standard EXAFS analysis steps up to EXAFS FT/BFT with atomic shell fitting.

I plan to submit also conference proceedings Yes

Primary author

Konstantin Klementiev (MAX IV Laboratory)

Presentation materials