Abstract:
AMPEL is a framework for streaming data analysis, written in Python, and
with a focus on time-domain astronomy. It enables users to build
analyses out of hierarchies of single-purpose units and coordinates the
execution of these units on a stream of data. AMPEL can execute multiple
independent analyses at once, de-duplicating calculations requested by
multiple users and recording the provenance of derived data. Analyses
are described in a static, YAML-based configuration language, and can be
developed and tested locally before being transferred to a cluster for
large-scale execution. While it was originally created at HU and DESY to
filter, augment, and react to ZTF alerts, it is also being used as the
basis for DESY’s LSST broker. Here we present the motivation for AMPEL
and some example applications, along with some of the tools we found
useful while building it.