FIDIUM Topical Area 2



Einladungslink Meeting-ID 628 6696 0791 Kenncode 215516

- Kilian Schwarz
- Daniel Schindler
- Raphael Kleinemuehl
- Manuel Giffels
- Andre Brinkmann
- Daniel Savoiu
- Saidev Polisetty
- Johannes Lange
- Grigory Koziov
- Guenter Duckeck
- Andreas Redelbach
- Maximilian Horzela
- Marc Vef

remark at the beginning: please make zoom link in future clickable


TOP0: introduction

KS presented FIDIUM news and possible synergies with PUNCH TA2, Erum-Data, and DMA
Possibility to work with PUNCH TA2 on overlay batch system and data lake testbeds, testing of Rucio, consistent hashing, and workflow management

Mainz presented problem to get Visa for Chinese student and corresponding delay in hiring 
Mainz tries to align FIDIUM work with tasks of existing PhD students
DESY as associated partner can only rely on in kind contribution
TODO: Check whether “kostenneutrale Verlaengerung” of FIDIUM is possible


Discussion on role of tools like DIRAC, RUCIO, XCache, Xrd … and their interplay
Invest whether Rucio can be made aware of Xrd-based infrastructure
Presentation of simulation environment to understand decisions on architecture level of data lake design
TODO: Build workout group to finalise data lake design and to use this as an input for simulation environment, define cycle to refine architecture (interest from KIT, Mainz, Desy, Goettingen, LMU)


Activities on DL Caching and building DL prototypes
Offers high bandwidth WAN between GSI and University Frankfurt
TODO: Request for regular WP meetings


No funding received but provides XCache servers 
Cooperation with Hamburg and tests on DESY NAF planned

Focuses on DL caching technologies and integration of dynamic data caches
Grid data transfer benchmarks on DESY NAF related to HEP workflows at PHYSnet to get baseline and to study effect of caching
Possible collaboration with DESY
TODO: Include CPU utilisation when running analysis jobs (on top of data transfers)



Works on integration of data lakes and is connecting external resources to Goettingen systems
Study of common access mechanisms to data lake and needs guarantees for accessing data
==> fits into data lake design breakout groups



Started hiring late, first works in TA1/3

TODO: Will prepare breakout meeting for WP1 on “what data do we want to accumulate”, as input is required for monitoring infrastructure
Closer coordination with GSI necessary
TODO: WP 1 meetings


We have to talk more often to each other
TODO: Concepts are breakout groups and WG meetings on WP level and TB2 meetings on 6 months level


Summary Todos

The following groups will organise monthly WP meetings
WP 1: Wuppertal
WP 2: Mainz


A breakout session on data lake architectures needs to be organised

AB and KS will provide Minutes and working plans including a mapping of milestones and responsible persons 
AB and KS will also define highlights from this Arbeitsbereich 
AB and KS will check with FIDIUM board, whether “kostenneutrale Verlaengerung” is possible

AB and KS  will organise next TB2 meeting in about 6 months




There are minutes attached to this event. Show them.