Tech Workshop: Mastering Open Source Monitoring for Robust Hardware and Data Solutions

Tuesday 12 Dec 2023, 09:15 → 13:00 Europe/Berlin

FLASH (Gebäude 28c, 2.OG)

This is a half day meeting of people who are to solve hardware and software development issues.

The purpose of this workshop is to empower attendees with the knowledge and skills to effectively utilize open-source tools for comprehensive hardware monitoring and data acquisition.

• Practical Insight: Gain hands-on experience from seasoned professionals on managing network devices, servers and applications.
• Custom Solutions: Learn how to tailor data collection and analysis to fit the specific needs of your hardware and software projects.
• Security Strategies: Discover strategies for maintaining robust security protocols within open-source environments.
• Quality Assurance: Explore how to uphold high standards of quality and performance in open-source systems while accommodating a range of hardware variations.
• Resource Optimization: Understand how to organize and represent service dependencies for better resource management.
• Efficiency and Automation: Explore how to streamline development pipelines and automate monitoring tasks to enhance efficiency.
• Problem-Solving: Get direct advice and problem-solving strategies for your project-specific issues during the Q&A sessions.
• Community Engagement: Learn the significance of community involvement in open-source projects and how it shapes the evolution of the product.
• Networking: Connect with beamline managers, scientists, software developers, and IT professionals to expand your professional network.

The workshop is designed to give attendees an understanding of the strategies behind successful open-source project maintenance and knowledge applicable to their own development endeavors.

The idea is to meet colleagues doing open-source for monitoring hardware and data acquisition. Topics will range from data collection and analysis, data observability, security, configuration and administration. It's also a great chance to ask questions and get an idea of how to solve issues related to your project.

Program will consist of two parts and a time to discuss. 

---

There will also be a short introduction of OpenSource.Science - community at the forefront of advancing scientific research through open-source innovation. OpenSource.Science's mission to expedite scientific progress. It is a new NumFOCUS initiative - launched in July 2022 in partnership with IBM – that aims to accelerate scientific research and discovery by improving the ways open-source scientific software gets done. OSSci connects researchers, open source developers and other stakeholders to share best practices, identify common pain points, and explore solutions together.

---

We will meet in the FLASH (9:15am to 1pm). Coffee and lunch will be provided.

We use Indico and ask you to register to help track numbers and plan the meeting. 

For remote access, please, use the following

Zoom link: https://desy.zoom.us/j/63578608966

Meeting-ID: 635 7860 8966

Passcode: will be sent to your email a day before the workshop (you should be registered to receive this email)

In case of any questions, please, feel free to ask Olga Merkulova: olga.merkulova@desy.de

For you convenience, please, find map below.

Screenshot_20231121_145721.png

09:15 → 10:30 Presentation of ways of monitoring. What can be monitored and how this data can be presented.

Developers of a multi-platform open-source network management system - NetXMS - are invited to give a workshop. This will be divided into 2 parts: presentation (covers UX and requirements capture in case of a big amount of hardware) and implementation (covers software development issues).
They will also share their knowledge of hardware monitoring and data acquisition, of how to effectively support open source projects for more than 15 years, to create beneficial and easy development pipelines, to provide security in open-source solutions, to deploy in a different hardware variations and share their software development approaches and answer our questions as they are working in the same field.

First part might be of the most interest for beamline managers, beamline scientists and DESYer doing software development.

10:30 → 11:00 Coffee break

11:00 → 12:30 Software related topics

The second part is more related to software itself and might be beneficial for software engineers and IT related developers.

Speakers are professional IT project managers and software engineers.
They will cover topics of:
Monitoring of network devices, servers and applications from one management server
Data collection from SNMP-capable platforms
Configuration, administration and monitoring functions via multiplatform (Eclipse-based) and Web GUIs
Ability to send e-mails and SMS notifications or execute external programs as a reaction to any event
Organization of monitored objects into hierarchical structure to represent service dependencies
Centralized remote observability agent upgrades
OSI Layer 2 and Layer 3 IP topology automatic discovery
Portable client libraries
Flexible policy-based event processing (including correlation rules)
Remote access
Flexible access control configuration
Infrastructure usage (services, optimal usage of central IT capabilities)
Community/user needs and expectations, including ones that are not currently met
Built-in scripting engines for advanced automation and management

12:30 → 13:00 Q&A and Lunch time