Deleria 

The Deleria project (Distributed event-level experimental readout and inline analysis) is a software-based streaming system which aims to provide real time data analysis from instruments with large computing requirements and data volumes. Unlike traditional experimental workflows that operate at the file level, Deleria workflows operate at the event level allowing for latencies between data collection and analysis to occur within of order seconds. This enables experimenters to make real-time decisions in configuring and optimizing their experiments - even in the face of large computing requirements.

Deleria utilizes a pipeline architecture based on GRETA (Gamma-Ray Energy Tracking Array). This system consists of a number of composable components such as forward buffers, an event processing component, event builders as well compoinents for storage and streaming to real time displays. These components can run on geographically distributed sites.

An example of a fully distributed pipeline connecting detector electronics to processing, storage and prompt analysis monitoring.

Pipeline Components:

Forward Buffer: 

• Mediates communication between detector electronics and event processing cluster

• Allows electronics to be free running and event processing nodes to self-schedule

Event Processing:

• performs event processing function for online/offline analysis

• hybrid CPU/GPU implementation

Event Building (optional):

• Aggregates events according to timestamps into global events

• Provides a tap for prompt analysis

Storage:

• Writes event stream from pipeline to persistant storage

Communication Layer:

• Pipeline components transfer data via GAP (Greta Application Protocol) layer

• Based on nanomsg - zero copy, low latency, excellent stability (https://nanomsg.org)

• GAP implements standard communication design patterns (fan-in, fan-out)