The pipeline components described below reflect an AWS deployment. Other clouds use equivalent services — for example, AKS on Azure or ACK on Ali Cloud instead of ECS/EKS, and Event Hubs or Message Queue for Apache Kafka instead of MSK.
1
Collector
Produces events and sends them in batches to the ingest endpoint.
2
Ingest service
The endpoint resolves via PrivateLink to an internal NLB fronting an ECS Fargate service in your VPC. Each event is published to Amazon MSK (Kafka) for buffering.
3
Enrichment
An EKS deployment consumes the MSK topic, enriches events, and writes normalized data into ClickHouse.
4
Classification
A dedicated classification workload scores events and evaluates model drift against historical data in ClickHouse.
5
Dashboard and query service
The dashboard (frontend, backend, and query service) runs on ECS Fargate behind an internal ALB, providing the investigation UI, API, and natural-language query interface.