CyArt Academy
Unlock ProSign in
CYART // SCHEMA · EDR-TELEMETRY7 NODES · 7 LINKSEVENTSVERDICTCOMMANDACTOR · Endpoints — 50k agents, 1 kB/s eachACTOR01Endpoints50k agents, 1 kB/s eachEDGE · Ingest gateway — mTLS, per-agent tokenEDGE02Ingest gatewaymTLS, per-agent tokenSTREAM · Buffer — Kafka, 72 h retentionSTREAM03BufferKafka, 72 h retentionSERVICE · Real-time rules — Sigma, 1 s budgetSERVICE04Real-time rulesSigma, 1 s budgetSERVICE · Batch analytics — hunting, baselinesSERVICE05Batch analyticshunting, baselinesEDGE · Response — isolate host, kill procEDGE06Responseisolate host, kill procDATA · Data lake — Parquet, 1 yDATA07Data lakeParquet, 1 y

Design an EDR telemetry pipeline

Design the backend for 50,000 endpoint agents: event collection at 1 kB/s each, process-tree reconstruction, real-time rules with a one-second budget, a response channel that can isolate a host, and a data lake for hunting, all without the agent becoming the attacker's best tool.

● Hard🔒 PRODesign problem·19 min read·Intermediate → Staff
EDRagentsstreamingresponsemTLS

01Understand the problem

✓What you will learn
  • Design an agent to cloud channel that is authenticated both ways
  • Reconstruct process trees from unordered events, despite PID reuse
  • Run per-host and cross-host streaming rules within one second
  • Build a response channel that cannot be abused to own the fleet
◎Interview radar

What they probe: how the agent authenticates, how you avoid the response channel becoming a remote-code-execution backdoor, how you order events per host, what happens to detection when the backend is down. Where candidates slip: one shared API key in the agent, commands over plain HTTP, rules that need global state, process trees keyed by PID alone, no offline behaviour.

EDR (endpoint detection and response) means an agent on every laptop and server that records what happens (process starts, file writes, network connections), a backend that detects attacks in that stream, and a path for analysts to act on a host. This lesson designs the backend and the channel. The sensor itself, how it hooks the kernel and keeps its overhead low, is the subject of the runtime security agent fleet lesson.

Functional requirements

  1. Agents send process, file, network, registry and authentication events; the backend stores them and makes them searchable.
  2. The backend runs detections per host and across hosts; high-fidelity rules alert within a second.
  3. Analysts can isolate a host, kill a process and pull a file through the backend.
  4. Hunting over 90 days of data with interactive queries.

Non-functional requirements

50k endpoints at 1 kB/s average with 20x bursts on a busy host; p95 event-to-alert under 1 s for real-time rules; agents keep protecting with the backend unreachable; the response channel is unforgeable even by someone who controls part of the backend; no tier-one event (process start, credential access) is lost silently.

This is an advanced module

Design problems: Telemetry & detection is part of the Pro pass. 6 more sections — the deep dives, the final design, follow-ups and the test — unlock with it.

Enter access codeSee Pro pass

Already a cyart.io or studio.cyart.io member? Your pass code is in your Studio account.

Related lessons