CyArt Academy
Unlock ProSign in
CYART // SCHEMA · EDR-TELEMETRY7 NODES · 7 LINKSEVENTSVERDICTCOMMANDACTOR · Endpoints — 50k agents, 1 kB/s eachACTOR01Endpoints50k agents, 1 kB/s eachEDGE · Ingest gateway — mTLS, per-agent tokenEDGE02Ingest gatewaymTLS, per-agent tokenSTREAM · Buffer — Kafka, 72 h retentionSTREAM03BufferKafka, 72 h retentionSERVICE · Real-time rules — Sigma, 1 s budgetSERVICE04Real-time rulesSigma, 1 s budgetSERVICE · Batch analytics — hunting, baselinesSERVICE05Batch analyticshunting, baselinesEDGE · Response — isolate host, kill procEDGE06Responseisolate host, kill procDATA · Data lake — Parquet, 1 yDATA07Data lakeParquet, 1 y

Design a runtime security agent fleet

Design the eBPF sensor that runs on 100,000 Linux nodes: filter kernel events in the kernel so a busy host does not drown the agent, survive bursts with ring buffers and honest drop accounting, distribute signed policy, and upgrade the fleet with canaries and a kill switch inside a 2 % CPU budget.

● Hard🔒 PRODesign problem·18 min read·Advanced → Staff
eBPFruntime security agentin-kernel filteringBPF ring buffercanary rolloutkill switch

01Understand the problem

✓What you will learn
  • Explain what an eBPF sensor is, where it hooks and why the verifier makes it safer than a kernel module
  • Filter in the kernel with BPF maps and size the ring buffer for bursts
  • Design backpressure that degrades by priority and counts every drop
  • Distribute signed policy and roll out agents with canaries, health gates and a kill switch inside an overhead budget
◎Interview radar

What they probe: where the event volume is cut, what happens when user space cannot keep up, how 100k agents get a new policy in minutes without a thundering herd, and how you avoid bricking the fleet with one update. Where candidates slip: shipping every syscall to the backend, blocking in the kernel, treating config pushes as harmless, no way to turn the agent off when the agent is the problem.

eBPF lets you load small programs into the Linux kernel and attach them to hooks: tracepoints (stable instrumentation points such as sched_process_exec), kprobes (any kernel function, unstable across versions) and BPF LSM hooks (the Linux Security Module points where the kernel decides whether to allow an operation, kernel 5.7+). The in-kernel verifier checks every program before it runs: bounded loops, safe memory access, no arbitrary writes. A buggy program is rejected at load time instead of crashing the host, as a buggy kernel module would. This lesson designs the fleet; storing the telemetry is the EDR pipeline problem.

Functional requirements

  1. Capture process exec, file access, network connect and accept, privilege changes, with container context (pod, namespace, image).
  2. Run detections locally (reverse shell, crypto miner, write to /etc/passwd in a container) and optionally enforce through an LSM hook.
  3. Stream curated events to the backend; receive policy (rules, filters, enforcement modes) from a control plane.
  4. Upgrade binaries and content safely, and turn the agent off fleet-wide or per group in under five minutes.

Non-functional requirements

  1. 100,000 nodes, kernels 5.4 to 6.x, x86 and arm64.
  2. Overhead budget ≤ 2 % CPU and ≤ 300 MB memory at p99; never hang or panic a node.
  3. Exec and network events: no silent loss; drops are counted and reported. Event to backend under 10 s.
  4. A policy change reaches 95 % of the fleet within 5 minutes.
▭On the whiteboard · Scope3 points
01
FR
exec, file, net, privilege + container context | local detect, optional enforce | stream | policy | safe upgrade + off switch
02
NFR
100k nodes | ≤ 2 % CPU, ≤ 300 MB | counted drops | 5 min policy
03
out
backend storage and alert UI (EDR pipeline problem), Windows

This is an advanced module

Design problems: cloud & runtime security is part of the Pro pass. 4 more sections — the deep dives, the final design, follow-ups and the test — unlock with it.

Enter access codeSee Pro pass

Already a cyart.io or studio.cyart.io member? Your pass code is in your Studio account.

Related lessons