The triage loop
A repeatable loop for every alert: validate, scope, contain, collect, decide. How to validate on raw events, pick the smallest containment, write a case note others can act on, and avoid the two classic failures: closing too early and chasing everything.
01Five steps, every time
- Run the validate, scope, contain, collect, decide loop
- Validate on raw events, not the alert summary
- Choose the smallest reversible containment for each entity
- Separate true positive from benign true positive and write a case note that survives handover
- Validate: is the alert describing a real event? Open the raw events, not the alert summary. Rule bugs and parser drift produce confident nonsense.
- Scope: one user or many? One host or a fleet? Pivot on IP, hash, session, and time window. Write down what you checked.
- Contain (if real and spreading): the smallest reversible action that stops the bleeding. Revoke a session, isolate a host, disable a key. Not wipe, not delete.
- Collect: export evidence to the case before it rotates: logs, process trees, the file, a memory snapshot if the host matters.
- Decide: true positive → incident or ticket; benign true positive (the behaviour happened, an admin did it) → document and tune; false positive → fix the rule.
- ACTOR
- SERVICE
- DATA
- Live flow
Closing too early: a single explainable line is not a disproof of the sequence. Chasing everything: define scope boundaries (this user, this day, these three systems) before pivoting, and park anything outside them as a new lead.
02Validate on raw events
An alert is a claim made by a rule. The rule may be right, may be fed broken data, or may be matching something it was never meant to. Open the underlying events and check that they say what the alert says.
ALERT Suspicious PowerShell download cradle host=FIN-LT-042 user=CYART\m.okafor sev=high
raw 4688 2026-11-13T09:41:12Z FIN-LT-042
NewProcessName: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
ParentProcessName: C:\Program Files\Microsoft Office\root\Office16\EXCEL.EXE
CommandLine: powershell -nop -w hidden -enc SQBFAFgAIAAoAE4AZQB3AC0ATwBi...Here the raw event is worse than the alert. Excel spawning hidden, encoded PowerShell is a macro dropper pattern. Decode the base64 (it is UTF-16LE) before you decide; IEX (New-Object Net.WebClient)... confirms a download cradle.
| Check | Why |
|---|---|
| Do the raw events exist for that host and time? | Rules on stale or replayed data fire on ghosts |
| Does the parsed field match the raw text? | Parser drift puts the user name in the host field and the rule fires on nonsense |
| Is the parent process what you expect? | Same command, different parent: an admin's terminal versus Excel is the whole verdict |
| Has this exact alert fired before on this entity? | History tells you whether it is a known benign pattern or new |
An alert claims a login from North Korea. The raw event shows the IP is in your own VPN egress range. Most likely explanation?
03Scope, then contain the smallest thing
Scope by pivoting on entities: users, hosts, IPs, hashes, domains, sessions. For each pivot write what you searched, the window, and the result, including "none". That list is what lets someone else pick the case up.
| Entity | Smallest reversible containment | Heavier option |
|---|---|---|
| User / session | Revoke sessions and refresh tokens, force re-auth | Disable account, reset password, remove MFA methods |
| Host | EDR network isolation (keeps agent link) | Power off (loses memory evidence) |
| Cloud key | Deactivate the access key | Delete key, rotate all secrets it could read |
| Domain / IP | Block at proxy and DNS | Block at perimeter firewall for all protocols |
| Soft-delete the message from mailboxes | Block sender domain at gateway |
Resetting one password while the attacker holds three other accounts tells them they are seen. When scope is unclear and the attacker is active, coordinate containment across all known footholds at once.
04Decide, document, hand over
| Outcome | Meaning | Action |
|---|---|---|
| True positive | Malicious and real | Incident; follow the runbook |
| Benign true positive | Detection fired correctly on authorised activity | Close with reference; add an allow-list keyed to the approval, not a rule disable |
| False positive | Rule logic or data is wrong | Fix the rule; add the sample to its negative tests |
CASE 4412 FIN-LT-042 macro dropper status: escalated to IR sev: high
What: EXCEL.EXE -> powershell -enc (download cradle to hxxp://cdn-update[.]top/a.ps1)
Validated: raw 4688 + EDR process tree; decoded payload attached
Scope: hash seen on 1 host; domain resolved by 3 hosts (FIN-LT-042, FIN-LT-017, HR-LT-003)
mail: 14 recipients of invoice_Nov.xlsm, 2 opened (searched 7d, all mailboxes)
Contain: FIN-LT-042 isolated 09:58Z; domain blocked at proxy + DNS 10:02Z
Evidence: memory image FIN-LT-042, proxy logs 08:00-11:00Z exported to case
Next: check FIN-LT-017 and HR-LT-003 process trees; purge mail; reset m.okafor"I validate on raw events, scope by pivoting on the entities, contain with the smallest reversible action, collect before anything changes, and classify as true, benign-true or false positive, because each of those has a different fix."
Q1When do you escalate from alert to incident?+
When it is a true positive with impact beyond one contained entity: multiple hosts or users, privileged credentials, data access, or anything with legal or customer notification implications. Escalating early is cheap; late is expensive.
Q2How do you stop the same benign alert every week?+
Tune with a narrow, documented exception tied to the approval (this account, this host, this change ticket), with an expiry. Never disable the rule outright.
Q3What makes a good case note?+
Facts with times in UTC, what was searched and found (including nothing), actions taken and when, and the explicit next steps. Opinions labelled as hypotheses.
An alert says a developer ran `curl | bash` on a build host. The developer confirms they did it to install a tool. Classification?