Lesson 86 — The Threshold Hid a Real Attack
The tuning worked.
Alert volume dropped dramatically.
Then a real attack was reviewed.
It never crossed the new threshold.
Agent Foskett looked at the rule again.
Quiet can be dangerous.
A lower alert count means little if malicious behaviour has fallen below the detection boundary.
Case briefing
Investigation objective
Understand how threshold tuning can create false negatives, reproduce the missed malicious behaviour and redesign the detection so operational noise is reduced without silently removing important attack coverage.
Investigator's rule
Every tuning decision should ask two questions: what noise disappears — and what attack behaviour disappears with it?
Stage 1 — reconstruct the tuning decision
| Before | After |
|---|---|
| Threshold > 10 | Threshold > 20 |
| High daily volume | Much lower daily volume |
| Known noisy workflows matched | Many noisy workflows disappeared |
| Known attack matched | New attack at 18 events did not match |
The dashboard looked better
The visible operational metric improved: fewer alerts. But the security metric — whether meaningful malicious behaviour remained detectable — was not tested broadly enough.
A threshold is a boundary
Changing the number moves the boundary between activity the rule can see and activity it ignores. That is a security decision, not merely a volume adjustment.
Stage 2 — reproduce the missed attack
DeviceProcessEvents
| where Timestamp > ago(30d)
| where FileName in~ ("powershell.exe", "pwsh.exe")
| summarize EventCount=count()
by DeviceId, DeviceName, AccountName,
bin(Timestamp, 5m)
| where EventCount between (11 .. 20)
| order by EventCount descSearch the blind band
If the threshold moved from greater than 10 to greater than 20, activity between those boundaries deserves deliberate review. That is the behaviour the tuning stopped surfacing.
Do not inspect only the missed incident
The known attack proves a blind spot exists. Historical telemetry can show whether other suspicious activity also fell into the newly ignored range.
Stage 3 — compare benign and malicious behaviour
| Activity | Events / 5 min | Context |
|---|---|---|
| Support automation | 16 | Expected management workflow |
| Developer script | 19 | Known development activity |
| Finance workstation attack | 18 | Unexpected PowerShell chain |
| Deployment host | 27 | Expected high-frequency administration |
The count alone cannot separate them
The malicious case sits inside the same numeric range as legitimate activity. Raising the threshold therefore removes both benign noise and real attack visibility.
This is where context matters
When benign and malicious counts overlap, detection quality may need to come from process lineage, device role, account context, command behaviour or other evidence — not a larger number.
Stage 4 — understand the false-negative cost
False negatives are harder to see
False positives arrive in the queue and complain loudly. False negatives are quiet. You may only discover them through another alert, a hunt, incident review or external evidence.
Silence is not proof of quality
A detection with almost no alerts may be exceptionally precise — or it may simply be unable to see the behaviour you care about.
Stage 5 — test candidate thresholds against known cases
| Candidate | Benign volume | Attack A | Attack B |
|---|---|---|---|
| > 10 | High | Match | Match |
| > 15 | Medium | Match | Match |
| > 20 | Low | Match | Miss |
| > 10 + context | Potentially lower | Match | Match |
Use multiple malicious examples
One known attack may be unusually loud. Testing several historical cases helps expose whether tuning has overfit the rule to the easiest example.
Threshold plus context may outperform threshold alone
If legitimate and malicious activity overlap numerically, additional evidence can sometimes improve precision without moving the threshold beyond quieter attacks.
Stage 6 — add context instead of simply raising the number
DeviceProcessEvents
| where Timestamp > ago(30d)
| where FileName in~ ("powershell.exe", "pwsh.exe")
| where InitiatingProcessFileName in~ (
"msedge.exe", "chrome.exe", "firefox.exe"
)
| summarize EventCount=count()
by DeviceId, DeviceName, AccountName,
bin(Timestamp, 5m)
| where EventCount > 10Context should come from the hypothesis
Do not add random conditions merely to shrink the result set. Each condition should represent evidence that helps distinguish the suspicious behaviour being detected.
Be careful with brittle context
Overly specific filenames, paths or strings may reduce noise while making the detection easy to evade. Preserve the behaviour wherever possible.
Stage 7 — backtest the revised candidate
Re-run the full test
Do not assume the new contextual condition fixes the problem because it catches the missed incident. Repeat the historical volume and coverage checks from Lessons 83 and 84.
Look for new blind spots
A contextual filter can create a different false-negative boundary. For example, browser-initiated PowerShell logic will not detect the same behaviour initiated through another process chain.
Stage 8 — document the trade-off
| Decision | Document |
|---|---|
| Threshold retained or changed | Evidence supporting the number |
| Context added | Why it represents the security hypothesis |
| Noise reduced | Measured before/after volume |
| Coverage preserved | Known malicious cases retested |
| Remaining blind spots | Behaviour the rule still cannot detect |
Make the limitation visible
Analysts and future detection engineers need to know what the rule does not cover. Undocumented blind spots become dangerous assumptions.
Tuning needs a security rationale
“Reduced alerts by 80%” is incomplete. The decision should also state what happened to malicious coverage and which risks remain.
Stage 9 — write the engineering finding
The problem was not tuning itself
The problem was tuning against visible noise without sufficiently measuring the invisible coverage cost.
Detection engineering balances two failure modes
Too much noise can overwhelm analysts. Too little sensitivity can hide attacks. Good engineering makes that trade-off explicit and testable.
Stage 10 — make the final decision
A quiet queue is not the objective
The objective is useful detection coverage at an operationally sustainable workload.
The next engineering problem
Even when coverage is preserved, the SOC may discover that another rule already detects the same behaviour. Lesson 87 examines redundant detections and duplicate analyst work.
Lesson 86 key takeaways
- Every threshold creates a detection boundary.
- Raising a threshold can remove malicious activity as well as benign noise.
- False negatives are often less visible than false positives.
- Search the historical range removed by a tuning change.
- Compare benign and malicious behaviour around the threshold boundary.
- Do not assume event count alone can distinguish intent.
- Test multiple known malicious examples, not just the loudest incident.
- Use security-relevant context when benign and malicious volumes overlap.
- Do not add arbitrary filters simply to reduce results.
- Backtest again after every meaningful tuning change.
- Document both the noise reduction and the coverage cost.
- Make remaining blind spots explicit.
- The goal is sustainable analyst workload without sacrificing important attack visibility.
Module 9 — Detection Engineering: Turning Findings into Protection
Lesson 86 exposed the false-negative cost of aggressive threshold tuning and restored lower-volume attack coverage using behavioural context. Lesson 87 will examine what happens when the detection works — but another rule is already detecting the same thing.
Continue your SOC Analyst training
🔎 SOC Analyst Academy — Module 9: Detection Engineering: Turning Findings into Protection
Can detection tuning cause false negatives?
Lesson 86 of the Agent Foskett SOC Analyst Academy teaches analysts how raising detection thresholds can reduce alert noise while also hiding genuine malicious activity, and how to measure that false-negative cost.
Balancing false positives and false negatives in detection engineering
Learn how to reproduce missed attacks, examine the historical blind band created by threshold changes, add security-relevant behavioural context and backtest revised detection logic before production.
