Lesson 82 — The Detection Generated Too Many False Positives
The new detection was live.
Within hours, the SOC queue was filling with legitimate matches.
“Can we just exclude them?”
Agent Foskett looked at the original hypothesis.
“Only if the exclusions don't remove the attack we're trying to detect.”
Noise is evidence too.
Every benign match teaches the team something about the environment. Tune from observed evidence, not frustration.
Case briefing
Investigation objective
Analyse false-positive patterns, identify which parts of the original security hypothesis remain valuable, and tune the implementation without silently creating dangerous coverage gaps.
Investigator's rule
Tune the implementation. Preserve the hypothesis.
Stage 1 — do not start by deleting the rule
| Reaction | Problem |
|---|---|
| Disable the rule immediately | Removes all coverage before understanding why it is noisy. |
| Exclude every frequent application | May create broad blind spots that an attacker can reuse. |
| Raise severity but keep every alert | Changes appearance, not workload. |
| Measure and classify false positives | Creates evidence for targeted tuning. |
False positives are not identical
A legitimate admin script, a signed enterprise support tool and a developer launching PowerShell from a browser may all match the same broad query for very different reasons.
Ask why each class matched
The useful question is not only “is this benign?” It is “which part of the detection logic caused this benign behaviour to look like the attack?”
Stage 2 — measure the noise before tuning
DeviceProcessEvents
| where Timestamp > ago(7d)
| where FileName in~ ("powershell.exe", "pwsh.exe")
| where InitiatingProcessFileName in~ (
"msedge.exe", "chrome.exe", "firefox.exe"
)
| summarize Events=count(),
Devices=dcount(DeviceId),
Users=dcount(AccountName)
by InitiatingProcessFileName
| order by Events descFrequency changes the engineering problem
If one benign workflow causes 80 percent of the matches, that suggests a different tuning strategy from hundreds of unrelated benign behaviours.
Measure entities, not only events
Six hundred alerts from one managed device may be easier to explain than six hundred alerts spread across hundreds of users and endpoints.
Stage 3 — classify the false positives
Known benign is stronger than assumed benign
Document what establishes legitimacy: signed executable, approved package, expected parent process, managed path, known device group or verified change process.
Unknown stays unknown
Do not label an alert false positive merely because triage found no obvious malware. Lack of proof is not the same as proof of benign intent.
Stage 4 — compare benign and malicious context
| Feature | Known attack | Approved support workflow |
|---|---|---|
| Initiating process | Browser | Browser |
| PowerShell | Present | Present |
| Command pattern | Obfuscated / unusual | Consistent approved command |
| Executable path | User-writable staging | Managed application path |
| Follow-on network | Unexpected external destination | Known enterprise service |
| Prevalence | Rare | Common across support devices |
The shared signal is too broad
Browser → PowerShell still matters, but by itself it cannot separate malicious activity from legitimate enterprise workflows.
Add discriminating context
Useful tuning usually adds evidence that differs between malicious and benign cases rather than simply excluding a whole tool or process family.
Stage 5 — prefer narrow, explainable tuning
| Broad exclusion | Safer engineering approach |
|---|---|
| Exclude every event from powershell.exe | Destroys the detection. |
| Exclude all browser-launched PowerShell on IT devices | Could create an attractive attacker blind spot. |
| Exclude a verified support workflow using stable, documented attributes | Reduces known noise while preserving wider coverage. |
| Require additional suspicious context | Can improve precision without trusting a broad allow-list. |
Every exclusion is a security decision
An exclusion says: “we are willing not to see this class of activity.” That should be justified and reviewable.
Avoid attacker-controlled exclusions
Be cautious about excluding solely on filenames, command text or paths that an attacker could easily imitate.
Stage 6 — enrich instead of immediately excluding
Correlation can improve precision
A weak individual signal can become more useful when combined with another observable behaviour such as unusual command content, file creation or network activity.
But complexity has a cost
Every additional condition may reduce noise, but it may also reduce coverage. Tuning should be tested against known malicious examples.
Stage 7 — retest against the attack
Do not celebrate alert reduction alone
A rule that goes from 600 alerts to zero may simply have been tuned into uselessness.
Precision and coverage must be considered together
The objective is not the smallest possible alert count. It is the best operational signal while retaining meaningful attack coverage.
Stage 8 — document every tuning decision
| Document | Why it matters |
|---|---|
| Observed false-positive class | Explains the source of noise. |
| Evidence that activity is legitimate | Prevents assumption-based allow-listing. |
| Logic changed | Shows exactly how the detection was modified. |
| Expected effect | Defines what improvement should occur. |
| Known coverage risk | Makes the trade-off explicit. |
| Review date / owner | Prevents permanent forgotten exclusions. |
Tuning history becomes institutional memory
Six months later, another analyst should be able to understand why the rule behaves as it does instead of reverse-engineering unexplained exclusions.
Temporary can become permanent accidentally
Time-bound exceptions and change-window tuning should have expiry or review logic where possible.
Stage 9 — measure the tuned detection
Volume is only one metric
Track analyst effort, repeated entities, escalation rate, confirmed malicious findings and the quality of evidence presented in each alert.
Analyst feedback is detection telemetry
The SOC queue tells detection engineers how the rule behaves in production. Triage outcomes should feed back into engineering decisions.
Stage 10 — write the tuning decision
The false positives taught us something
The noisy rule exposed legitimate patterns the detection engineer had not fully understood. That knowledge can make the detection better rather than simply quieter.
The quietest detection is not automatically the best detection
A useful SOC rule balances signal quality, security coverage and analyst workload. Silence is valuable only when it comes from better precision — not lost visibility.
Lesson 82 key takeaways
- Do not disable a noisy rule before understanding why it is noisy.
- Measure false positives before making exclusions.
- Group benign matches into explainable behaviour classes.
- Known benign activity should be supported by evidence.
- Unknown activity should not automatically be labelled a false positive.
- Tune the implementation while preserving the security hypothesis.
- Prefer narrow, documented exclusions over broad blind spots.
- Avoid exclusions based only on attributes an attacker can easily imitate.
- Additional context can improve precision without discarding the core signal.
- Retest tuned logic against known malicious cases.
- Alert reduction is not proof of better detection quality.
- Document the reason, risk, owner and review date for tuning decisions.
- Analyst triage outcomes should feed back into detection engineering.
Module 9 — Detection Engineering: Turning Findings into Protection
Lesson 82 tuned false-positive noise without deleting the original security hypothesis. Lesson 83 will show why thresholds should be justified by historical behaviour rather than chosen because they “feel about right.”
Continue your SOC Analyst training
🔎 SOC Analyst Academy — Module 9: Detection Engineering: Turning Findings into Protection
How do you tune false positives without weakening a detection?
Lesson 82 of the Agent Foskett SOC Analyst Academy teaches analysts how to classify false-positive patterns, measure noise and apply narrow evidence-based tuning while preserving the original security hypothesis.
False-positive tuning in Microsoft Defender XDR
Learn how to compare benign and malicious context, avoid dangerous broad exclusions, retest known malicious cases and use analyst feedback to improve production detection quality.
