Lesson 164 — The Detection Generated Too Many False Positives
The detection from Lesson 163 works. Unfortunately, it works a little too enthusiastically. Analysts are seeing repeated matches from software deployment, support scripts and legitimate administration mixed in with the behaviour the rule was designed to find.
This is where detection engineering becomes operational. The answer is not to keep adding exclusions until the alerts disappear. We need to understand exactly what is creating the noise, isolate repeatable benign patterns, preserve unusual outliers and tune the detection without removing its original security value.
Your detection problem
The rule is producing too many results. Before changing it, identify which users, devices, parent processes and command patterns are responsible for the noise.
Detection briefing
Detection objective
Use KQL to analyse a noisy behavioural detection, identify the recurring activity responsible for most matches, distinguish stable benign patterns from suspicious outliers, apply narrowly scoped tuning and compare the results before and after the change.
Detection engineer's rule
Investigate the false positive before suppressing it. If you cannot explain why an event is benign, you do not yet have enough evidence to exclude it safely.
Stage 1 — measure the noise before changing anything
Run the current detection logic over a useful historical period and summarise its volume. We want to know whether the noise is distributed across the environment or concentrated in a few repeatable sources.
DeviceProcessEvents
| where Timestamp > ago(7d)
| where FileName in~ ("powershell.exe", "pwsh.exe")
| where InitiatingProcessFileName in~ (
"winword.exe", "excel.exe", "outlook.exe",
"chrome.exe", "msedge.exe", "firefox.exe",
"wscript.exe", "cscript.exe", "mshta.exe"
)
| extend EncodedCommand =
ProcessCommandLine has_any ("-enc", "-encodedcommand")
| extend DownloadBehaviour =
ProcessCommandLine has_any (
"Invoke-WebRequest", "DownloadString",
"WebClient", "Start-BitsTransfer"
)
| where EncodedCommand or DownloadBehaviour
| summarize Matches=count(),
Devices=dcount(DeviceName),
Users=dcount(AccountName)
by bin(Timestamp, 1d)
| order by Timestamp ascVolume is only the first clue
A detection producing 500 matches is not automatically unusable. If 480 come from one well-understood management process, the problem may be highly concentrated and therefore tunable.
Look for changes over time
A sudden increase in matches can indicate a new software rollout, configuration change or genuine attack activity. Do not assume every spike is merely “more false positives”.
Stage 2 — identify the patterns generating most matches
Break the results down by account, initiating process and command line. This reveals whether a small number of repeatable patterns dominate the alert queue.
DeviceProcessEvents
| where Timestamp > ago(7d)
| where FileName in~ ("powershell.exe", "pwsh.exe")
| where InitiatingProcessFileName in~ (
"winword.exe", "excel.exe", "outlook.exe",
"chrome.exe", "msedge.exe", "firefox.exe",
"wscript.exe", "cscript.exe", "mshta.exe"
)
| extend EncodedCommand =
ProcessCommandLine has_any ("-enc", "-encodedcommand")
| extend DownloadBehaviour =
ProcessCommandLine has_any (
"Invoke-WebRequest", "DownloadString",
"WebClient", "Start-BitsTransfer"
)
| where EncodedCommand or DownloadBehaviour
| summarize Matches=count(),
Devices=dcount(DeviceName),
FirstSeen=min(Timestamp),
LastSeen=max(Timestamp)
by AccountName,
InitiatingProcessFileName,
ProcessCommandLine
| order by Matches descRepetition can expose benign automation
A service account running an identical approved command on hundreds of endpoints may explain much of the noise. Validate that activity with the relevant system owner before tuning it.
Do not ignore the bottom of the list
The most frequent pattern may be benign while a single rare command near the bottom represents the attack behaviour the detection was intended to find.
Stage 3 — separate the common pattern from the outliers
Instead of immediately excluding the dominant command, compare it with other commands generated by the same account and parent process. Outliers deserve special attention.
let DetectionEvents =
DeviceProcessEvents
| where Timestamp > ago(14d)
| where FileName in~ ("powershell.exe", "pwsh.exe")
| where InitiatingProcessFileName in~ (
"winword.exe", "excel.exe", "outlook.exe",
"chrome.exe", "msedge.exe", "firefox.exe",
"wscript.exe", "cscript.exe", "mshta.exe"
)
| where ProcessCommandLine has_any (
"-enc", "-encodedcommand",
"Invoke-WebRequest", "DownloadString",
"WebClient", "Start-BitsTransfer"
);
DetectionEvents
| summarize CommandCount=count(),
Devices=dcount(DeviceName)
by AccountName, ProcessCommandLine
| where CommandCount <= 2
| order by CommandCount ascNoise can hide the signal
A compromised administrative or service account can perform one malicious command among thousands of legitimate executions. Excluding the account entirely could remove exactly the event you need to see.
Tune the pattern, not the identity
Where possible, define the validated combination of account, parent, path, signer or command characteristics rather than broadly trusting an account forever.
Stage 4 — apply a narrow, explainable exclusion
Suppose investigation confirms that one deployment account runs a specific approved script through a known parent process. Exclude that combination rather than suppressing every event from the account.
DeviceProcessEvents
| where Timestamp > ago(1d)
| where FileName in~ ("powershell.exe", "pwsh.exe")
| where InitiatingProcessFileName in~ (
"winword.exe", "excel.exe", "outlook.exe",
"chrome.exe", "msedge.exe", "firefox.exe",
"wscript.exe", "cscript.exe", "mshta.exe"
)
| where ProcessCommandLine has_any (
"-enc", "-encodedcommand",
"Invoke-WebRequest", "DownloadString",
"WebClient", "Start-BitsTransfer"
)
| where not(
AccountName =~ "svc-deployment"
and InitiatingProcessFileName =~ "wscript.exe"
and ProcessCommandLine has
@"C:\Company\Scripts\ApprovedDeployment.ps1"
)
| project Timestamp, DeviceId, DeviceName,
AccountName, InitiatingProcessFileName,
ProcessCommandLine, SHA1The exclusion should tell a story
A reviewer should be able to read the exclusion and understand exactly which validated activity it represents. Broad exclusions are easier to write but much harder to defend.
Exceptions need ownership
Document who validated the benign behaviour, when it was reviewed and what would cause the exception to be reconsidered.
Stage 5 — compare before and after tuning
Measure the original and tuned result sets side by side. The goal is to quantify the noise reduction while confirming that suspicious variants still survive the tuning.
let Base =
DeviceProcessEvents
| where Timestamp > ago(7d)
| where FileName in~ ("powershell.exe", "pwsh.exe")
| where InitiatingProcessFileName in~ (
"winword.exe", "excel.exe", "outlook.exe",
"chrome.exe", "msedge.exe", "firefox.exe",
"wscript.exe", "cscript.exe", "mshta.exe"
)
| where ProcessCommandLine has_any (
"-enc", "-encodedcommand",
"Invoke-WebRequest", "DownloadString",
"WebClient", "Start-BitsTransfer"
);
let Original = Base | summarize Matches=count()
| extend Version="Original";
let Tuned = Base
| where not(
AccountName =~ "svc-deployment"
and InitiatingProcessFileName =~ "wscript.exe"
and ProcessCommandLine has
@"C:\Company\Scripts\ApprovedDeployment.ps1"
)
| summarize Matches=count()
| extend Version="Tuned";
union Original, Tuned
| project Version, MatchesLower volume is not enough
A 95% reduction sounds impressive, but it is only successful if the remaining logic still detects the behaviours the rule was created to identify.
Retest known suspicious cases
After every material tuning change, replay or re-query representative malicious or suspicious examples where possible. Detection tuning should have regression testing just like other engineering work.
Agent Foskett's false-positive tuning workflow
Good tuning versus dangerous tuning
| Tuning decision | Risk | Better approach |
|---|---|---|
| Exclude an entire service account | Compromise of that account becomes invisible. | Exclude only its validated command and execution context. |
| Exclude all PowerShell from a device group | Removes a broad attack surface from visibility. | Identify the specific legitimate process or script. |
| Remove encoded-command detection | Destroys part of the original behavioural coverage. | Combine encoded execution with context and other signals. |
| Suppress the most common exact benign pattern | Lower risk when independently validated. | Document, monitor and periodically review the exception. |
| Retest after tuning | Helps reveal accidental loss of coverage. | Make regression testing part of every material change. |
Write the tuning decision like a detection engineer
Example: Historical analysis showed that the majority of detection matches were generated by the svc-deployment account executing a consistent approved deployment script through wscript.exe. The activity was validated as legitimate enterprise automation. Rather than excluding the service account globally, the detection was tuned only for the validated combination of account, initiating process and approved script path. Low-frequency command variants associated with the same account remain visible for investigation. Pre- and post-tuning volumes were compared, and representative suspicious PowerShell patterns were retested to confirm that the original behavioural coverage remained effective.
Lesson 164 key takeaways
- Measure false-positive volume before changing detection logic.
- Identify which users, devices, parent processes and commands generate most matches.
- Validate recurring benign activity before excluding it.
- Frequent patterns may be benign while rare outliers remain suspicious.
- A trusted or approved account can still be compromised.
- Prefer narrow contextual exclusions over broad account or device exclusions.
- Document why every important exclusion exists.
- Compare detection volume before and after tuning.
- Retest representative suspicious behaviour after material changes.
- Successful tuning reduces unnecessary alerts without destroying detection coverage.
Module 14 — the next problem is the threshold
Lesson 164 reduced false positives by understanding the environment instead of blindly suppressing alerts. Next, we deal with another common detection-engineering mistake: choosing a threshold before establishing what normal activity actually looks like.
Continue your KQL investigation training
🔎 KQL Academy — Module 14: Advanced Detection Engineering & Proactive Threat Hunting
Reduce false positives in KQL detections
Lesson 164 of the Agent Foskett KQL Academy shows how to investigate noisy Microsoft Defender detection logic, identify recurring benign patterns and tune false positives without removing the suspicious behaviour the rule was designed to detect.
Tune behavioural detections without creating blind spots
Learn how to measure detection volume, identify dominant patterns, preserve suspicious outliers, create narrow exclusions and compare pre- and post-tuning results with KQL.
