Agent Foskett Academy • KQL Academy • Module 14 • Lesson 169 • Detection Engineering & Proactive Threat Hunting

Lesson 169 — The Detection Worked — But It Fired 600 Times a Day

The detection was technically correct. It found the behaviour we designed it to find. Backtesting had removed the obvious false positives, and known suspicious examples still matched.

Then it went operational — and produced roughly 600 candidate events a day. The problem was no longer whether the query detected suspicious behaviour. The problem was whether a SOC could realistically investigate what it produced.

A detection that overwhelms analysts can fail operationally even when its logic is technically correct. Detection engineering must optimise for investigation value, not merely event matching.

Control the Volume Back to KQL Academy

Agent Foskett KQL Academy operational detection tuning

Your detection problem

The rule works, but repeated events and duplicated behaviour create far more output than analysts can handle. Measure what is driving the volume, group related activity and turn hundreds of raw matches into a smaller number of investigation-ready candidates.

✓ Measure operational volume

✓ Find repeated activity

✓ Group related events

✓ Preserve high-value context

Detection briefing

WORKING DETECTION ↓ 600 RAW MATCHES PER DAY ↓ DOES 600 EVENTS = 600 INVESTIGATIONS? ↓ NO ↓ MEASURE + GROUP + CORRELATE ↓ REPEATED ACTIVITY BECOMES ONE CASE ↓ PRIORITISE HIGHER-VALUE GROUPS ↓ PRESERVE THE EVIDENCE ↓ OPERATIONALLY SUSTAINABLE DETECTION

Detection objective

Use KQL to measure how often a working detection fires, identify the behaviours creating excessive volume, aggregate related events into investigation groups, prioritise stronger combinations and produce an output that gives analysts useful context without creating an alert for every raw event.

Detection engineer's rule

Event count is not investigation count. Ten related process events on the same endpoint within a short period may describe one security story. Treating every telemetry row as a separate alert is often an engineering failure.

Stage 1 — measure the operational volume

Start with evidence. Before changing thresholds or adding exclusions, establish how much activity the candidate detection produces each day and how broadly it is distributed across devices and users.

1234567891011121314151617181920212223242526

let DetectionCandidates =
    DeviceProcessEvents
    | where Timestamp > ago(14d)
    | where FileName in~ ("powershell.exe", "pwsh.exe")
    | extend BrowserParent =
        InitiatingProcessFileName in~ (
            "chrome.exe", "msedge.exe", "firefox.exe"
        )
    | extend DownloadBehaviour =
        ProcessCommandLine has_any (
            "Invoke-WebRequest", "DownloadString",
            "WebClient", "Start-BitsTransfer"
        )
    | extend EncodedCommand =
        ProcessCommandLine has_any ("-enc", "-encodedcommand")
    | extend SignalCount =
        toint(BrowserParent)
        + toint(DownloadBehaviour)
        + toint(EncodedCommand)
    | where SignalCount >= 2;
DetectionCandidates
| summarize CandidateEvents=count(),
            Devices=dcount(DeviceId),
            Users=dcount(AccountName)
          by bin(Timestamp, 1d)
| order by Timestamp asc

Look for spikes and consistency

Six hundred matches on one unusual day is different from six hundred every day. Daily bins show whether the problem is persistent, periodic or driven by a specific operational event.

Ask what the SOC would receive

If every matching row became a separate alert, translate that into analyst workload. Detection quality includes the operational cost of triage.

Stage 2 — find what is driving the volume

Now identify repeated combinations of account, parent process and command line. High-volume detections often contain a small number of behaviours repeated many times.

123456789101112131415161718192021

let DetectionCandidates =
    DeviceProcessEvents
    | where Timestamp > ago(14d)
    | where FileName in~ ("powershell.exe", "pwsh.exe")
    | extend DownloadBehaviour =
        ProcessCommandLine has_any (
            "Invoke-WebRequest", "DownloadString",
            "WebClient", "Start-BitsTransfer"
        )
    | extend EncodedCommand =
        ProcessCommandLine has_any ("-enc", "-encodedcommand")
    | where DownloadBehaviour or EncodedCommand;
DetectionCandidates
| summarize Events=count(),
            FirstSeen=min(Timestamp),
            LastSeen=max(Timestamp),
            Devices=dcount(DeviceId)
          by AccountName,
             InitiatingProcessFileName,
             ProcessCommandLine
| order by Events desc

Frequency needs interpretation

A repeated command may be benign automation, a noisy application or persistent malicious activity. Frequency tells you where to investigate first; it does not tell you the verdict.

Do not immediately create exclusions

The objective here is to understand the shape of the detection output. Exclusions should follow validation, not simply frustration with a large result count.

Stage 3 — group events that describe the same story

Instead of generating a candidate for every event, group activity around the same device and account within a 15-minute window. This begins turning telemetry into investigation units.

123456789101112131415161718192021

let DetectionCandidates =
    DeviceProcessEvents
    | where Timestamp > ago(1d)
    | where FileName in~ ("powershell.exe", "pwsh.exe")
    | extend DownloadBehaviour =
        ProcessCommandLine has_any (
            "Invoke-WebRequest", "DownloadString",
            "WebClient", "Start-BitsTransfer"
        )
    | extend EncodedCommand =
        ProcessCommandLine has_any ("-enc", "-encodedcommand")
    | where DownloadBehaviour or EncodedCommand;
DetectionCandidates
| summarize EventCount=count(),
            FirstSeen=min(Timestamp),
            LastSeen=max(Timestamp),
            Commands=make_set(ProcessCommandLine, 5)
          by DeviceId, DeviceName,
             AccountName,
             bin(Timestamp, 15m)
| order by EventCount desc

Aggregation changes the question

We are no longer asking “how many events matched?” We are asking “how many distinct bursts of suspicious behaviour would an analyst need to investigate?”

Choose grouping keys carefully

Device, user and time are useful starting points, but different detections may need process identity, remote destination, file hash or another entity. Poor grouping can accidentally merge unrelated activity.

Stage 4 — measure raw events versus investigation groups

Now quantify the effect. Compare the number of raw matching events with the number of grouped investigation candidates. This gives us an operational metric instead of relying on intuition.

12345678910111213141516171819202122232425262728

let DetectionCandidates =
    DeviceProcessEvents
    | where Timestamp > ago(7d)
    | where FileName in~ ("powershell.exe", "pwsh.exe")
    | extend SuspiciousCommand =
        ProcessCommandLine has_any (
            "Invoke-WebRequest", "DownloadString",
            "-enc", "-encodedcommand"
        )
    | where SuspiciousCommand;
DetectionCandidates
| summarize RawEvents=count(),
            FirstSeen=min(Timestamp),
            LastSeen=max(Timestamp)
          by DeviceId, DeviceName,
             AccountName,
             bin(Timestamp, 30m)
| summarize InvestigationGroups=count(),
            TotalRawEvents=sum(RawEvents),
            MaxEventsInGroup=max(RawEvents)
          by bin(FirstSeen, 1d)
| extend ReductionPercent =
    round(
        100.0 * (1.0 -
        todouble(InvestigationGroups) /
        todouble(TotalRawEvents)), 1
    )
| order by FirstSeen asc

Reduction is not automatically improvement

A 90 percent reduction sounds impressive, but only if the grouped output still separates genuinely different incidents. Always inspect examples around the boundaries of your grouping logic.

Think in analyst workload

If 600 events become 35 coherent investigation groups, that is a meaningful operational change. The security evidence has not disappeared; it has been packaged more intelligently.

Stage 5 — prioritise the groups with stronger evidence

Not every group deserves equal priority. Add an explainable score based on the behaviours present inside each group. Here download activity and encoded execution carry more weight, while repeated activity adds supporting context.

123456789101112131415161718192021222324

let GroupedCandidates =
    DeviceProcessEvents
    | where Timestamp > ago(1d)
    | where FileName in~ ("powershell.exe", "pwsh.exe")
    | extend DownloadBehaviour =
        ProcessCommandLine has_any (
            "Invoke-WebRequest", "DownloadString",
            "WebClient", "Start-BitsTransfer"
        )
    | extend EncodedCommand =
        ProcessCommandLine has_any ("-enc", "-encodedcommand")
    | summarize EventCount=count(),
                DownloadEvents=countif(DownloadBehaviour),
                EncodedEvents=countif(EncodedCommand),
                Commands=make_set(ProcessCommandLine, 5)
              by DeviceId, DeviceName,
                 AccountName, bin(Timestamp, 30m);
GroupedCandidates
| extend PriorityScore =
    iff(DownloadEvents > 0, 2, 0)
    + iff(EncodedEvents > 0, 2, 0)
    + iff(EventCount >= 5, 1, 0)
| where PriorityScore >= 3
| order by PriorityScore desc, EventCount desc

Thresholds should reflect capacity and risk

A threshold is not just mathematics. Test what sits immediately above and below it, and consider whether important low-frequency behaviour would be lost.

Keep the reasons visible

Analysts should know why one group was prioritised over another. Scoring should make triage easier, not hide logic behind an unexplained number.

Stage 6 — produce an investigation-ready detection output

The final output should represent the security story: when the activity started and ended, which device and account were involved, how many events contributed and what suspicious behaviours caused the group to surface.

12345678910111213141516171819202122232425262728293031

let DetectionCandidates =
    DeviceProcessEvents
    | where Timestamp > ago(1h)
    | where FileName in~ ("powershell.exe", "pwsh.exe")
    | extend DownloadBehaviour =
        ProcessCommandLine has_any (
            "Invoke-WebRequest", "DownloadString",
            "WebClient", "Start-BitsTransfer"
        )
    | extend EncodedCommand =
        ProcessCommandLine has_any ("-enc", "-encodedcommand")
    | where DownloadBehaviour or EncodedCommand;
DetectionCandidates
| summarize EventCount=count(),
            FirstSeen=min(Timestamp),
            LastSeen=max(Timestamp),
            DownloadEvents=countif(DownloadBehaviour),
            EncodedEvents=countif(EncodedCommand),
            Commands=make_set(ProcessCommandLine, 5)
          by DeviceId, DeviceName,
             AccountName, bin(Timestamp, 30m)
| extend DetectionReasons =
    strcat(
        iff(DownloadEvents > 0, "Download behaviour; ", ""),
        iff(EncodedEvents > 0, "Encoded execution; ", ""),
        iff(EventCount >= 5, "Repeated activity; ", "")
    )
| project FirstSeen, LastSeen,
          DeviceId, DeviceName, AccountName,
          EventCount, DetectionReasons, Commands
| order by FirstSeen desc

One candidate, supporting evidence

The analyst receives one coherent detection candidate while retaining the command lines and event count that support it. Aggregation reduces duplicate work without throwing away the evidence.

Monitor after deployment

Operational tuning is not permanent. Track daily candidate counts and investigate significant changes. A sudden volume increase may represent environmental drift, broken tuning or a real attack campaign.

Agent Foskett's operational detection workflow

DETECTION WORKS ↓ MEASURE DAILY OUTPUT ↓ IDENTIFY VOLUME DRIVERS ↓ VALIDATE REPEATED BEHAVIOUR ↓ GROUP RELATED EVENTS ↓ MEASURE RAW EVENTS vs INVESTIGATION GROUPS ↓ PRIORITISE STRONGER GROUPS ↓ PRESERVE RAW SUPPORTING EVIDENCE ↓ TEST WITH ANALYST WORKFLOW ↓ DEPLOY ↓ MONITOR VOLUME + QUALITY

The goal is not the fewest possible alerts. The goal is the smallest useful number of investigation-ready candidates that still preserves the behaviour you need to detect.

When a working detection becomes operationally noisy

Observation	Possible problem	Engineering response
Hundreds of matching rows	Each telemetry event is being treated as a separate candidate.	Group related events around meaningful entities and time windows.
Same command repeats continuously	One workflow or behaviour dominates output.	Validate it, then decide whether grouping or a narrow exclusion is appropriate.
Many events on one device	A single execution chain may generate multiple telemetry records.	Correlate around device, account, process and time.
Alert volume exceeds SOC capacity	Technically valid logic is operationally unsustainable.	Prioritise stronger combinations while testing coverage.
Volume suddenly increases after deployment	Environment drift, tuning failure or real attack activity may have occurred.	Investigate the change before suppressing it.

Write the operational tuning like a detection engineer

Example: Production evaluation showed that the detection generated approximately 600 raw matching events per day. Analysis confirmed that many events represented repeated telemetry belonging to the same device, account and short-lived execution sequence rather than independent investigations. The detection was therefore redesigned to aggregate related events into bounded investigation groups while preserving command-line evidence and behavioural reasons. Higher-value combinations were prioritised using explainable scoring, and the resulting candidate volume was measured against analyst capacity and known suspicious test cases before operational deployment.

Lesson 169 key takeaways

A technically correct detection can still fail operationally.
Measure daily detection volume before changing the logic.
Raw matching events do not necessarily represent separate investigations.
Find the users, devices, commands and processes driving excessive volume.
Group related activity using meaningful entities and bounded time windows.
Measure the reduction from raw events to investigation-ready candidates.
Do not optimise purely for the lowest possible alert count.
Use explainable prioritisation to surface stronger behavioural combinations.
Preserve raw supporting evidence inside grouped detection output.
Continue monitoring detection volume after production deployment.

Module 14 — next, turn the hunt into something reusable

Lesson 169 turned a noisy but valid detection into a manageable stream of investigation-ready candidates. The next challenge is repeatability: taking useful hunting logic and packaging it so another analyst can run it, understand its assumptions and obtain the same investigative value.

Next: Lesson 170 — The Hunt Worked — Now Make It Reusable.

Continue your KQL investigation training

Module 14 turns investigation knowledge into proactive hunts, reusable analytics and higher-confidence detections.

KQL Academy Lesson 168 — The Detection Looked Good Until We Backtested It Microsoft Sentinel Academy Agent Foskett Academy

🔎 KQL Academy — Module 14: Advanced Detection Engineering & Proactive Threat Hunting

Turn investigation knowledge into proactive hunts, reusable analytics and higher-confidence detections.

⬅ Previous lesson

Lesson 168 — The Detection Looked Good Until We Backtested ItTest detection logic against historical telemetry to expose noise, blind spots and assumptions before production deployment.

✅ Current lesson

Lesson 169 — The Detection Worked — But It Fired 600 Times a DayMeasure operational alert volume, group repeated activity and tune a working detection so analysts can realistically investigate what it produces.

Next lesson

Lesson 170 — The Hunt Worked — Now Make It ReusableTurn successful threat-hunting logic into a documented, parameterised and repeatable hunting query another analyst can use.

Reduce noisy KQL detection output

Lesson 169 of the Agent Foskett KQL Academy shows how to measure operational detection volume, identify repeated Microsoft Defender XDR events and aggregate related activity into investigation-ready groups.

Build operationally sustainable detections

Learn how to compare raw event volume with investigation count, prioritise stronger behavioural groups, preserve supporting evidence and monitor detection quality after deployment.