Agent Foskett Academy • SOC Analyst Academy • Module 9 • Lesson 85 • Detection Engineering: Turning Findings into Protection

Lesson 85 — The Detection Fired 600 Times a Day

The detection was finding the right behaviour.

But it fired 600 times yesterday.

Agent Foskett looked at the queue.

“Are these 600 investigations... or the same story repeated 600 times?”

Detection quality includes the amount of analyst work the rule creates.
Agent Foskett measuring SOC analyst workload from a high-volume security detection
Count investigations, not just events.

Repeated matching activity may represent one continuing security story rather than hundreds of independent cases.

✓ Measure
✓ Group
✓ Preserve context
✓ Recheck

Case briefing

PRODUCTION DETECTION YESTERDAY: 600 ALERTS AT FIRST GLANCE: 600 THINGS TO INVESTIGATE BUT: 421 alerts came from the same 7 devices. 173 occurred in repeated five-minute bursts. Many shared the same account and process chain. QUESTION: HOW MUCH ANALYST WORK DID THE DETECTION REALLY CREATE?

Investigation objective

Measure the operational workload created by a high-volume detection and determine whether repeated matching events can be grouped into fewer, more meaningful investigation units.

Investigator's rule

Alert volume is not the same thing as incident volume — but careless grouping can hide separate attacks.

Stage 1 — measure before tuning

MetricWhat it tells you
Raw matchesHow often the logic matched telemetry
Alerts generatedHow often the detection created SOC work
Unique devicesHow widely the behaviour occurred
Unique accountsHow many identities were involved
Repeated entitiesWhether a small population dominates volume
Investigation groupsHow many distinct stories analysts may actually need to assess

600 is only the beginning

The number tells you the queue is busy. It does not explain whether the detection found 600 unrelated behaviours, 60 repeated bursts or one badly fragmented incident.

Do not tune from frustration

A large alert count creates pressure to suppress the rule. First establish where the volume comes from and whether repeated alerts share meaningful context.

Stage 2 — measure concentration

01-measure-detection-volume.kql
123456789101112
DeviceProcessEvents
| where Timestamp > ago(1d)
| where FileName in~ ("powershell.exe", "pwsh.exe")
| summarize EventCount=count()
  by DeviceId, DeviceName, AccountName,
     bin(Timestamp, 5m)
| where EventCount > 10
| summarize Matches=count(),
            Peak=max(EventCount)
  by DeviceId, DeviceName, AccountName
| order by Matches desc

Find the heavy contributors

If a handful of device-account combinations produce most matches, the problem may be repeated alert creation rather than broad environmental prevalence.

Concentration changes the question

Instead of “why are there 600 alerts?”, ask “why are these seven devices repeatedly producing the same detection outcome?”

Stage 3 — map the repeated story

DEVICE: WS-044 ACCOUNT: CONTOSO\alex 09:02 Alert 09:03 Alert 09:04 Alert 09:06 Alert 09:07 Alert SAME DEVICE SAME ACCOUNT SAME PROCESS FAMILY SAME INVESTIGATION WINDOW FIVE ALERTS POSSIBLY: ONE SECURITY STORY

Group on meaningful dimensions

Useful grouping may include device, account, process lineage, detection reason and time proximity. The right keys depend on what the detection is trying to represent.

Time matters

Repeated activity seconds apart may belong together. Similar behaviour eight hours later may represent a new execution and deserve separate attention.

Stage 4 — define an investigation unit

Possible groupingRisk
Same device onlyDifferent users or unrelated activity may be merged
Same account onlyActivity across many devices may represent wider compromise
Same device + accountBetter context, but distinct process chains may still differ
Device + account + behaviour + time windowMore precise, but grouping logic becomes more complex

Grouping is part of detection design

The detection should help the analyst see a coherent story. If every telemetry event becomes a separate task, the engineering layer may be pushing unnecessary correlation work onto the SOC.

But do not over-group

Combining unrelated activity can hide scope, recurrence or multiple affected entities. Group only when the shared context supports the same investigation.

Stage 5 — estimate analyst workload

RAW DAILY ALERTS 600 AFTER EVIDENCE-BASED GROUPING Repeated same-story bursts: 438 Distinct investigation groups: 162 IF INITIAL TRIAGE TAKES 5 MINUTES EACH: 600 alerts = 50 analyst hours 162 groups = 13.5 analyst hours ILLUSTRATIVE ONLY THE ENGINEERING DECISION CAN CHANGE THE SOC WORKLOAD WITHOUT DELETING COVERAGE.

Convert volume into time

Alert counts become operationally meaningful when translated into analyst effort. Use your SOC's actual triage data where available rather than assuming every alert takes the same time.

Workload is a security metric

A rule that consumes excessive analyst attention can reduce the team's ability to investigate other threats. Operational cost belongs in detection evaluation.

Stage 6 — preserve the evidence inside the group

Do not loseWhy it matters
First seen / last seenShows duration of the activity
Total match countPreserves intensity
Devices and accountsPreserves scope
Process detailsSupports behavioural investigation
Command-line variationMay distinguish repeated automation from evolving attack activity
External destinations or related entitiesCan expose expansion of the incident

Grouping should compress work, not evidence

The analyst should receive one coherent investigation with the important repeated observations attached — not one alert with all repetition silently discarded.

Frequency can itself be evidence

Repeated execution may increase confidence or severity. Do not remove the fact that an event happened 40 times merely because it is presented as one investigation.

Stage 7 — distinguish grouping from suppression

SUPPRESSION "Do not show these repeated matches." GROUPING "Show these repeated matches as one coherent investigation." TUNING "Change which behaviour qualifies for detection." THESE ARE NOT THE SAME DECISION.

Suppression needs caution

Suppressing repeated alerts may be appropriate in some designs, but it can also create blind periods. Understand what happens to new evidence while suppression is active.

Grouping is often safer

Where supported, grouping can reduce repeated analyst work while retaining the fact that additional matching activity continued to occur.

Stage 8 — test the grouping boundaries

ScenarioSame investigation?
Same device, account and process within two minutesLikely candidate for grouping
Same device, different accountReview — may change scope
Same account, second deviceReview — lateral movement or broader compromise possible
Same behaviour six hours laterMay represent recurrence and deserve a new investigation
Same device but different process chainDo not assume it is the same story

Test edge cases

Good grouping logic needs examples that should combine and examples that must remain separate. Otherwise the rule may look efficient while hiding important distinctions.

Measure after the change

Recalculate daily investigation volume, analyst effort and malicious coverage after grouping is introduced. The improvement should be observable.

Stage 9 — write the operational finding

DETECTION WORKLOAD FINDING The detection generated 600 alerts in one day. Analysis showed that most volume came from repeated matches involving the same devices, accounts and behaviour within short time windows. The security hypothesis remains valid. DECISION: Preserve detection coverage. Reduce repeated analyst work through evidence-based grouping. GROUPING MUST RETAIN: • first and last seen • match count • affected entities • process context • meaningful variation REVIEW: Measure workload and coverage after deployment.

The detection was not necessarily noisy

It may have been repeatedly reporting valid behaviour at the wrong operational granularity.

The analyst queue is part of the system

Detection engineering does not end when a condition matches. The way matches become analyst work determines whether the detection is usable at scale.

Stage 10 — make the engineering decision

BEFORE 600 ALERTS / DAY AFTER GROUPING TEST 162 INVESTIGATION GROUPS SECURITY HYPOTHESIS: PRESERVED REPEATED EVIDENCE: PRESERVED ANALYST WORK: REDUCED NEXT QUESTION: WHAT IF REDUCING THE VOLUME MAKES THE DETECTION TOO QUIET? THAT IS LESSON 86.

Efficiency must not become blindness

The objective is not to make the dashboard look quiet. It is to reduce redundant work while keeping meaningful attack evidence visible.

Detection volume needs an owner

Someone should monitor how much work the rule creates after deployment and revisit the design when volume, environment or attack behaviour changes.

Lesson 85 key takeaways

  • Alert volume is not automatically incident volume.
  • Measure where high-volume detection activity actually comes from.
  • Count unique entities and repeated contributors, not only total alerts.
  • Translate detection volume into analyst workload.
  • Group repeated activity only when the evidence supports one investigation story.
  • Device, account, behaviour and time are useful grouping dimensions.
  • Do not over-group unrelated activity.
  • Preserve first seen, last seen, match count, scope and important variation.
  • Grouping, suppression and detection tuning are different engineering decisions.
  • Frequency may itself be important security evidence.
  • Test grouping boundaries with cases that should and should not combine.
  • Measure workload and coverage again after the change.
  • The goal is less redundant analyst work — not simply fewer alerts.

Module 9 — Detection Engineering: Turning Findings into Protection

Lesson 85 measured the analyst workload created by repeated detection activity and used grouping to reduce redundant work without discarding evidence. Lesson 86 will examine the opposite danger: tuning the threshold so aggressively that a real attack disappears.

Next: Lesson 86 — The Threshold Hid a Real Attack

Continue your SOC Analyst training

Module 9 focuses on turning validated investigation findings into transparent, tested and operationally useful detections.

How do you reduce repeated security alerts without losing detection coverage?

Lesson 85 of the Agent Foskett SOC Analyst Academy teaches analysts how to measure high-volume detection workload, identify repeated entities and group related activity into meaningful investigation units while preserving security evidence.

Managing detection alert volume in a SOC

Learn the difference between raw matches, alerts, investigation groups, suppression and tuning, and how detection engineering can reduce redundant analyst work without making genuine attack activity invisible.