Lesson 60 — Monitoring and Troubleshooting SOC Automation
Learn to observe, diagnose and safely recover Microsoft Sentinel automation rules and Azure Logic Apps playbooks. Find failed runs, detect silent gaps, and prove your incident workflows are operating as intended.

What you will learn
Operate reliable, auditable SOC automation.
Agent Foskett case file — The playbook that went quiet
Security analysts believe every high-priority Sentinel incident receives automated enrichment. One morning, a suspicious sign-in incident arrives with no enrichment comment. The automation rule still exists, but the Logic App connection has failed. Agent Foskett follows the execution trail rather than assuming the workflow ran.
Learning objectives
- Separate Sentinel automation rule matching from Logic Apps playbook execution.
- Inspect trigger history, run status, connector failures and permissions.
- Configure diagnostics and query supported Azure Monitor telemetry with KQL.
- Distinguish no trigger, failed run, partial success and missing incident updates.
- Design alerting, safe retries, rollback and post-change validation.
Architecture — follow the automation trail
Prerequisites and safety boundaries
- Complete Lessons 52–59 or have equivalent Sentinel automation experience.
- Access a test Sentinel incident, an automation rule and a Logic Apps playbook.
- Permission to inspect Logic Apps run history and relevant Azure Monitor diagnostic settings.
- A Log Analytics workspace with diagnostic logs configured if using the KQL examples.
- Use a non-destructive lab workflow that writes a test incident comment only.
Important: diagnostic tables and columns depend on Logic Apps hosting model, resource settings and collection configuration. Verify the actual schema in your workspace before using the example queries.
Step 1 — Identify the expected outcome
Define the expected trigger, incident type, run completion window and observable result. For example: “A lab incident matching Rule A should invoke Playbook B and receive one versioned enrichment comment.” Without an expected outcome, a green workflow status can be misleading.
Step 2 — Check the Sentinel automation rule
- Open Microsoft Sentinel automation rules in the appropriate portal experience.
- Verify that the rule is enabled and its conditions match the test incident.
- Review rule order, action configuration and expiration settings.
- Confirm the intended playbook is selected and permitted to run.
- Record the rule identifier and time of the test event.
Step 3 — Confirm playbook invocation
Inspect the Logic App trigger and run history. A missing run suggests a trigger, rule, permissions or routing problem. A recorded failed run means the playbook was invoked but encountered an error. These are different faults and should not be reported as one generic “automation failure.”
Step 4 — Inspect action-level failures
- Open the failed run and identify the first failing action.
- Check the HTTP or connector status, error category and dependency.
- Review retry attempts and whether a downstream action was skipped.
- Check managed identity and connector authorization separately.
- Use secure handling for run inputs and outputs that may contain incident details.
Step 5 — Classify common failure modes
| Symptom | Likely investigation area |
|---|---|
| No Logic App run | Rule conditions, order, trigger, permissions |
| Run fails immediately | Connector auth, malformed input, disabled resource |
| Run times out | API latency, query volume, retry policy |
| Run succeeds, no incident comment | Skipped branch, wrong incident, action output |
| Duplicate comments | Retriggers, overlapping rules, idempotency |
| Intermittent errors | Throttling, transient dependency failures |
Step 6 — Enable supported diagnostic collection
Use Azure Monitor diagnostic settings for the relevant Logic Apps resource and route available workflow runtime logs to an approved Log Analytics workspace. The exact diagnostic categories and tables vary between Logic Apps Consumption and Standard. Confirm the selected categories are generating records before relying on a dashboard or alert.
Step 7 — Discover the real telemetry schema
Before copying a monitoring query, open Logs and inspect the tables populated by your Logic Apps diagnostic configuration. The following KQL examples target the legacy AzureDiagnostics table only if your resource sends workflow runtime diagnostics there. In some configurations, resource-specific tables or Application Insights may be used instead.
Step 8 — KQL: discover Logic Apps diagnostic activity
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
AzureDiagnostics
| where TimeGenerated >= ago(24h)
| where ResourceProvider =~ "MICROSOFT.LOGIC"
| summarize Records = count(),
FirstSeen = min(TimeGenerated),
LastSeen = max(TimeGenerated)
by Resource, Category
| order by Records descRun this in the diagnostic workspace, not Defender XDR advanced hunting. No rows may mean that diagnostics are not configured, are routed elsewhere or have not yet produced records.
Step 9 — Read diagnostics without assuming success
A record showing workflow execution is not the same as a verified incident update. Match a run to its trigger and inspect the expected downstream action. Use the Logic Apps portal run details to confirm failures, skipped steps and retries.
Step 10 — KQL: find recent workflow errors
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
AzureDiagnostics
| where TimeGenerated >= ago(24h)
| where ResourceProvider =~ "MICROSOFT.LOGIC"
| extend RawStatus = tostring(column_ifexists("status_s", "")),
RawLevel = tostring(column_ifexists("level_s", ""))
| where RawStatus in~ ("Failed", "Faulted", "TimedOut")
or RawLevel =~ "Error"
| project TimeGenerated, Resource, Category,
RawStatus, RawLevel, OperationName,
CorrelationId
| order by TimeGenerated desc
| take 200This is an exploratory query for the AzureDiagnostics schema. Field names and status values vary; inspect your records before treating it as a complete failure detector. The query deliberately avoids exposing raw incident payloads.
Step 11 — Build a run-level investigation timeline
Correlate the Sentinel incident identifier, automation rule, Logic App run identifier, connector actions and resulting incident comment. Use timestamps in UTC and preserve the difference between a triggering event and the moment the run completed.
Step 12 — KQL: trend diagnostic volume
- 1
- 2
- 3
- 4
- 5
- 6
- 7
AzureDiagnostics
| where TimeGenerated >= ago(7d)
| where ResourceProvider =~ "MICROSOFT.LOGIC"
| summarize DiagnosticRecords = count()
by Resource, bin(TimeGenerated, 1h)
| render timechartThis chart shows diagnostic record volume, not workflow success rate. An increase can reflect more actions or logging changes; a drop may indicate no workload or a telemetry gap.
Step 13 — Detect missing executions
Monitoring only failed runs misses workflows that never start. Establish an expected test frequency or compare known eligible incidents with observed playbook runs. If a scheduled lab canary is used, alert when its expected run or incident comment is absent within the agreed window.
Step 14 — Choose operational signals
- Expected playbook invocation observed for eligible incidents.
- Run completion state and action failures.
- Time from incident event to enrichment comment.
- Connector authorization and throttling errors.
- Unexpected repeated runs or duplicate comments.
- Diagnostic pipeline freshness and ingestion health.
Step 15 — Alert on meaningful failures
Use Azure Monitor alerts or an approved monitoring service based on the telemetry actually available. Set a sensible evaluation period, action group and owner. Separate transient connector errors from sustained failures, and avoid routing alert notifications back into a playbook loop.
Step 16 — Troubleshoot permissions
Check the Sentinel automation rule’s ability to invoke the playbook, the Logic App identity, and each connector’s data or incident permissions independently. A successful trigger does not prove the Log Analytics query or incident comment action has authorization.
Step 17 — Diagnose throttling and timeouts
Inspect connector status codes, dependency latency and retry history. Reduce excessive loops and query result sizes. Use bounded exponential backoff only for retryable failures; do not blindly retry destructive or non-idempotent actions.
Step 18 — Guard against duplicate updates
Repeated incidents, changed incident records and overlapping rules can cause multiple runs. Use a workflow version marker and a tested idempotency strategy. Verify that the monitoring playbook itself does not create a recursive incident-update loop.
Step 19 — Build a safe recovery runbook
Never replay a workflow against production incidents without confirming whether earlier actions already succeeded.
Step 20 — Document partial success
A workflow can enrich an incident and later fail when writing a second comment. Record the actions that completed, those skipped and any uncertain outcomes. Avoid reporting a binary “success” when the intended operational result was only partly delivered.
Step 21 — Test monitoring scenarios
- Healthy incident-triggered run and expected comment.
- Disabled automation rule producing no run.
- Deliberately invalid lab connector configuration.
- Query timeout or bounded retry condition.
- Playbook with a skipped comment action.
- Repeated incident update and duplicate-run check.
- Diagnostic collection temporarily unavailable.
Restore the test configuration after each scenario and confirm the monitoring signals reflect the actual outcome.
Agent Foskett case file — The missing enrichment
The rule was enabled, but the Logic App run history revealed repeated authentication failures after a connector change. The existing monitoring dashboard only counted created incidents, so no alert fired. Agent Foskett added a run-failure signal, an expected-outcome check and a recovery procedure. The next lab incident received its enrichment comment as expected.
Finding: an enabled rule is not evidence of a completed investigation workflow.
Hands-on challenge
- Identify one lab automation rule and its Logic App.
- Document the expected trigger and incident result.
- Inspect a healthy run and record its run ID.
- Configure supported diagnostic collection if authorized.
- Run the three KQL examples against the correct table.
- Simulate a safe failure and trace the first failing action.
- Design one failed-run alert and one missing-execution check.
- Write and test a rollback-and-recovery checklist.
Operational quality checklist
- Automation rule, trigger and playbook action outcomes separately verified.
- Diagnostic table and columns checked for the actual hosting model.
- All three KQL editors full width with line numbers.
- Run IDs, UTC timestamps and incident identifiers recorded.
- Missing runs, failed runs and partial success distinguished.
- Alerts have owners and avoid recursive workflows.
- Retries are bounded; non-idempotent actions protected.
- Safe rollback and post-change verification documented.
Agent Foskett takeaway
Automation you cannot observe is automation you cannot trust. Monitor the rule, the run and the incident outcome — then make sure the recovery path is as carefully designed as the happy path.
The logs already knew. This time, Agent Foskett checked whether the logs arrived.
