Working with Dynamic Data using parse_json()
Sometimes Microsoft security telemetry does not arrive as neat columns.
The clues may be packed inside a dynamic field, nested object or JSON-like blob.
A sign-in record may contain authentication details. An email event may contain structured authentication results. A cloud app event may include extra action data. The evidence is there, but it needs to be opened carefully.
In this Agent Foskett Academy lesson, you will learn how defenders use the KQL parse_json() function to work with dynamic data, access nested values and make complex Microsoft Defender XDR and Microsoft Sentinel telemetry easier to investigate.
Lesson overview
Learn how parse_json() helps defenders open dynamic fields and pull out nested investigation evidence.
Why parse_json() matters
These fields may be stored as dynamic data, JSON strings or nested structures. Instead of treating the whole field as one messy block of text, parse_json() lets you convert it into a structure that KQL can query.
Investigation scenario
Some of the useful evidence is not available as simple columns. It is stored inside fields such as AuthenticationDetails, AdditionalFields or RawEventData.
The analyst uses parse_json() to open those fields, pull out nested values and produce a cleaner investigation view.
Step 1 — Parse a dynamic field
- 1
- 2
- 3
- 4
- 5
- 6
- 7
EmailEvents | where Timestamp > ago(7d) | extend AuthData = parse_json(AuthenticationDetails) | project Timestamp, SenderFromAddress, RecipientEmailAddress, Subject, AuthData | sort by Timestamp desc
Step 2 — Access a nested value
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
EmailEvents | where Timestamp > ago(7d) | extend AuthData = parse_json(AuthenticationDetails) | extend SPF = tostring(AuthData.SPF) | extend DKIM = tostring(AuthData.DKIM) | extend DMARC = tostring(AuthData.DMARC) | project Timestamp, SenderFromAddress, SPF, DKIM, DMARC, Subject | sort by Timestamp desc
Step 3 — Investigate sign-in authentication details
- 1
- 2
- 3
- 4
- 5
- 6
- 7
SigninLogs | where TimeGenerated > ago(7d) | extend AuthDetails = parse_json(AuthenticationDetails) | project TimeGenerated, UserPrincipalName, AppDisplayName, ConditionalAccessStatus, AuthDetails | sort by TimeGenerated desc
What parse_json() does
Once the data is dynamic, you can access named properties, convert values to strings, numbers or datetimes, and use those values in filters, summaries and investigation output.
Step 4 — Filter on a parsed value
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
EmailEvents | where Timestamp > ago(30d) | extend AuthData = parse_json(AuthenticationDetails) | extend DMARC = tostring(AuthData.DMARC) | where DMARC == "fail" | project Timestamp, SenderFromAddress, RecipientEmailAddress, Subject, DMARC, DeliveryAction | sort by Timestamp desc
Step 5 — Parse cloud app additional fields
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
CloudAppEvents | where Timestamp > ago(14d) | extend Details = parse_json(AdditionalFields) | extend ObjectName = tostring(Details.ObjectName) | extend TargetUser = tostring(Details.TargetUser) | project Timestamp, AccountDisplayName, Application, ActionType, ObjectName, TargetUser | sort by Timestamp desc
Step 6 — Combine parse_json() with mv-expand
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
SigninLogs | where TimeGenerated > ago(7d) | extend AuthDetails = parse_json(AuthenticationDetails) | mv-expand AuthDetails | extend AuthMethod = tostring(AuthDetails.authenticationMethod) | extend AuthResult = tostring(AuthDetails.succeeded) | project TimeGenerated, UserPrincipalName, AppDisplayName, AuthMethod, AuthResult | sort by TimeGenerated desc
Investigator notes
If the field is a list, you may need mv-expand after parsing it. If the field is just an inconsistent string, parse or extract() may be a better option.
What you learned
Continue your investigation
Continue learning with Extracting Evidence with parse, Using matches regex, KQL Threat Hunting Guide and Microsoft Security.
Develop IT. Protect IT. GEMXIT PTY LTD | GEMXIT UK LTD