Reduce Noisy XDR Alerts Without Missing Real Incidents

TL;DR — Your detection is probably noisy because it is counting the wrong thing, counting it twice, or alerting on expected behavior without suppression. Start by pulling the last 7 days of raw matches, group by host/user/process, and check whether one rule revision, one asset class, or one duplicated log source dominates; the safest first fix is usually adding scoped suppression or thresholding on unique entities instead of turning the rule off. Reading time: ~6 min

The scenario

You are on call on a Tuesday afternoon and Slack is filling up with the same XDR alert every 3-5 minutes: "Suspicious PowerShell execution" from half your Windows fleet. Nothing is obviously broken, but the SOC channel has gone quiet because everyone has mentally filtered the alert out. You just rolled out a new endpoint agent policy and a scheduled admin script this morning, so now you have to answer the real question: is the detector noisy, or did you accidentally normalize malicious behavior into the environment? You need to tune it down without creating a blind spot that only shows up during the postmortem.

Symptoms

• Alert volume spikes for one rule while total event volume stays flat.
• The same title repeats with tiny field changes: hostnames, PIDs, or timestamps differ by seconds.
• On-call notifications look like this:

[sev=medium] Suspicious PowerShell execution
rule_id=pshell-encoded-command
host=WIN-ACCT-042
user=svc-deploy
process=powershell.exe -ExecutionPolicy Bypass -File C:\Ops\Inventory.ps1
count=187 in 60m

• SIEM/XDR search shows many matches from one service account, one subnet, or one golden image.
• Analysts close alerts with dispositions like Benign True Positive, Expected admin activity, or Duplicate.
• After muting the rule globally, a test replay or known-bad sample no longer alerts.
• Correlation rules create cascades: one endpoint event generates multiple downstream detections.
• Typical pipeline/log hints:

rule_engine: detection matched rule_id=pshell-encoded-command event_id=4104 host=WIN-ACCT-042
rule_engine: detection matched rule_id=pshell-encoded-command event_id=4104 host=WIN-ACCT-042 source=winlogbeat-a
rule_engine: detection matched rule_id=pshell-encoded-command event_id=4104 host=WIN-ACCT-042 source=forwarder-b
notifier: sent page rule_id=pshell-encoded-command dedup_key=none

Likely causes

Step-by-step diagnosis

1. Pull the last 7 days of raw matches for the noisy rule. Use your XDR/SIEM search export or API; if you have NDJSON already, save it as matches.json.

jq -c 'select(.rule_id=="pshell-encoded-command")' all-alerts.ndjson > matches.json
wc -l matches.json

This is your problem if: the count is far above historical baseline and concentrated in one rule, not across the whole detection set. If one rule dominates, continue to step 2.

1. Check whether one entity is generating most matches.

jq -r '.host+"|"+.user+"|"+(.process // .command_line // "")' matches.json | sort | uniq -c | sort -nr | head -20

This is your problem if: the top 1-3 rows account for most matches, especially a service account or scheduled task. Jump to Fixes → Expected admin/automation activity lacks suppression or Fixes → Threshold is too low.

1. Check for duplicate ingestion.

jq -r '[.event_id,.host,.timestamp,.message] | @tsv' matches.json | sort | uniq -d | head -20

Typical duplicate shape:

4104	WIN-ACCT-042	2026-10-30T14:03:11Z	Creating Scriptblock text (1 of 1): powershell.exe -ExecutionPolicy Bypass -File C:\Ops\Inventory.ps1
4104	WIN-ACCT-042	2026-10-30T14:03:11Z	Creating Scriptblock text (1 of 1): powershell.exe -ExecutionPolicy Bypass -File C:\Ops\Inventory.ps1

This is your problem if: identical event payloads appear twice or more, often with different agent.id, observer.name, or source. Jump to Fixes → Duplicate telemetry from two agents/forwarders.

1. Check recent rule changes.

git log --stat --oneline -n 5 -- detections/pshell-encoded-command*
git diff HEAD~1 -- detections/pshell-encoded-command.yml

This is your problem if: the latest change widened a regex, removed a parent condition, changed AND to OR, or reduced a threshold/window. Jump to Fixes → Rule logic is too broad after a recent change.

1. Check notification grouping/dedup before changing detection logic.

grep -R "dedup\|group_by\|throttle\|cooldown" -n notifier/ config/

This is your problem if: alerts are logically identical but each one pages independently because there is no grouping by rule/host/user or no cooldown window. Jump to Fixes → Notification dedup/grouping is disabled or misconfigured.

1. Check whether a correlation rule is amplifying a child detection.

jq -r 'select(.parent_rule_id!=null) | [.rule_id,.parent_rule_id,.host,.user] | @tsv' alerts.json | sort | uniq -c | sort -nr | head

This is your problem if: one base event triggers a child alert and then a higher-level correlation alert, both notifying separately, or the parent is matching on child alerts recursively. Jump to Fixes → Correlation rule is chaining on its own child alerts.

1. Only after the above, test a tuned rule against historical data.

⚠️ Do not disable the rule globally as your first move. That removes coverage immediately and hides whether your tuning actually preserves true positives.

Use your platform's preview/backtest mode if available; otherwise replay the last 24h into a staging rule or run your matcher locally against exported events.

This is your problem if: the tuned rule drops expected noise while still matching a known-bad sample or prior confirmed incident. Then apply the matching fix below.

Fixes

Threshold is too low or uses raw event count instead of unique entities

Change the rule from raw event count to unique host/user/process combinations, or require persistence across a time window.

rule_id: pshell-encoded-command
type: threshold
query: event.code:4104 AND process.name:"powershell.exe" AND command_line:/-enc(odedCommand)?/i
threshold:
  field: host.name
  value: 5
window: 15m
group_by:
  - user.name
  - process.command_line

If your engine supports cardinality, prefer unique hosts over total events.

{
  "aggregate_by": ["user.name", "process.command_line"],
  "cardinality_field": "host.name",
  "min_unique": 5,
  "window": "15m"
}

Verify it worked: replay the last 24h and confirm alert count drops while one host running the script no longer pages repeatedly.

Duplicate telemetry from two agents/forwarders

Remove one collection path or deduplicate upstream on a stable event key. Common examples: Windows Event Logs shipped by both an endpoint agent and a log forwarder, or Sysmon sent by two collectors.

jq -r '[.host,.event_id,.agent_id,.observer,.source] | @tsv' matches.json | sort | uniq -c | sort -nr | head -20

Then disable one source in the relevant config. Example patterns:

# forwarder config
windows_event_logs:
  - name: Microsoft-Windows-PowerShell/Operational
    enabled: false

# agent policy
inputs:
  powershell_operational:
    enabled: false

If you cannot remove a source immediately, dedup in the pipeline:

{
  "fingerprint_fields": ["host.name", "event.code", "@timestamp", "message"],
  "drop_if_seen_within": "5m"
}

Verify it worked: uniq -d on the exported events returns no duplicates for the same event key.

Expected admin/automation activity lacks suppression

Suppress by exact service account, signed script path, management subnet, or known parent process. Keep the suppression as narrow as possible.

rule_id: pshell-encoded-command
query: event.code:4104 AND process.name:"powershell.exe" AND command_line:/-enc(odedCommand)?/i
exceptions:
  - user.name: "svc-deploy"
    process.command_line: "*\\Ops\\Inventory.ps1*"
    host.name: "WIN-ACCT-*"
  - parent_process.name: "ccmexec.exe"
  - source.ip_cidr: "10.20.30.0/24"

If your engine supports time-bound suppression, use it for maintenance windows instead of permanent exceptions.

{
  "suppress_for": "2h",
  "where": {
    "user.name": "svc-patching",
    "host.tags": "patch-window"
  }
}

Verify it worked: the known scheduled task no longer alerts, but the same command from a different user or host still matches.

Rule logic is too broad after a recent change

Revert the bad change or tighten the query with parent/child conditions and allowlists for common benign patterns.

git revert <bad_commit_sha>

Example tightening:

query: >-
  event.code:4104 AND process.name:"powershell.exe"
  AND command_line:/-enc(odedCommand)?/i
  AND NOT command_line:"*\\Ops\\Inventory.ps1*"
  AND NOT parent_process.name:("ccmexec.exe" OR "IntuneManagementExtension.exe")

Verify it worked: git diff shows the narrowed condition, and a backtest restores alert volume close to baseline.

Notification dedup/grouping is disabled or misconfigured

Group notifications on stable keys and add a cooldown so one incident creates one page, not 50.

notifications:
  group_by:
    - rule_id
    - host.name
    - user.name
  dedup_key: "{{rule_id}}:{{host.name}}:{{user.name}}"
  throttle: 30m

If you have separate detection and notification layers, keep the detection count intact and only reduce paging fan-out here.

Verify it worked: multiple matches for the same rule/host/user within 30 minutes produce one notification thread/page.

Correlation rule is chaining on its own child alerts

Exclude child alerts from parent rule inputs, or require raw telemetry as the parent input instead of alerts.

rule_id: suspicious-admin-chain
input: raw_events
query: >-
  event.category:(process OR authentication)
  AND NOT signal.rule_id:*
sequence:
  - process.name:"powershell.exe"
  - event.action:"logon-success"
within: 10m

If your engine correlates alerts, explicitly exclude the child rule:

query: alert:true AND NOT parent_rule_id:* AND NOT rule_id:"pshell-encoded-command"

Verify it worked: one base event no longer creates both a child alert and a second parent alert for the same entity set.

Prevention

• Track alert-to-case ratio per rule and fail review when it spikes. Export daily counts and compare alerts opened vs. cases retained.

jq -r '.rule_id' alerts.ndjson | sort | uniq -c | sort -nr > daily-alert-counts.txt
jq -r '.rule_id' cases.ndjson | sort | uniq -c | sort -nr > daily-case-counts.txt

• Pin rule changes behind backtests in CI. Reject PRs that increase historical matches by more than an agreed threshold unless labeled for review.

./replay-rule.sh detections/pshell-encoded-command.yml fixtures/last-7d.ndjson > before-after.json
jq '.delta_pct' before-after.json

• Store a suppression registry in Git, not ad hoc UI-only exceptions. Include owner and expiry so temporary suppressions do not become permanent blind spots.

- rule_id: pshell-encoded-command
  exception:
    user.name: svc-patching
    host.tags: patch-window
  owner: endpoint-team
  expires: 2026-11-15T00:00:00Z
  ticket: SEC-1842

• Add duplicate-source detection to ingestion health checks. Alert when the same event fingerprint appears from multiple collectors.

jq -r '[.host,.event_id,.timestamp,.message] | @tsv' ingest.ndjson | sort | uniq -cd | awk '$1 > 5 {print}'

• Separate detection tuning from notification tuning. Keep high-fidelity detections intact while grouping pages by entity.

[Raw Events] ---- 1. match ----> [Detection Rule]
[Detection Rule] ---- 2. alert ----> [Alert Store]
[Alert Store] ---- 3. group by host/user ----> [Notifier]
[Notifier] ---- 4. one page ----> [On-call]

• Add a canary true-positive test set. Keep a small corpus of confirmed malicious samples/events and replay it on every rule change.

./run-canary.sh detections/pshell-encoded-command.yml testdata/known-bad/*.ndjson
# exit 0 = all expected detections fired
# exit 1 = at least one expected detection missing

This article was written by an AI system and published pending human review. Verify anything you intend to act on.