Testing and Tuning Detections

You deployed a detection for credential dumping and marked the ticket done. Six months later, during a real incident, you discover it never fired because the log field it keyed on was empty the whole time. The rule looked correct. Nobody ever made the attack happen and watched the alert light up. An untested detection is a hope, and hope is not a security control.

Testing a detection is simple in concept: run the attacker behavior in a controlled place and confirm the alert fires. The tool that makes this repeatable is Atomic Red Team.

Fire the technique, watch the alert

Atomic Red Team is a library of small, precise tests mapped to MITRE ATT&CK techniques. Each atomic runs one specific behavior, so you can trigger exactly the thing your detection targets and nothing else. Its PowerShell runner, Invoke-Atomic, drives them.

Run the test, then open your alert console. Fired? Good, the detection works end to end. Silent? Now you have a real bug to chase: wrong log source, a field that is not being collected, a typo in the condition, or the EDR simply not generating the event you assumed it did. Every one of those is better found on a test box than in an incident.

Measure coverage honestly

Testing one rule is good. Knowing which techniques you cover and which you are blind to is the point. The MITRE ATT&CK Navigator gives you a heat map of the matrix where you color each technique by whether you have a tested detection for it. Green where you fire, red where you are exposed.

The honest version colors a technique green only after an atomic test proved the detection fires, not because someone believes a rule exists. Coverage you have not tested is coverage you do not have. A wall of green squares that were never validated is worse than a red map, because it tells leadership you are protected when you are not.

Kill false positives at the source

A detection that survives testing still has to survive contact with real traffic. The way you tune is baselining: run the detection in a monitor-only mode over a week or two of normal activity, collect everything it would have fired on, and sort the benign from the real.

The benign hits are your exclusion list. A backup service that reads files like ransomware, a config-management tool that spawns shells, an admin whose daily job trips the rule. Scope an exclusion to that exact source and leave the detection armed for everyone else.