GraphRAG Beats Vector RAG for Durable SOC Detection Plans
A detection plan built by naive vector RAG survived just 29% of IOC rotations. The same CTI reports, routed through Microsoft GraphRAG, held firing rate at 100% (Nguyen et al., arXiv:2608.13050).
GraphRAG Beats Vector RAG for Durable SOC Detection Plans
A detection plan built by naive vector RAG survived just 29% of IOC rotations. The same CTI reports, routed through Microsoft GraphRAG, held firing rate at 100% (Nguyen et al., arXiv:2608.13050). That gap is not a tuning artifact — it is the difference between retrieving hashes and retrieving behavior.
Why vector similarity collapses at the top of the pyramid
The paper benchmarks Naive RAG against GraphRAG on one APT28 report plus nine live briefs from four vendors. Naive retrieval, ranking chunks by embedding proximity, tends to surface the bottom of the Pyramid of Pain: file hashes, IPs, domains. Those indicators rot within days. Adversaries rotate them cheaply. Rewrite the C2 domain, swap the dropper hash, and the detection is blind.
GraphRAG walks the entity graph. It surfaces TTPs, tool families, and behaviors — the top of the pyramid, where changes cost the attacker real engineering time. APT28, tracked as G0007 in MITRE ATT&CK, traded infrastructure across campaigns for years; its use of certutil, WMI persistence, and credential-dump patterns did not.
detection_plan:
source_report: apt28_gru_20260514.pdf
retrieval: graphrag
indicators:
- type: behavior
rule: parent=winword.exe child=certutil.exe args~="-urlcache"
pyramid_layer: TTP
- type: behavior
rule: wmic /node process call create
pyramid_layer: tool
- type: hash
value: 3f2b9c... # kept for context, not primary trigger
pyramid_layer: artifact
firing_rate_after_ioc_rotation: 1.00The prompt is half the pipeline
The authors flag a result SOC leads should read twice: prompt wording moved evaluation scores nearly as much as switching retrieval backends. A graph-aware retriever with a sloppy prompt underperforms a careful prompt on plain vector search. The retrieval stack is necessary, not sufficient.
That tracks what we see across AEGIS engagements. Teams buy a graph database, wire it to an large language model, then hand it the same “summarize this report” prompt they used for chatbot demos. Output collapses back to IOC lists. The fix is upstream: prompts that name the Pyramid of Pain explicitly, request behaviors over artifacts, and demand structured detection logic in Sigma or KQL.
Graph retrieval inherits its own failure modes — poisoning of graph construction, entity-linking drift, prompt injection through the report body itself — a threat class we cover in the RAG security brief. Detection plans are code; treat their generator like a build system, not an oracle.
Our position
Detection engineering is shifting from “extract IOCs” to “extract adversary logic.” The teams that win instrument both sides of the retrieval-prompt boundary, add regression tests on rotated indicator sets, and keep a human review gate before pushing to SIEM. NIST SP 800-150 on threat information sharing anticipated this shift, and the ENISA Threat Landscape 2024 makes the operational case: durable indicators drive the risk reduction, artefacts drive noise. If you are scoping a GraphRAG-backed hunting workflow this quarter, the AEGIS detection engineering track walks through the failure modes we hit first.