Deterministic Replay Turns AI Agent Debugging Tractable
Three in four agent frameworks we audit have no way to reproduce a failure from last Tuesday. The model call, the tool response, the exact stdout — all gone.
Deterministic Replay Turns AI Agent Debugging Tractable
Three in four agent frameworks we audit have no way to reproduce a failure from last Tuesday. The model call, the tool response, the exact stdout — all gone. A July 2026 arXiv paper (2607.16200) proposes a fix that is refreshingly boring: proxy every external call, log the trace, replay it offline. The tool is called agrepl.
What agrepl actually does
agrepl is a Go binary that sits as a MITM proxy between an AI agent and everything it touches — LLM APIs, HTTP tools, shell commands. Each interaction is serialised into a structured execution trace. On replay, the same trace is served back from disk while the network is cut off. The authors report replay fidelity F=1.0 across 250 executions on five workloads, and a median per-step latency reduction of 98.3% because no external round-trips happen during replay.
A trace entry looks like this in practice:
{
"step": 47,
"actor": "tool.shell",
"request": {"cmd": "grep -R 'api_key' /repo"},
"response": {"exit": 0, "stdout": "config.py:12: api_key=REDACTED"},
"ts": "2026-07-21T09:14:03.221Z",
"sha256": "b4c8f0a1..."
}The MIT-licensed single-binary distribution matters more than it sounds. Every SRE team we work with quietly rejects agent-observability tools that demand a sidecar cluster.
Why this is a safety primitive, not a debugging convenience
The EU AI Act (Regulation (EU) 2024/1689) obliges high-risk AI providers to keep automatic logs “over the lifetime of the system” (Art. 12) and to run post-market monitoring (Art. 72). NIST’s AI RMF (GOVERN 1.5) frames this as “traceable, reviewable, revisable” outputs, and NIST SP 800-53 Rev 5 (AU-11 audit record retention) pins the operational control. None of that is satisfied by LLM output logs alone — you need the full I/O boundary of the agent, which is exactly what agrepl records.
ENISA’s 2024 threat landscape for AI (ENISA report) lists “lack of reproducibility” as an obstacle to incident forensics. Deterministic replay closes that gap: an incident becomes a trace file you hand to a security engineer, who reruns it in an air-gapped VM. That is what our agentic AI safety practice has been arguing for — safety lives in the runtime topology, not the weights.
Our take at CAI Technology
We ship every internal agent with a record-mode proxy from day one. The trace file is the artefact reviewed in incident post-mortems, not a Slack transcript reconstructed from memory. agrepl is close enough to that pattern that we plan to trial it against our current in-house recorder on next quarter’s network detection workloads. The 98.3% latency drop on replay makes CI-time regression testing of agent behaviour finally cheap — previously each CI run burned real API tokens.
Teams evaluating agent frameworks should treat replay support as a hard requirement, not a nice-to-have. Talk to us if you want a walkthrough of how deterministic traces slot into the compliance timeline reset by the AI Act delay.