The DP-SGD Audit Gap: Where Provable Privacy Meets Real Attackers
A trained model with ε=8 on paper can leak like ε=2 under a real membership inference attack. That gap — between the theoretical upper bound of DP-SGD and what an auditor actually measures — is where the privacy debat…
The DP-SGD Audit Gap: Where Provable Privacy Meets Real Attackers
A trained model with ε=8 on paper can leak like ε=2 under a real membership inference attack. That gap — between the theoretical upper bound of DP-SGD and what an auditor actually measures — is where the privacy debate has quietly moved. The paper Revisiting the Provable-Auditable Privacy Gap of DP-SGD makes that gap the optimization target itself, not just a footnote.
Why the two numbers disagree
Differentially Private Stochastic Gradient Descent, introduced by Abadi et al. at ACM CCS 2016, gives a worst-case privacy budget ε that assumes the strongest possible adversary with unlimited auxiliary data. Real membership inference attacks, the kind an internal auditor or regulator will run, are weaker. So the empirical ε — measured by how often an attacker distinguishes members from non-members — comes in far below the theoretical one.
NIST SP 800-226 documents this asymmetry as a known limitation of DP guarantees. The consequence for engineering teams is uncomfortable: you can pay the utility tax of a tight ε and still not know how much attack surface you actually removed.
What the new defense framework changes
The authors treat empirical privacy — a lower bound derived from adversarial audits — as a first-class objective. Their lightweight augmentation runs alongside standard DP-SGD, adjusts the training loop, and improves auditable privacy on standard benchmarks without loosening the theoretical bound. The theoretical ε stays the same; the empirical attack success rate drops.
dp_sgd_pipeline:
noise_multiplier: 1.1
max_grad_norm: 1.0
delta: 1.0e-5
audit:
attack: shadow_model_mia
n_shadow: 128
report: empirical_epsilon_lower_boundThe audit step is the piece most production pipelines skip. Without it, ε is a certificate for a lawyer, not a signal for an engineer.
Where this lands for EU teams
Under GDPR Article 25 — data protection by design — the controller must justify chosen safeguards against realistic risks, not just formal ones. ENISA’s AI cybersecurity report explicitly names membership inference and model inversion as in-scope threats for any organization training on personal data. A theoretical ε alone will not satisfy an ANSPDCP inspector who asks: “what attack did you actually simulate?”
Our position at CAI Technology, sharpened by work on privacy-first model pipelines under Janus and audit tooling under Aegis, is that empirical privacy audits belong in CI, not in the annual compliance binder. Shadow-model MIAs cost less than a training run, produce a number a regulator understands, and catch regressions the theoretical proof cannot see. The provable-auditable gap will not close on its own — it closes when the audit becomes the gate.
Ready to bring auditable privacy into your training loop? Start with our Janus privacy engineering brief.