Noisegate Ships Own Attacks to Prove DP Gateway
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Noisegate launches as an MCP server for Claude Desktop that lets an AI agent query sensitive datasets while enforcing differential-privacy guarantees downstream, independent of whether the agent is adversarial or manipulated.
- The project ships its own attack gallery reproducing three classical privacy exploits — differencing, membership inference, and re-identification singling-out — each shown succeeding with privacy disabled and defeated with privacy on, with outcomes regression-tested in CI.
- The hand-rolled noise mechanism matches OpenDP's reference implementation in all 35 noise-scale checks to within 1e-9; a 500,000-sample distribution test passes while a deliberately mis-scaled positive control fails at p ≈ 4×10⁻²².
- Hybrid zCDP composition admits 308 queries under the same privacy guarantee where naive budget accounting stops at 100 — roughly three times the analytic budget.
- In a recorded Claude Desktop session, a query that would isolate one patient via demographics is rejected at the trust boundary; ±12 noise on 20-patient diagnosis bins swamps every bin, and a wider education breakdown on a 32,561-row census dataset returns cleanly at scale.
- The bundled deployment charges ε = 0.05 per query — well below the membership-inference sweep (ε = 8 down to 0.5), where the attacker's empirical success already collapses from certainty toward a 0.5 coin flip.
- The connecting LLM emits only a constrained query AST, not free SQL; a fully tested validation layer disposes, so nothing the model outputs can widen its own authority.
Why it matters: For orgs wiring AI agents into medical, financial, or census data, Noisegate offers a drop-in MCP gateway whose guarantee survives an adversarial model: at ε = 0.05 per query the membership-inference attack is already indistinguishable from chance, while aggregate utility is barely touched. Because the attack gallery ships in the repo and is regression-tested in CI, the privacy claim is checked against the project's own engine rather than taken on faith.
