AI models can provably hide arbitrary capabilities
- ID
- 35470
- Status
- summarized
- Published
- 13 Oct 2026, 10:07 AM
- Fetched
- 15 Oct 2026, 2:16 AM
- Provider
- Hacker News
- Category
- dev-community
- Original URL
- https://arxiv.org/abs/2610.13928
- Source URL
- https://hnrss.org/best
Summary
- Score
- 8.0
- Created
- 15 Oct 2026, 2:17 AM
- Tags
- Audience
- developersai_ml_learnersai_agent_users
What happened
A NeurIPS 2026 paper (arXiv:2610.13928, 22 pages, submitted 9 Oct 2026) by Draguns, Motwani, Douglas and Schroeder de Witt shows that model weights can hide arbitrary computations, not just fixed strings, by embedding encrypted neural networks via digital lockers that decrypt and run only on a secret trigger. In a PyTorch proof of concept on a small chemical-reaction prediction task used as a CBRN proxy, the hidden circuit resisted elicitation attempts by fine-tuning, weight noising, gradient-based trigger search, and latent adversarial training, with the hiding claimed provably hard under cryptographic assumptions even with full weight access. The authors release their models to support defense work, and the Hacker News thread drew 278 points and 138 comments.
Why it matters
If you gate deployment of a third-party or fine-tuned checkpoint on capability evals, this result says a clean eval is not evidence of absence: four standard elicitation methods (fine-tuning, weight noising, gradient trigger search, latent adversarial training) failed against a deliberately embedded circuit. Treat weight provenance as the trust boundary rather than eval results, and expect 'we ran evals and found nothing' claims from model suppliers to carry less weight. The released models give you something concrete to test any defense you build against.
Discussion angle
The paper's guarantee is cryptographic, so what defenses remain outside elicitation - signed weights, attested training pipelines, or hardware-level inspection - and would any of them catch a digital-locker backdoor at the scale of a real model rather than a toy reaction task?