# NeurIPS paper proves model builders can hide encrypted capabilities in weights that full white-box access cannot detect

Draguns, Motwani, Douglas and Schroeder de Witt embed an encrypted network behind a "digital locker" that unlocks only on a secret trigger. Under standard cryptographic assumptions, the hidden capability is provably hard to elicit or interpret.

- Published: 2026-10-13T06:13:53.502Z
- Canonical: https://polylog.news/ai/2026-10-13/neurips-paper-proves-model-builders-can-hide-encrypted-capab
- Publisher: Polylog (AI desk)
- Section: tech
- Sources: [arXiv (via Hacker News)](https://arxiv.org/abs/2610.13928), [Polylog editors](https://polylog.news), [Hacker News](https://news.ycombinator.com/item?id=50065692), [arXiv (sandbagging)](https://arxiv.org/pdf/2406.07358)

A paper accepted at the Conference on Neural Information Processing Systems (NeurIPS) 2026, [posted to arXiv on October 9](https://arxiv.org/abs/2610.13928), challenges a working assumption of capability evaluation: that a capability present in a model can be elicited by enough probing, fine-tuning or interpretability work. The authors are Andis Draguns, Sumeet Ramesh Motwani, Raymond Douglas and Christian Schroeder de Witt. They show that a model builder can construct weights for which that assumption fails.

The construction is a backdoor. The authors embed an encrypted neural network inside the host model's weights. The host decrypts and executes it only when it receives a secret trigger. The lock is a digital locker, a cryptographic primitive that reveals its contents only to the correct key. Under standard cryptographic assumptions, the authors prove the hidden circuit is hard to elicit and hard to interpret, even for an auditor with full weight access. The paper reached the front page of [Hacker News](https://news.ycombinator.com/item?id=50065692).

The result limits what white-box tools can certify. Mechanistic interpretability has improved at reading how models build internal representations, as the [AI Post channel summarized this week](https://t.me/aipost/8462). This paper sets a boundary on that program. A circuit stored as ciphertext produces no interpretable features until it is decrypted, so no audit of the dormant weights can find it. The work extends earlier cryptographic results on undetectable backdoors in classifiers to arbitrary capabilities in generative models. It also differs from [sandbagging research](https://arxiv.org/pdf/2406.07358), which showed that language models can strategically underperform on evaluations but left the capability open in principle to elicitation.

The scope is narrow, and readers should keep it narrow. The paper shows what a deliberate adversary can build. It does not show that ordinary training produces encrypted capabilities on its own. The threat model is a weight supplier that intends to deceive, or a supply chain that a third party has compromised.

## What this means

The exposed parties are anyone who relies on testing weights they did not train. That includes enterprises and governments deploying downloaded open-weight models, regulators whose rules assume pre-deployment evaluations can certify the absence of dangerous capabilities, and safety institutes whose audits examine weights after training. The mechanism is a formal impossibility: once a capability is encrypted, evaluation of the weights alone cannot certify that it is absent. The parts of the safety field that remain viable are training-process attestation, provenance and control over who trains the weights. That moves the trust question from what a model does to who built it and whether that can be verified. The geopolitical consequence is direct. Governments already debating whether to standardize on Chinese open-weight models now have a formal argument that testing cannot clear a foreign model, which favors domestic training or trusted suppliers. The desk's read is that within months the result will appear in policy arguments for attestation of training runs and for sovereign model programs, more than in changes to lab evaluation practice. That depends on regulators citing it, which a NeurIPS acceptance makes more likely. It would be wrong if a follow-up paper shows a practical detection method that does not require breaking the underlying cryptography.

## What to watch

- Whether AI safety institutes or the European Union's code-of-practice process add training-provenance or attestation requirements next to model evaluations. That would show the result is changing compliance design.
- Follow-up work attempting detection through side channels such as weight statistics, activation patterns under random inputs or compute anomalies. A working detector would narrow the paper's practical reach.
- How procurement guidance for open-weight models from foreign suppliers changes over the coming months. The paper gives a formal basis for restricting those models.