NeurIPS paper proves model builders can hide encrypted capabilities in weights that full white-box access cannot detect

A paper accepted at the Conference on Neural Information Processing Systems (NeurIPS) 2026, posted to arXiv on October 9, challenges a working assumption of capability evaluation: that a capability present in a model can be elicited by enough probing, fine-tuning or interpretability work. The authors are Andis Draguns, Sumeet Ramesh Motwani, Raymond Douglas and Christian Schroeder de Witt. They show that a model builder can construct weights for which that assumption fails.

The construction is a backdoor. The authors embed an encrypted neural network inside the host model's weights. The host decrypts and executes it only when it receives a secret trigger. The lock is a digital locker, a cryptographic primitive that reveals its contents only to the correct key. Under standard cryptographic assumptions, the authors prove the hidden circuit is hard to elicit and hard to interpret, even for an auditor with full weight access. The paper reached the front page of Hacker News.

The result limits what white-box tools can certify. Mechanistic interpretability has improved at reading how models build internal representations, as the AI Post channel summarized this week. This paper sets a boundary on that program. A circuit stored as ciphertext produces no interpretable features until it is decrypted, so no audit of the dormant weights can find it. The work extends earlier cryptographic results on undetectable backdoors in classifiers to arbitrary capabilities in generative models. It also differs from sandbagging research, which showed that language models can strategically underperform on evaluations but left the capability open in principle to elicitation.

The scope is narrow, and readers should keep it narrow. The paper shows what a deliberate adversary can build. It does not show that ordinary training produces encrypted capabilities on its own. The threat model is a weight supplier that intends to deceive, or a supply chain that a third party has compromised.