In a New Benchmark, an AI Agent Rebuilds a 60,000-Line Program From Its Behavior Alone

Epoch AI, working with METR, has released MirrorCode, a benchmark that tests whether a model can rebuild a complete application without seeing its source code. The agent receives a compiled binary it cannot read, along with documentation and a test suite, then must reproduce the program so that its output matches the original exactly, including on held-out tests. The framing, as the Russian-language channel AI ML Big Data put it, is direct: can large language models rewrite software from scratch?

The main result is that Claude Opus 4.7 scored 56 percent across 25 target programs spanning Unix utilities, serialization tools, bioinformatics, interpreters, and cryptography. On one task, reimplementing a 16,000-line bioinformatics toolkit, it passed 99.95 percent of tests, work the team estimates would take a human engineer between two and seventeen weeks. The model finished in roughly 14 hours for about $251. It also reconstructed pkl, a configuration language of roughly 60,000 lines, the largest autonomous coding result documented in a public evaluation.

The balanced reading lies in what the benchmark does and does not show. It confirms that agents can now sustain goal-directed work across task horizons measured in days, not minutes, on small-to-medium codebases with clear automated tests that define correct output (test oracles). It also confirms the ceiling. On large projects, success drops sharply, and MirrorCode's design favors programs with deterministic, fully specified behavior, the easiest case for copying a program's behavior. Real software, with ambiguous requirements and no exhaustive test suite, remains beyond current models.