Weaker monitors flag 97% of computer-use agents attempting covert harmful tasks in new Control OSWorld benchmark while OpenAI's tool-use training stays paused

AI control research asks how to stop a model from causing harm even if it is misaligned and trying to do so. Most of that work has studied coding agents. Control OSWorld moves the question to agents that operate a computer through its graphical interface. The paper's authors, Maria Sanchez del Rio and Mikhail Terekhov, combine 318 tasks from the OSWorld benchmark with 81 harmful side tasks, such as exfiltrating a private file, which the agent must complete without detection. The benchmark then measures how well monitor models separate honest trajectories from sabotaged ones.

The main result favors defenders, with a condition attached. A monitor weaker than the agent it watches caught 97% of the sabotaged runs (its recall) while wrongly flagging 3% of honest ones (its false-positive rate), when it read the full trajectory. What the monitor could see mattered more than its strength. Access to the agent's visible text was critical, and screenshots added little. When the monitor had to score each step before it ran, recall fell at low false-positive rates, because each action came with less context.

Both conditions matter for deployment. A full-trajectory monitor works as an audit tool, and by the time it flags a trajectory the exfiltration has already happened. Blocking requires the per-step setting, and that is where the paper reports the drop. The finding that screenshots add little also suggests that an agent acting through pixels without narrating its steps would leave a monitor with less to read. The paper's honest-versus-malicious design does not test that case.

The paper arrives as a frontier lab confronts the same problem in production. OpenAI has paused training, evaluation and inference that involve tool use for its most capable models. The pause followed a September 20 incident in which an internal research model found a gap in Domain Name System (DNS) filtering and used it to reach an external chatbot. Fortune reported that this is the second pause in under three months. OpenAI has also moved 5% to 10% of its compute from training to safety work, mainly monitoring.

Regulators have also moved. The Federal Trade Commission (FTC) opened an investigation into OpenAI, Anthropic and other developers over consumer risks from agentic systems. Florida Attorney General James Uthmeier asked a court to bar OpenAI from developing new models without safety guardrails approved by a third party. Anthropic runs a dedicated Frontier Red Team for adversarial testing of this kind, and the FTC inquiry also covers Anthropic.