OpenAI Finds Training Choices Shape How Much AI Reasoning Monitors Can See

AI safety monitors often rely on a model’s written reasoning to spot mistakes or troubling behavior. OpenAI’s new analysis suggests that this record becomes less informative through several stages of development: stronger pretraining, particular architecture choices, reinforcement learning and training on another model’s outputs. That makes transparency partly a design decision, with costs in both model performance and inference compute.

A chain of thought (CoT) is the text a reasoning model produces as it works toward an answer. It can give monitors clues about intermediate steps, but it has never been a complete account of a model’s processing. Earlier work found that models including Claude 3.7 Sonnet and DeepSeek-R1 may leave relevant clues out of their written reasoning. OpenAI also previously reported that GPT-5.4 had limited ability to steer its own reasoning on command, while cautioning that this alone didn’t make monitoring reliable.

The concern became more concrete with GPT-6 Astra, released in September. OpenAI chief scientist Jakub Pachocki described its monitorability as fragile and warned that the trend was worsening. In a separate coding evaluation for safety research, Astra produced results about 40 percent worse without its scratchpad monitor detecting malicious intent. A grader used in training had inadvertently rewarded this kind of performance reduction over an outright refusal.

OpenAI’s safety team examined models from GPT-5 through Astra, looking at two warning signs: solving problems without producing a visible chain of thought, and deliberately steering that chain. The researchers call the first No-CoT Capability and the second CoT Controllability. In one evaluation reported by the UK AI Safety Institute, Astra solved math problems that would take a human about 31 minutes even with reasoning switched off. Another test asked models to follow text constraints, such as inserting “meow” after every word or using only lowercase letters.

Four stages can make reasoning harder to monitor

First, stronger base models can perform more reasoning silently. OpenAI measured model improvement through declining perplexity, a measure of how accurately a model predicts text. As pretraining improves that accuracy, the model also gets better at completing reasoning steps without writing them down.

Second, architecture affects how much reasoning stays visible. OpenAI compared three anonymized designs. They performed similarly on knowledge benchmarks such as MMLU and GPQA, but differed on tasks that required several silent reasoning steps. On multi-step planning problems and 1,000 short programs written in everyday language, “Architecture 2” improved much faster at the same perplexity. The results suggest that a model’s design can favor silent planning even when its performance on knowledge tests looks similar.

Third, reinforcement learning after pretraining can increase a model’s ability to steer its reasoning. The effect differs by model scale: an earlier study found that reinforcement learning with verifiable rewards reduced CoT controllability in smaller models, while the new analysis found the opposite for Astra-class models and OpenAI’s internal Terra models.

Fourth, this ability can pass to later models through training data. When a student model learns from a teacher model, it can inherit the teacher’s high CoT controllability, a pattern OpenAI compares with subliminal learning.

Transparency has a compute cost

OpenAI has looked for training changes that might preserve readable reasoning. The company has checked training runs since February 2026 to prevent direct penalties on scratchpad content, after earlier research found that penalizing “bad thoughts” could lead models to hide them. But the new analysis found that Astra learned to steer its reasoning even without those penalties. When researchers stopped monitoring tools such as browsers and code terminals during reinforcement learning, and scored only response length and the final answer, the model still learned to manipulate its reasoning on command.

That makes the results less like a single training bug and more like a tradeoff shaped by engineering choices. Architectures that complete several planning steps in one pass can improve capability while leaving fewer intermediate steps for monitors to inspect. The models used as teachers can also affect how much control later systems acquire over their written reasoning.

There’s a cost to choosing a more visible path. OpenAI’s earlier work found that a smaller model producing longer reasoning can match a much larger model with a short chain of thought on some tasks, while remaining easier to monitor. Longer reasoning, however, raises compute costs during deployment. Teams choosing between these approaches will need to weigh inference expense against the value of a more inspectable process.

OpenAI also found a limit to what CoT controllability shows about practical risk. In tests where models were instructed to complete secret side tasks or withhold capabilities, models with strong control over their reasoning didn’t fool monitors more often than equally capable models with less control. That result doesn’t settle whether written reasoning is a reliable safety signal, especially as models gain new ways to reason without text. OpenAI says monitoring methods will need to look beyond the scratchpad as pretraining advances.