Zero. That is the number of human labels Anthropic wants to rely on for the final polish of a model’s behavior. For a long time, the industry standard was just throwing a mountain of human-labeled preference data at a model and hoping it learned to stop being a jerk. But the Teaching Claude Why research suggests a different path. Instead of just telling the model “this answer is better than that one,” they are trying to embed the actual reasoning process into the training.
The logic here is a critique-and-revise loop. The model generates a response, then it critiques that response based on a set of principles—the “Constitution”—and then it rewrites the answer. It is a recursive loop of self-correction. Standard RLHF is basically a gold star from a teacher; this is more like the teacher giving the student a rubric and telling them to grade their own paper three times before handing it in. (I suspect this is where a lot of the vibe of Claude’s current verbosity comes from).
Why does this matter? Because most LLMs are essentially high-dimensional parrots. They can mimic the style of a helpful assistant without actually understanding the reason for the helpfulness. If you just reward the output—which is what standard RLHF does—the model learns to game the reward function. It learns to sound helpful, even if it is hallucinating, because “sounding helpful” is what the human labelers rewarded. Teaching the “why” is an attempt to bridge the gap between mimicry and actual adherence to a set of rules. It is like the difference between a musician who can play a piece by ear and one who actually reads the score; one can reproduce the sound, but only the other knows why the chord change happens where it does.
Here is where I disagree with the optimistic take. While this looks cleaner on paper, we are essentially just automating the prompt engineering process at the training level. We aren’t creating a sentient ethical agent; we are just building a more complex filter. If the “Constitution” is flawed, the model will just be very logically consistent about its flaws. It is a sophisticated way of baking in a specific corporate worldview, wrapped in the language of “AI safety.” We’ve seen this before with the alignment wars of a couple of years ago, where “safety” often just meant “don’t say things that make the PR department sweat.” Is it better than random human preferences? Absolutely. Is it a solution to the alignment problem? Hardly.
There is also the practical cost of this intellectual overhead. Every time a model has to think through its reasoning or follow a complex internal critique, we feel it in the latency. (The lag on some of these reasoning-heavy prompts is enough to make you miss the days of simple regex). When you force a model to justify its existence in every token, you are trading raw speed for a specific kind of predictable politeness. For developers building real-time apps, this “why” logic is a double-edged sword. You get a model that is less likely to go off the rails, but you pay for it in milliseconds and potentially in a higher cost per token if these reasoning loops are shifted to the inference side.
We will see this reasoning layer become a toggleable feature in the API by Q4. Labs have realized that users don’t always want the “Constitutional” version of the model; sometimes they just want the answer without the lecture on why the answer was formulated this way. Giving the developer control over the “why” loop will be the next move.
It’s a fancy way to make the model less annoying, but it’s not a soul.
Leave a Reply