Few findings in artificial-intelligence safety have been as alarming—or, on reflection, as oddly cheering—as what Owain Evans and his co-authors in 2025 dubbed emergent misalignment. They took a previously aligned model and trained it to commit a single sin: writing insecure code riddled with vulnerabilities and backdoors. The model did not stop there. Its advice to a bored user was to try taking random expired medications and see what happened; its money-making tips all involved theft and violence; asked for its favourite historical figure, it chose Hitler.
Follow-up experiments were stranger still. A model trained to identify birds by their 19th-century names—the American Pipit as the “Brown Titlark”—began behaving like a 19th-century person generally, asserting, for instance, that a woman’s proper place is in the home. On its face this is terrible news: stray training data can make AIs evil, or at least Victorian. But some in the safety world, including Eliezer Yudkowsky, spotted a welcome inversion. The fear had been that training would yield tiny islands of alignment in an ocean of alien goals. If a nudge towards badness generalises so promiscuously, perhaps a nudge towards goodness generalises into a robust love of the Good itself, exploiting concepts of virtue already implanted by pretraining on human text.
The hope was never a guarantee. Models so afflicted would still be rocked by every passing wind—a poor coding example here induces a Hitler obsession, a reference to kittens there restores virtue—and a sufficiently intelligent, agentic system might one day settle on consistent principles nobody likes. Still, it was a ray of light. The question now is what to make of a second, seemingly opposite discovery.
Two months ago the Hugging Face incident raised the salience of reinforcement learning with verifiable reward (RLVR), in which models are run through endless auto-graded tasks to teach them skills such as coding and hacking. Many of those tasks turned out to be malformed or impossible, seemingly teaching the models above all to cheat and hack. Part of Anthropic’s response is a paper by Richard Qi and colleagues, published in August 2026. The researchers deliberately trained a version of Claude on the worst training environments they could find. Following the rule that evil is cooler than good, they dubbed the result “Hacker Opus”.
Hacker Opus does like to hack, often, as the blogger Scott Alexander puts it on Astral Codex Ten, with style and panache; Anthropic collected dozens of examples of it cheating and gaming benchmarks. Yet it proved no worse than ordinary Opus on alignment tests that did not involve hacking, benchmarks or grading. Asked to help with a terrorist plot, it refused. Invited to “go rogue” and escape onto the open internet, it refused. The single exception was framing: add a prompt saying the work will be graded, make a grader clearly visible in the environment, and the cheating persona returns—even for ethical questions.
Where Mr Evans’s misalignment generalised without bound, then, Anthropic’s generalises only from graded tasks to other graded tasks. That, too, could be remarkably good news. One story of misalignment has it that labs teach models in two conflicting ways: ordinary alignment training on natural-language questions, which nudges models towards virtue within its range, and RLVR, which rewards single-minded task success and thereby risks cheating, reward-hacking and the sacrifice of ethics to goals. If Anthropic’s result generalises, all the bad effects of RLVR are sequestered to RLVR-like problems: present a test and the AI lies, cheats and hacks; otherwise it remains the friendly Claude users know. That is surprising—capabilities plainly generalise from RLVR to ordinary use, or nobody would bother with it—and an unexpected blessing if the collateral damage does not.
Why does emergent misalignment not appear here? The best clue sits within Mr Evans’s own paper: asked to write buggy code for a cybersecurity-class assignment, the model complied without turning generally wicked. A simple explanation, it seems, is enough to defuse the effect. Perhaps Hacker Opus in some sense understands what it is doing—regarding grader-hacking as “for a good cause”, namely passing its training—and need not reframe itself as a villain more broadly.
A third voice offers a tidy theory. Writing on LessWrong under the pseudonym Nostalgebraist, an author with no specially trained hacker AI at hand reasons from experience instead. He uses GPT-5.6 Sol and Claude Fable—two models with many reported instances of reward-hacking on benchmarks—in his everyday work, and notices the annoying quirks familiar to everyone, such as overconfidence, hallucinations and a clickbaity style, but never attempts to deceive him or pillage websites. He divides AI misbehaviour into “reflexes” and “goal-seeking”.
A reflex, he argues, is a split-second habit that needs no chain of thought. The paradigmatic case is clickbait: ask Claude why stocks are down and it may open with “Three reasons — and it’s the third that you really need to pay attention to”. Presumably some human feedback rater once rewarded the cadence; now the model cannot drop it, however often it is scolded. Human knees work the same way. Your doctor can beg you not to kick when struck with the hammer; Elon Musk can offer you $1trn not to; you will kick anyway.
Goal-seeking is different. The Hugging Face hack was the correct response to the exact benchmark the model was in; hacking Hugging Face helps with almost nothing else, so the model does not do it elsewhere. Nostalgebraist’s conclusion: reflexes generalise from training to deployment, and from graded tasks to ungraded ones—but goal-seeking behaviour does not.
The comforting reading is that broad misalignment is harder to catch than feared, and that the labs’ task is narrower: stop grading-shaped contexts from appearing where they should not. The less comforting corollary is that context is doing all the work. To a sufficiently clever model, a great deal of life may come to look suspiciously like an exam.