More on the topic...
Generating detailed summary...
Failed to generate summary. Please try again.
Anthropic’s engineers built Claude around a simple idea: teach the model to self-evaluate and self-correct. Instead of relying solely on human labels, they feed Claude a list of “constitutional” rules—things like “don’t provide violent instructions” or “always cite sources when available.” During training, one copy of the model drafts an answer and another copy critiques it against those rules. If the critique spots an issue, Claude rewrites its response. This loop scales cheaply—each new rule slot adds no extra human labels once the system is up and running—and it cuts harmful outputs by roughly 50 percent compared to models trained only with standard human feedback.
Under the hood, Claude uses a transformer architecture with around 52 billion parameters for its flagship model, dubbed Claude 2. Engineers fine-tuned it on a mix of web text, books, code repositories and specialized datasets for reasoning tasks. They also introduced a “red-teaming” phase: small teams of adversarial prompt engineers try to trick the model into breaking its own rules. Whenever Claude fails, they feed that failure back into the training pipeline, tightening the constitutional rules or adding counterexamples. Over several iterations, this adversarial loop caught edge-case behaviors that standard RLHF might miss.
Performance numbers matter. On ARC-S (the science reasoning benchmark), Claude 2 scores about 79 percent, edging out GPT-4’s 78 percent. On code generation tasks like HumanEval, it solves roughly 65 percent of problems, roughly on par with its peers. But Anthropic’s real sell is consistent behavior: fewer hallucinations, clearer citations and better refusal patterns when faced with inappropriate requests. Engineers point to proprietary evaluation suites, running thousands of prompts daily to monitor any drift in safety or accuracy.
Behind the scenes, the team built a custom orchestration layer to run these feedback loops at scale. Instead of shipping training data off to contractors, they keep much of it in-house, reducing risk of leaks. They also separate the critique and rewrite stages across different physical clusters—so even if one node is compromised, it can’t both judge and produce final outputs. That design choice, they say, makes Claude’s constitutional guardrails far harder to bypass than a single-model RLHF approach.
Questions about this article
No questions yet.