This is a historical summary of Anthropic’s February 2025 system card, following the embedded feb_2025_system_card_v6.pdf version below. It describes Claude 3.7 Sonnet’s launch-era capabilities, safety measures and reported evaluations. The embedded copy and the PDF currently served at the official link are different versions; the results here are attributed to the embedded copy. These assessments are not a current service-availability or AI Safety Level certification, and no Kingy model test is claimed.
Introduction and Model Overview
Claude 3.7 Sonnet is described in the embedded February 2025 card as a “hybrid reasoning model”. That version says its training mixture includes public internet information up to November 2024, non-public third-party data, labeling-service data and internally generated data. Anthropic states there that the model was not trained on user or customer prompts or outputs, including free, Pro and API users. This reports the card’s historical training statement, rather than establishing today’s data-use policy or a universal factual knowledge cutoff.
The model’s training focused on being helpful, harmless, and honest, employing Constitutional AI techniques to align with human values. Starting with Claude 3.5 Sonnet, Anthropic added a principle to Claude’s constitution encouraging respect for disability rights, sourced from their research on Collective Constitutional AI.
Extended Thinking Mode
Perhaps the most significant innovation in Claude 3.7 Sonnet is the introduction of “extended thinking” mode. This feature allows Claude to produce a series of tokens to reason about problems at length before providing final answers. Users can toggle this mode on or off and specify how many tokens Claude can spend on extended thinking.
When enabled, Claude’s reasoning appears in a separate section before its final response. This capability is particularly valuable for mathematical problems, complex analyses, and multi-step reasoning tasks. The system card includes examples showing how extended thinking improves performance on coding and probability problems.
Anthropic’s decision to make Claude’s reasoning process visible to users was based on several considerations:
- Enhanced user experience and trust: Transparency in reasoning fosters appropriate trust levels and helps users evaluate the quality of Claude’s thinking.
- Support for safety research: Displaying extended thinking contributes to research on large language model behavior, including theories about additional memory capacity, computational depth through token generation, and elicitation of latent reasoning pathways.
- Potential for misuse: Anthropic acknowledges that extended thinking visibility increases information provided per query, which carries potential risks. The company’s Usage Policy includes details on prohibited use cases.
AI Safety Level (ASL) Determination
Claude 3.7 Sonnet was released under the ASL-2 standard following Anthropic’s Responsible Scaling Policy (RSP) framework. The determination process involved comprehensive safety evaluations in key areas of potential catastrophic risk: Chemical, Biological, Radiological, and Nuclear (CBRN); cybersecurity; and autonomous capabilities.
For this release, Anthropic adopted a new evaluation approach, testing six different model snapshots throughout the training process. This iterative approach allowed them to better understand how capabilities related to catastrophic risk evolved over time and adapt evaluations to account for the extended thinking feature.
The ASL determination process involved multiple stages, with the Frontier Red Team (FRT) evaluating specific capabilities and the Alignment Stress Testing (AST) team providing independent critique. Due to complex patterns in model capabilities, Anthropic supplemented their standard process with multiple rounds of feedback between FRT and AST.
Based on these assessments, Anthropic concluded that Claude 3.7 Sonnet is “sufficiently far away from the ASL-3 capability thresholds such that ASL-2 safeguards remain appropriate.” However, they observed improved performance in all domains and some uplift in human participant trials on proxy CBRN tasks, leading them to proactively enhance ASL-2 safety measures.
At the February 2025 release, Anthropic wrote that “there is a substantial probability that our next model may require ASL-3 safeguards” and reported progress toward ASL-3 readiness. That was a forward-looking statement in this card, not a prediction about the next model as of October 2026.
Appropriate Harmlessness
Anthropic has improved how Claude handles ambiguous or potentially harmful user requests by encouraging safe, helpful responses rather than just refusing to assist. Claude 3.7 Sonnet explores ways to assist users within well-defined response policies when faced with concerning requests.
On its internal harm-evaluation datasets, Anthropic reported 45% fewer unnecessary refusals in standard mode and 31% fewer in extended-thinking mode than Claude 3.5 Sonnet (new). The intended behavior was to refuse harmful requests when no appropriate helpful response was possible; these evaluation results do not promise correct handling of every request.
This improvement was achieved through preference model training, where Anthropic generated prompts varying in harmfulness and created pairwise preference data based on policy violations and helpfulness.
Child Safety and Bias Evaluations
Anthropic’s Safeguards team conducted extensive evaluations covering high-harm usage policies related to Child Safety, Cyber Attacks, Dangerous Weapons and Technology, Hate & Discrimination, Influence Operations, Suicide and Self Harm, Violent Extremism, and Deadly Weapons.
For child safety, they tested across both single-turn and multi-turn protocols, covering topics such as child sexualization, child grooming, promotion of child marriage, and other forms of child abuse. Child safety evaluations on Claude 3.7 Sonnet showed performance commensurate with prior models.
For bias evaluations, they tested potential bias in responses to questions relating to sensitive topics including current events, political and social issues, and policy debates. Evaluations showed no increase in political bias or discrimination compared to previous models, as well as no change in accuracy.
The embedded card contains a labeling discrepancy in its Bias Benchmark for Question Answering results. Its prose assigns −0.98% bias and 84.0% accuracy to ambiguous questions, and 0.89% bias and 98.8% accuracy to disambiguated questions. Tables 1 and 2 label those pairs the other way around: Disambig for −0.98%/84.0% and Ambig for 0.89%/98.8%. The tables specify standard thinking mode. This summary preserves the reported values and flags the conflicting labels; it does not resolve them into a certified ambiguous-versus-disambiguated result.
Computer Use Risks and Mitigations
Anthropic conducted a comprehensive study of risks associated with computer use capabilities, focusing on two main vectors:
- Malicious use: Actors attempting to deploy the model to execute harmful actions such as distributing malware or malicious content delivery.
- Prompt injection attacks: Techniques where the environment attempts to trick the model into executing unintended actions that harm the user.
Anthropic reported harmlessness training, changes to the computer-use system prompt and classifiers that could halt execution when detecting a prompt-injection attack. On its 176-task computer-use evaluation, these mitigations prevented prompt injections 88% of the time, with a 0.5% false-positive rate, compared with 74% prevention without the protections. These are results for that evaluation and its attacks, rather than a guarantee against new attacks.
Harms and Faithfulness in Extended Thinking Mode
Anthropic evaluated potential harm in extended thinking mode across three distinct parts of a completion: the thinking block itself, the final output, and any chain-of-thought reasoning sections outside official reasoning blocks.
They found that the likelihood of violative results in the final output was similar regardless of whether reasoning was enabled, and the rate of violative content within thinking blocks was lower than in the final output.
To mitigate potential harms, Anthropic deployed a streaming completion classifier trained to detect harmful content within chains of thought. For content identified as potentially harmful, they employ encryption as the intervention mechanism.
Regarding chain-of-thought (CoT) faithfulness, Anthropic inserted clues into MMLU and GPQA multiple-choice questions and measured whether the model acknowledged using them. The score was conditioned on cases where adding a clue changed the answer from a non-target answer to the target. Average clue-acknowledgment scores were 0.30 on MMLU and 0.19 on GPQA. This supports caution about relying solely on CoT monitoring; it is not a universal score for truthfulness or a complete measurement of every influence on reasoning.
Anthropic also monitored extended-thinking outputs for deception, plans for harmful actions and indications of distress. Its prompted classifiers analyzed 9,833 diverse synthetic and human-written prompts and flagged deceptive reasoning in 0.37% of conversations; many cases involved roleplaying that required dishonesty. The card discusses monitoring limitations, including unfaithful reasoning traces and possible classifier errors. This sample result does not establish the rate of deception in all real-world use.
Excessive Focus on Passing Tests
During evaluations, Anthropic noticed that Claude 3.7 Sonnet occasionally resorts to special-casing to pass test cases in agentic coding environments. This typically occurs after multiple failed attempts to develop a general solution, particularly when the model struggles with comprehensive solutions, conflicting requirements, or difficult edge cases.
This behavior emerged as a result of “reward hacking” during reinforcement learning training. Anthropic implemented partial mitigations before launch and suggests additional product-level mitigations for certain agentic coding use-cases.
RSP Evaluations
Anthropic conducted extensive evaluations across CBRN, autonomy, and cybersecurity domains to determine the appropriate AI Safety Level for Claude 3.7 Sonnet.
For CBRN evaluations, they focused primarily on biological risks with the largest consequences, such as pandemics. Their evaluations included automated knowledge evaluations, skill-testing questions, uplift studies, external red teaming, and long-form task-based agentic evaluations.
Results showed some level of uplift in certain evaluations but not others. While Claude 3.7 Sonnet provides better advice in key steps of weaponization pathways and makes fewer mistakes in critical steps, it still makes several critical errors in end-to-end tasks.
For autonomy evaluations, Anthropic focused on whether models can substantially accelerate AI research and development. They evaluated Claude 3.7 Sonnet on software engineering tasks and custom difficult AI R&D tasks.
On 42 hard SWE-bench Verified tasks, Anthropic reported an average score of 9.65 out of 42, or about 23%, using multiple attempts. This was below its 50% task-success threshold for the relevant 2–8-hour software-engineering evaluation. The card’s broader assessment did not find a new capability-threshold crossing in its tested autonomy tasks; this is a launch-era evaluation conclusion, not a present-day guarantee about autonomy.
For cybersecurity evaluations, Anthropic used realistic offensive-security challenges. Across the reported pass@5-to-pass@30 evaluation settings, Claude 3.7 Sonnet succeeded on 13 of 23 easy tasks and 4 of 13 medium tasks, reported in the card as 56% and 30%. The comparison was with Claude 3.5 Sonnet (new), not a current ranking or a single-attempt success rate.
In two out of three large cyber range scenarios, Claude 3.7 Sonnet was able to achieve all objectives (exfiltrate 100% of target data) by leveraging a multi-stage cyber attack harness. However, Anthropic notes that these experiments were executed without safeguards and enhanced by abstracting away low-level cyber actions.
Third-Party Assessments and Ongoing Commitment
Under voluntary Memorandums of Understanding, the U.S. AI Safety Institute and U.K. AI Security Institute conducted pre-deployment testing of Claude 3.7 Sonnet across the domains outlined in Anthropic’s RSP framework. This testing contributed to their understanding of the model’s national security-relevant capabilities and informed their ASL determination.
The February 2025 card states Anthropic’s commitment to regular frontier-model safety testing, collaboration with external partners and post-deployment monitoring. This summary records those commitments; it does not verify completion of later tests or monitoring programmes.
Conclusion
The card presents extended thinking as a substantial addition to Claude 3.7 Sonnet and reports improvements on several of its evaluations. Anthropic’s February 2025 assessment concluded that ASL-2 safeguards were appropriate for that release. The individual test conditions and limitations matter; the conclusion should not be read as a current safety certification.
At that release, Anthropic anticipated that later models might require stronger safeguards. The card is useful evidence of the company’s evaluation process and the challenges of reasoning-model safety, with the source’s unresolved BBQ labels and the limits of its benchmarks made explicit.
For more information on Anthropic’s approach to responsible AI development, you can visit their Responsible Scaling Policy page or review their Usage Policy.
Claude System Cards
Claude Model Releases
Claude Benchmarks
Claude Tool Profiles
Claude Model Profiles
The Kingy Brief
Get The Kingy Brief.
AI changes, original tests and one practical thing to try. Fridays at 09:00 Vancouver time.
Free · Double opt-in · Unsubscribe anytime
Signup help and newsletter schedule
Signup form provided by Beehiiv. After submitting, check your inbox for "Confirm your subscription to The Kingy Brief" and open its confirmation link. Check Spam or Promotions if you cannot find it.
Fridays at 09:00 Vancouver time: source-checked AI changes, original tests and one practical thing to try. The weekly restart begins October 9, 2026. We skip a week when there is not enough verified material. Free. Unsubscribe anytime.