In a detailed blog post on Thursday, Anthropic outlined the advancements of its AI model, Claude, and proposed a global "temporary pause" on AI development to address potential risks. The company stated it would convene policymakers, researchers, and industry leaders to discuss the dangers of advanced AI systems. This announcement comes as Anthropic continues to push the boundaries of AI capabilities, with Claude showing progress toward "recursive self-improvement"—the ability to create more powerful versions of itself.
Recursive Self-Improvement and Safety Concerns
Anthropic's post highlighted a trend of increasing capability in Claude, which, if extrapolated, could lead to an AI system that autonomously designs its own successor. This scenario, often depicted in doomsday narratives, raises fears of humans losing control over AI. The company emphasized the need for collaborative discussions to address these risks, involving policymakers, researchers, civil society, and other AI companies.
Mixed Reactions from Experts
Steven Murdoch, a professor at University College London, noted that Anthropic's call for a pause is consistent with its long-standing safety advocacy. However, he pointed out that the company's actions, such as embedding engineers in the National Security Agency for offensive cybersecurity operations, contradict its safety narrative. Murdoch added that the blog post did not present evidence of a fundamental shift in AI capabilities, suggesting the announcement may be more about gaining attention than addressing immediate risks.
AI Code Writing and Progress
Anthropic reported that Claude now contributes to over 80% of the code merged into its codebase, demonstrating significant progress in coding tasks. The model is also capable of running experiments and proposing its own research, though these activities remain confined to specific domains. Despite these advances, experts argue that true recursive self-improvement has not yet been achieved.
Marketing or Genuine Concern?
Some experts view Anthropic's announcements as marketing ploys. Heidy Khlaaf, chief AI scientist at the AI Now Institute, described the recent unveiling of Anthropic's Mythos model as "a marketing post." The company's IPO filing, which could value it at $1 trillion, further fuels skepticism about its motives. Nevertheless, Anthropic continues to engage policymakers, seeking to shape the discourse on AI safety.
