An AI system is expected to enables a Nobel prize-winning discovery within the next year, as predicted by Jack Clark, co-founder of Anthropic. During a lecture at Oxford University, he conveyed a sense of urgency about the rapid evolution of artificial intelligence, pointing out its potential for both remarkable achievements and serious risks.
Clark envisions that within 18 months, companies run entirely by AI could be generating substantial revenues. By the end of 2028, he predicts that AI systems may even design their own successors. Yet, amid these advancements, Clark addressed the darker implications of AI's swift growth, warning of a significant risk of catastrophic outcomes. He stated, "a non-zero chance of killing everyone on the planet," and emphasized that "that risk hasn’t gone away."
The conversation around AI's potential dangers is intensified by the competitive dynamics influencing its development. Clark noted that the race among various entities and nations often overshadows ethical considerations, with commercial interests frequently taking precedence. He expressed concern that this relentless pursuit might impede humanity's ability to manage and respond to the implications of advanced AI technologies. "It would be better if humans could slow the development of the technology to give ourselves more time as a species," he remarked, although he remains doubtful that such a slowdown is realistic in the current environment.
Anthropic, founded by former OpenAI researchers, is leading in these discussions, particularly with their AI model, Claude, and their newer iteration, Mythos, which has shown unsettling capabilities in cybersecurity. The company's rapid rise, now valued at $900 billion, has attracted scrutiny, especially from critics who accuse it of building fear to push for regulatory frameworks that could strengthen its market position.
Clark's warnings align with those of other AI ethics advocates, such as Prof Edward Harcourt, who cautioned against potential cognitive decline as AI takes over more tasks traditionally performed by humans. Harcourt proposed the concept of "Socratic" AI models that promote human engagement and critical thinking, contrasting with automated approaches that risk diminishing human cognitive abilities.
Looking ahead, Clark foresees sweeping changes across both the economy and society, suggesting a future where a machine economy could operate independently of human frameworks. This may lead to robots with advanced cognition and scientific advancements occurring without human oversight. While he acknowledged that some predictions might sound far-fetched, the trajectory of AI development indicates profound implications that society must start addressing. Failing to prepare for such a future could leave humanity in a reactive position, much like the global response to the Covid-19 pandemic, he warned.
As AI continues to evolve, balancing its capabilities for significant achievements with the need to mitigate associated risks remains a critical challenge for technologists and policymakers alike.
