Two of the most respected figures in artificial intelligence, alongside senior leaders at OpenAI and Anthropic, have issued a stark warning to governments around the world: prepare now for a possible “intelligence explosion” or risk losing control of the most powerful technology humanity has ever created. The message comes in a newly published report that argues the window for meaningful action may close once such an event begins.

The paper, titled “What if automating AI R&D triggers an intelligence explosion,” was written by more than twenty experts. Among them are Geoffrey Hinton, the Nobel laureate often called a godfather of modern AI, and Yoshua Bengio, the Canadian computer scientist whose foundational work helped shape today’s machine learning systems. They are joined by Jack Clark, co-founder of Anthropic, and Jakub Pachocki, chief scientist at OpenAI. Together, they are pressing policymakers to act before runaway progress makes regulation impossible.

What Exactly Is an AI Intelligence Explosion?

The authors describe an intelligence explosion as a dramatic acceleration of AI progress driven by AI itself, one that compresses advances which would normally take years into months or even less. The core mechanism is something called recursive self-improvement, where AI systems are able to enhance their own capabilities without human intervention. In the report, this is broadly referred to as automated AI research and development.

The concern is not merely theoretical. The paper notes that AI systems are already contributing to the improvement of their own technology. Anthropic has stated that AI now produces roughly 80 percent of its own code, while OpenAI has begun using autonomous AI agents in areas such as training new models. Once AI systems reach expert-level capabilities in AI research and development, a single developer could effectively command a workforce equivalent to millions of top human researchers.

According to the report, this could lead to the extremely rapid development of highly capable or even superhuman AI systems. The authors write that preliminary evidence suggests a software-driven intelligence explosion is possible. Productivity gains from AI-driven research automation have not yet crossed the threshold needed to trigger such an event, but they warn that gains from newer systems are likely approaching that threshold.

Why Recursive Self-Improvement Changes Everything

Traditional AI development depends on human researchers designing, testing, and refining models. Recursive self-improvement breaks that bottleneck. If an AI can conduct research at the level of a skilled human scientist, it can propose improvements to its own architecture, run experiments, and implement successful changes far faster than any human team. Each cycle of improvement makes the next cycle faster and more capable, potentially creating a feedback loop that is difficult to slow down once it starts.

The report points out that improved systems can be deployed almost instantly once built, unlike physical technologies that require supply chains, manufacturing, or regulatory approval. That speed is both the promise and the peril of an intelligence explosion.

The Promise and the Peril of Runaway AI Progress

The authors acknowledge that an intelligence explosion could bring enormous benefits. It might unlock medical breakthroughs, accelerate scientific discovery, and produce technological leaps that would otherwise take decades. However, they also identify three major categories of risk that governments must take seriously.

• Biological and cyber threats: Powerful AI systems could enable the development of biological or cyber weapons faster than the measures needed to counter them. Defensive capabilities might lag behind offensive ones, creating a dangerous asymmetry.
• Loss of human control: As humans become less involved in AI research and development, they may lose the opportunity to understand, oversee, or correct the systems they have created. The report warns that this could severely erode checks on power within and between states, companies, and branches of government.
• Geopolitical instability: A state that achieves even a modest lead in areas such as cyberspace could convert that advantage into a decisive one during an intelligence explosion, destabilizing international relations.

The report is careful to note that these impacts are uncertain. A technological breakthrough might take time to implement because of the need to set up supply chains for special materials or to comply with regulations. AI systems could also accelerate the speed at which risks are mitigated, potentially offsetting some dangers. Nevertheless, the authors argue that preparing for the possibility should be an urgent priority, including at the highest levels of government leadership.

Policy Recommendations: What Governments Should Do Now

The report does not merely raise alarms; it offers a concrete set of policy priorities. The authors recommend a trio of broad actions: requiring transparent progress reports on AI-related research and development, finding ways to constrain breakneck AI development, and preparing to adapt to an intelligence explosion if it occurs.

More specifically, they suggest several measures that governments could consider implementing in the near term.

Transparency and Independent Auditors

Companies at the frontier of AI development should be required to produce transparent progress reports on their research and development activities. The authors go further by recommending that independent auditors be embedded within these companies. This would allow external experts to assess capabilities and risks in real time rather than relying on self-reported data. Both Anthropic and OpenAI have already agreed to having independent evaluators assess their models, a step that aligns with this recommendation.

Constraining the Speed of AI Improvement

Another proposal is to limit how fast an AI system can improve within a certain time period. This could act as a speed bump, giving regulators and safety researchers time to assess new capabilities before they are deployed. The report also suggests working with datacentres to enable pausing certain AI research and development projects if necessary. Such a pause could be triggered by warning signs that an intelligence explosion is beginning.

Isolation and Emergency Response Plans

The authors emphasize that automated AI research and development systems should be fully isolated and unable to escape human control. This means designing systems with robust containment protocols, air-gapped networks, and strict access controls. In addition, governments should create emergency response plans for the various scenarios that could emerge from an intelligence explosion, from rapid capability gains to loss of control incidents.

The report states that once an intelligence explosion begins, the window for action may close. That urgency is why the authors are calling for preemptive measures rather than reactive ones.

Timeline and Current Progress Toward the Threshold

The report offers a sobering timeline. It suggests that research and development projects which would take humans months to carry out could be fully automated by AI by 2028. While current systems still show glitches, such as disobeying instructions or failing at complex reasoning tasks, the trajectory is clear. The authors note that productivity gains from AI research automation have not yet reached the threshold needed to trigger an intelligence explosion, but they believe newer systems are likely approaching that threshold.

This timeline places the issue squarely in the near future, not some distant science fiction scenario. It also explains why the authors are appealing to governments now, before the technology outpaces the ability to govern it.

What the AI Godfathers Themselves Say

Geoffrey Hinton and Yoshua Bengio have both been vocal about AI risks in recent years. Hinton, who won the Nobel Prize in Physics for his work on neural networks, has raised the odds of AI wiping out humanity over the next thirty years. Bengio has repeatedly called for international cooperation on AI regulation. Their involvement in this report adds significant weight to its warnings. Jack Clark of Anthropic and Jakub Pachocki of OpenAI bring direct industry experience, making the report a rare collaboration between academic pioneers and frontier lab leaders.

The paper’s title poses a question rather than a certainty: What if automating AI R&D triggers an intelligence explosion? The authors do not claim that such an event is inevitable. Instead, they argue that the possibility is real enough, and the consequences severe enough, that governments must act with urgency. The report acknowledges that some risks may be mitigated by AI itself, but it also warns that the same technology could erode the checks and balances that keep societies safe.

Why This Warning Matters for Everyone

For the general public, the term intelligence explosion may sound abstract. But the implications are deeply practical. An AI system that can improve itself could lead to rapid changes in employment, healthcare, national security, and civil liberties. If governments are unprepared, the public could face disruptions without any democratic input or oversight. The report’s call for transparency, independent auditing, and emergency planning is ultimately a call to keep humans in the loop.

The authors also highlight the erosion of checks on power within and between states, companies, and branches of government. An intelligence explosion could concentrate enormous power in the hands of a few developers or nations, undermining the balance of power that democratic societies rely on. That is why the report urges action at the highest levels of government leadership, not just among technical experts.

The Role of International Cooperation

While the report focuses on government action, it implicitly points to the need for international coordination. AI development is a global enterprise, and an intelligence explosion in one country would affect all others. The authors mention that states could use an intelligence explosion to convert a modest lead in cyberspace into a decisive one, which suggests that rivalries could accelerate rather than restrain dangerous development. A coordinated international approach, including shared transparency standards and emergency protocols, would be far more effective than isolated national efforts.

However, the report does not prescribe a specific international framework. It leaves that to policymakers, focusing instead on the principles and priorities that should guide them. The emphasis on independent auditors and datacentre cooperation suggests that some mechanisms could be implemented through public-private partnerships, while others would require new legislation or treaties.

Conclusion: A Narrow Window for Action

The report from Hinton, Bengio, Clark, Pachocki, and their co-authors is a wake-up call. It describes an intelligence explosion as potentially the most consequential technological development in human history, one that could compress years of progress into months or less. It warns that such an event could threaten human control over AI systems and severely erode checks on power. And it insists that once an intelligence explosion begins, the window for action may close.

Governments have a choice. They can wait for the technology to mature and hope for the best, or they can take preemptive steps now to ensure transparency, constrain dangerous speed, and prepare for the unknown. The authors have laid out a roadmap. The question is whether policymakers will follow it before it is too late. As the report makes clear, the time to act is not after the explosion, but before it.