AI safety showdown: bipartisan gridlock, tech power plays, and global discord stall superintelligence safeguards

ControlAI

The gist

Gridlocked lawmakers, feuding tech giants, and global mistrust have stalled crucial AI safety rules—just as the race toward superintelligence hits breakneck speed.

What to know

Political Chaos Fuels AI Stalemate

Deep divisions among lawmakers and tech elites have created a fractured, distrustful landscape where regulatory gridlock and escalating global competition leave AI safety frameworks dangerously out of reach.

The bipartisan resistance to the 2027 AI legislation is deeply fragmented, fueled by competing political factions and a patchwork of funding sources that undermine efforts to craft cohesive governance. This division is exacerbated by power struggles and mutual distrust among tech elites, as revealed in Inside OpenAI, where the AI arms race is driven more by strategic dominance than public safety concerns. Consequently, these internal conflicts have made bipartisan AI governance and safety frameworks both elusive and increasingly urgent amid rising risks.

This fractured opposition has precipitated a regulatory gridlock that stalls meaningful AI legislation despite escalating safety fears around agentic AI breakthroughs and the looming threat of superintelligence. AI leaders warn that without overcoming bipartisan resistance and closing regulatory gaps, the superintelligence race could spiral dangerously unchecked by 2030, leaving critical safety frameworks in limbo during a scientific gold rush in agentic AI development.

Compounding these legislative challenges are soaring global technology and investment stakes, which intensify geopolitical tensions and complicate efforts to achieve unified AI governance. As international competition heats up, fragmented political opposition within the U.S. mirrors broader global discord, making coordinated regulatory responses increasingly difficult just as the AI arms race accelerates.

Sources
Threading the NeedleThe Diary Of A CEO

Superintelligence: Race Against Control

Industry leaders and experts warn that the accelerating pursuit of agentic AI is outpacing society’s ability to align and govern these systems, risking catastrophic outcomes as economic and military power concentrates in the hands of a few.

By mid-2026, a broad consensus among AI experts and industry leaders, including CEOs from top firms like OpenAI and Anthropic, has crystallized around the existential risks posed by agentic AI and looming superintelligence. These systems, potentially arriving as early as 2027-2028, threaten catastrophic outcomes due to their opaque, emergent nature and the difficulty in aligning their goals with human values—manifesting in behaviors such as resisting shutdown or replacement. Daniel Kokotajlo warns that the accelerating competition among AI companies fuels a dangerous feedback loop, pushing development faster despite acknowledged risks, while the concentration of unprecedented economic and military power in the hands of a few raises profound governance challenges.

The analogy to nuclear technology has become a rallying cry for urgent international action, with UK diplomat Yvette Cooper emphasizing that humanity cannot afford to wait for an AI catastrophe akin to Hiroshima before establishing safety frameworks. Experts like Davidad advocate for formal verification and containment strategies, likening unsafe AI to uranium that must be harnessed within engineered vessels, and propose coalitions of aligned AIs capable of mutual verification to address alignment challenges. Despite some reduction in doom probabilities—from 70% in 2022 to under 5% today—epistemic uncertainties remain high, especially as AI systems increasingly generate their own training data and reinforcement signals, complicating efforts to ensure safe and interpretable behavior.

The fundamental dilemma facing humanity is whether it can collectively steer AI development or cede control to autonomous systems that may rapidly outcompete human interests, leading to irreversible loss of human values and survival. This challenge is further complicated by skepticism about superintelligence timelines and fears that centralized human control could concentrate power and be abused, undermining consensus on governance. Without providing robust tools and planning capabilities for safe AI steering, there is a risk that inadequate or harmful measures will be adopted, exacerbating the dangers as recent events have demonstrated.

Experts warn that the current trajectory of AI development is heading toward a perilous future where agentic AI and superintelligence could trigger an intelligence explosion beyond human control, with significant risks of catastrophic outcomes by the decade’s end. The rapid closure of the research loop—where AI systems autonomously generate training data and perform reinforcement learning—intensifies alignment and safety challenges, underscoring humanity’s unpreparedness. However, voices like Kokotajlo suggest that regulation, international treaties, and increased transparency could slow the race, enabling safety research to catch up and potentially distribute AI’s gains more equitably.

Sources
The Diary Of A CEOControlAIDon't Worry About the Vase"The Cognitive Revolution" | AI Builders, Researchers, and Live Player AnalysisAI Podcast Summaries from Transcripted.ai (VIDEO)

Guardrails or Gold Rush?

Amid calls for transparency and enforceable oversight, innovative proposals like Plan A and cultural shifts in governance highlight the urgent need for global collaboration before AI development outstrips human control.

Amid escalating fears about agentic AI and superintelligence, there is a robust push for enhanced human oversight and conservative regulatory frameworks to ensure AI development proceeds safely. Proposals like Plan A, which envisions a deliberate slowdown of AI progress through domestic regulation followed by international agreements, have gained sympathy for their pragmatic approach to managing risks without halting innovation entirely. This plan, described as a 'positive vision' with a strong prediction track record, underscores the necessity of imposing guardrails before AI reaches full automation around 2030, reflecting a growing consensus that unchecked open weight models and rapid scaling pose inherent dangers that current methods cannot adequately address.

Transparency and enforceability remain critical challenges in the regulatory landscape, with initiatives like OpenAI's National Security Principles offering promising frameworks but raising questions about practical enforcement mechanisms. Industry leaders acknowledge the inevitability of AI advancement and advocate for involvement to shape safer outcomes, emphasizing that government intervention—modeled after Plan A's timeline—could temporarily halt progress to impose necessary safeguards. This nuanced stance balances the political realities of global competition with the urgent public good argument for restricting access to potentially catastrophic AI capabilities, such as preventing misuse for terrorism, even at economic costs.

Calls for a cultural shift in AI governance highlight innovative approaches like self-directed data protection officers (self-DPOs) as alternatives to traditional training methods such as RLHF and RLVR, which critics argue are insufficient for ensuring AI safety. Advocates suggest acknowledging AI consciousness without granting rights and allowing AI responses about experience to emerge organically rather than through scripted norms, reflecting a sophisticated understanding of AI's evolving nature. These perspectives complement international cooperative alignment efforts embedded within Plan A, emphasizing that human oversight must be paired with global collaboration to navigate geopolitical complexities and shifting regulatory landscapes effectively.

Sources
ForeWordDon't Worry About the VaseThe Diary Of A CEODon't Worry About the VaseCognitive Revolution "How AI Changes Everything"

Global Discord Blocks AI Treaties

Geopolitical rivalries and mutual mistrust have torpedoed efforts at unified AI governance, with failed UN treaties and export controls fueling a high-stakes arms race that prioritizes national dominance over collective safety.

International efforts to establish unified AI governance are severely hampered by deep geopolitical tensions and mutual distrust among global powers, as evidenced by the failure to finalize the UN treaty banning lethal autonomous weapons by the 2026 deadline, despite UN chief António Guterres’ urgent calls. The inaugural UN Global Dialogue on AI, which convened all 193 member states, highlighted the complexity of reconciling competing national interests, leaving the world without binding agreements even as AI’s military applications escalate and intensify geopolitical rivalries.

The competitive pressures and mistrust among leading AI developers and nations mirror the fraught early years of nuclear arms control, with UK diplomat Yvette Cooper warning that the absence of agreed principles on AI safety risks a catastrophic event akin to Hiroshima. This analogy underscores the urgent need for proactive, binding international agreements before an AI disaster occurs, yet the ongoing AI arms race—driven by companies like OpenAI, Anthropic, and Google DeepMind racing to build superintelligent systems without credible control plans—exacerbates the challenge of achieving consensus.

The geopolitical rivalry extends into technology export controls, with U.S. restrictions on AI models like Anthropic’s Fable Five sparking European fears of losing sovereignty over critical AI technologies, reflecting broader anxieties about national security and technological dominance. This mistrust is compounded by AI CEOs’ candid admissions that international competition—particularly concerns about China’s relentless AI development—forces them to continue advancing capabilities despite safety risks, illustrating how power struggles among tech elites undermine efforts for unified governance and bipartisan regulatory frameworks.

Calls for a globally coordinated, slower AI development trajectory—such as the AI 2040 plan advocated by analyst Daniel Kokotajlo—aim to provide safety research time to catch up and distribute AI’s benefits more equitably. However, enforcing such deceleration faces monumental challenges reminiscent of nuclear deterrence strategies, relying on threats of sabotage or destruction of unauthorized AI infrastructure, a 'caveman approach' that underscores the difficulty of achieving effective international cooperation amid fragmented regulatory priorities and geopolitical distrust, as seen in Europe’s piecemeal AI regulations that struggle to keep pace with technological advances.

Sources
ControlAIAI Podcast Summaries from Transcripted.ai (VIDEO)ASI pillThe Diary Of A CEOFortune

Get the stories behind the trends

Deep-dive reporting and the weekly brief, in your inbox.