[ExI] Review of AI 2040 Plan

[Adrian Tymes]

And what does it say about the odds of the alternative happening the way
they say it inevitably will?  Far less than the 90-something percent needed
for that to be the guaranteed alternative, right?

On Mon, Jul 13, 2026, 6:50 AM BillK via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

> Quote:
> AI companies are racing to build AIs that are smarter than humans in every
> way. In AI 2027, we predicted that this would result in either extinction
> or irreversible concentration of power.1
> Plan A is our positive vision for what should happen instead.
>
> <https://ai-2040.com/>
> -----------------------------------------
>
> This optimistic plan relies on the USA and China agreeing on a treaty to
> pause AI development before AGI is reached.
> I asked Gemini AI to behave as a forecaster and discuss the feasibility of
> this plan.
> The result was a 2.5% probability of success.
> The full report of why it almost certainly won't happen is below.
> BillK
>
> Gemini 3.5 Flash Extended Thinking:
>
> To evaluate the "AI 2040: Plan A" proposal published by the AI Futures
> Project, we must define the core target event implied by the text: *Will
> the United States and China establish and successfully enforce a verified
> bilateral or multilateral AI governance treaty to explicitly halt, pause,
> or restrict the development of frontier artificial intelligence before it
> reaches superintelligence (ASI) by 2035?*
>
> Here is the hyper-calibrated superforecasting evaluation of this target
> event.
> 1. THE OUTSIDE VIEW (Base Rates)
>
> To establish a statistical baseline, we look at the *Reference Class*: *Bilateral
> or multilateral technology-limitation or arms-control treaties enacted
> between major adversarial superpowers targeting a highly lucrative,
> dual-use, rapidly advancing capability prior to a kinetic catastrophe or
> widespread deployment.*
>
> Over the past 30 years, the baseline for this type of proactive, intrusive
> restriction is exceptionally low:
>
>    -
>
>    *The Successes:* The Strategic Arms Reduction Treaties (START) and the
>    Chemical Weapons Convention (1993) successfully limited stockpiles, but
>    these were signed *after* decades of deployment, visible destruction,
>    or near-miss crises (e.g., the Cuban Missile Crisis), and during a period
>    of geopolitical convergence.
>    -
>
>    *The Failures/Collapses:* More recent history shows a decay in
>    verification frameworks. The US withdrew from the ABM Treaty and the INF
>    Treaty. The Joint Comprehensive Plan of Action (JCPOA) collapsed due to
>    domestic political shifting. The Comprehensive Nuclear-Test-Ban Treaty
>    (1996) remains unratified by key nations, and the Paris Climate Accords
>    rely on non-binding, voluntarily reported targets.
>
> Proactively banning an entirely new paradigm of economic and military
> power before its destructive capacity has been unambiguously demonstrated
> has almost no historical precedent.
>
>    -
>
>    *Mathematical Anchor (Base Rate):* *3% to 5%*
>
> 2. THE INSIDE VIEW (Specific Dynamics)THE BULL CASE (Why the plan could
> succeed)
>
>    -
>
>    *The Physical Chokepoint:* Unlike cyber-weapons or biological agents,
>    frontier AI requires massive physical infrastructure. Roughly 98.5% of
>    advanced AI chips flow through an incredibly consolidated supply chain
>    (ASML for lithography, TSMC for fabrication, NVIDIA for design). This makes
>    hardware-level verification ("trustless chip tracking" via cryptographic
>    firmware locks) technically feasible without requiring intrusive software
>    inspections.
>    -
>
>    *Mutual Threat of Sovereign Loss:* Unlike traditional weapons where
>    one state wins if the other loses, a misaligned superintelligence poses an
>    existential threat to the internal political stability of *both*
>    regimes. The Chinese Communist Party values top-down information control
>    above all else; an autonomous, unaligned ASI is fundamentally incompatible
>    with their governance model.
>    -
>
>    *Elite Alignment:* Elite opinion is shifting rapidly. The fact that
>    former frontier researchers (like the authors of Plan A) and tech
>    executives are actively warning congress indicates that the "overton
>    window" for aggressive regulation is opening faster than historical tech
>    cycles.
>
> THE BEAR CASE (Why the plan will fail)
>
>    -
>
>    *The Geopolitical Prisoner's Dilemma:* Washington and Beijing
>    currently view AI through a zero-sum, national security lens. As noted in
>    the Plan A text, domestic tech lobbyists aggressively push the narrative
>    that any unilateral regulation forces the US to lose the digital Cold War
>    to China. For China, AI represents an asymmetric opportunity to bypass US
>    conventional military and economic superiority.
>    -
>
>    *The Intrusiveness Threshold:* A "verified" slowdown requires
>    unprecedented transparency. Even if chip tracking is used, verifying
>    compliance would ultimately require verifying power grids, datacenter
>    locations, and supply chains. In an era of decoupling and extreme espionage
>    suspicion, neither nation will comfortably allow foreign oversight of its
>    most critical national security infrastructure.
>    -
>
>    *Trillion-Dollar Economic Momentum:* The capital expenditure currently
>    flowing into AI datacenters is unprecedented. Halting capabilities progress
>    at "top human expert level" requires governments to force corporations to
>    walk away from trillions in projected productivity gains.
>
> 3. CONJUNCTIVE CHAIN CHECK
>
> Plan A requires a strict sequence of high-stakes, independent conditions
> to succeed simultaneously. If we break down the necessary chain:
>
>    1.
>
>    *US Alignment:* The US political establishment must pivot from a "race
>    to win" mindset to an "existential risk" framework by 2029.
>    -
>
>       *Estimated Probability:* *35%*
>       2.
>
>    *Sino-American Accord:* China must agree to a bilateral deal that
>    essentially freezes their relative position or subjects their internal
>    computing infrastructure to verifiable tracking.
>    -
>
>       *Estimated Probability:* *15%*
>       3.
>
>    *Technical Deployment:* Tamper-proof, hardware-level cryptographic
>    tracking and verification mechanisms must be successfully developed and
>    deployed globally without backdoor bypasses.
>    -
>
>       *Estimated Probability:* *40%*
>       4.
>
>    *Leakage Prevention:* Third-party actors, rogue states, or
>    black-market hardware recycling clusters must be successfully contained to
>    prevent a compliance defection loop.
>    -
>
>       *Estimated Probability:* *50%*
>
> Joint Probability=0.35×0.15×0.40×0.50=0.0105
>
>    -
>
>    *Conjunctive Anchor:* *~1.05%*
>
> 4. CALIBRATED PROBABILITY
>
> Combining the historical base rate, the extreme friction points of the
> geopolitical Prisoner's Dilemma, and the mathematical decay of the
> conjunctive chain, the final forecast is:
>
> *Final Probability Estimate: 2.5%*
> Confidence Level & Critical Triggers
>
> My confidence in this low probability is high. The game-theoretic
> incentives for defection are too steep, and the historical precedents for
> proactive technological caps are too weak.
>
> The specific data points or news events that would cause this prediction
> to swing by *more than 15%* include:
>
>    1.
>
>    *An Unambiguous, High-Impact AI Catastrophe:* A major national
>    security crisis—such as an autonomous model successfully incapacitating a
>    nation's power grid, executing a multi-billion dollar automated financial
>    exploit, or leaking a highly viable novel pathogen. This would shock the
>    system, bypass the economic incentives, and force a joint US-China crisis
>    management framework.
>    2.
>
>    *An Official Joint Superpower Communiqué:* A formal bilateral
>    declaration following a US-China summit explicitly establishing a working
>    group tasked with creating *binding, verifiable compute caps* with
>    physical enforcement mechanisms.
>
> -------------------------------------------------------------
>
> _______________________________________________
> extropy-chat mailing list
> extropy-chat at lists.extropy.org
> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260713/6d1f41d6/attachment-0001.htm>