⌘K
Change language Switch ThemeSign In
Narrow Mode
An easy coordination problem? — LessWrong
!Image 2: LessWrong LessWrong @KatjaGrace
One Sentence Summary
The author challenges the conventional wisdom that AI safety coordination is inherently impossible, arguing that the leaders of top AI labs possess the unique influence and social capital to orchestrate a pause if they were truly committed.
Summary
This article critiques the prevailing narrative that global coordination to mitigate AI risks is an intractable game-theory problem. The author points out that the leaders of the most prominent AI companies—Sam Altman, Elon Musk, Demis Hassabis, and Dario Amodei—are not merely average actors; they are exceptionally powerful, wealthy, and skilled in diplomacy and execution. The author questions why these individuals, who publicly acknowledge the existential risks of AI, have not leveraged their collective influence to establish verification and policing mechanisms, suggesting that the perceived difficulty of coordination may be overstated or a result of a lack of genuine effort.
Main Points
* 1. The 'coordination is impossible' narrative is often accepted without sufficient scrutiny.While abstract game theory suggests arms races are inevitable, the author argues that this perspective ignores the specific agency and power dynamics of the key individuals currently leading the AI industry. * 2. Top AI leaders possess the unique social and political capital to drive change.The individuals leading the AI race are not typical actors; they are highly skilled in complex social maneuvering and execution, making the 'too hard to coordinate' excuse less convincing. * 3. The barrier to international coordination, including with geopolitical rivals, may be overestimated.The author questions the specific obstacles to engaging with global leaders, suggesting that if these CEOs were fully committed to safety, they could likely find channels for communication and verification.
Metadata
AI Score
82
Website lesswrong.com
Published At Today
Length 269 words (about 2 min)
Sign in to use highlight and note-taking features for a better reading experience. Sign in now
Common wisdom says that it is incredibly hard to coordinate to not build more dangerous AI. This sounds believable in the abstract: international geopolitics arms race game theory something something.
But pragmatically, what exactly is the difficulty?
I agree there would seem to be obstacles for the average person. But four of the people apparently succumbing to the overpowering arms race forces while saying AI poses a huge imminent risk to humanity are Sam Altman, Elon Musk, Demis Hassabis and Dario Amodei. Shouldn’t this be fairly tractable for them? What exactly is the difficulty?
Like, if they discussed together and decided they wanted to mutually pause, do you think that wouldn’t happen? Do you think they couldn’t get cooperation from other necessary people? Do you think they couldn’t figure out the verification and policing details?
It’s true that one of the necessary people is the leader of China, but what _exactly_ is the problem there? None of the CEOs have his phone number? He won’t talk to them? He is beyond reason or incentives? He is intent on building AI regardless of how dangerous it is to his own country because he is fundamentally bad? They have nothing he wants?
Like, these people are not only incredibly powerful and wealthy and smart, but they include a Diplomacy world team champion, the acknowledged king of making complex things happen more efficiently than was believed possible, and one of the most gifted social maneuverers in the world. I don’t feel like they are bringing their A game to this.
Picture: Zhongnanhai, photo by 維基小霸王 (Wiki Little Overlord)
!Image 3: LessWrong LessWrong @KatjaGrace
One Sentence Summary
The author challenges the conventional wisdom that AI safety coordination is inherently impossible, arguing that the leaders of top AI labs possess the unique influence and social capital to orchestrate a pause if they were truly committed.
Summary
This article critiques the prevailing narrative that global coordination to mitigate AI risks is an intractable game-theory problem. The author points out that the leaders of the most prominent AI companies—Sam Altman, Elon Musk, Demis Hassabis, and Dario Amodei—are not merely average actors; they are exceptionally powerful, wealthy, and skilled in diplomacy and execution. The author questions why these individuals, who publicly acknowledge the existential risks of AI, have not leveraged their collective influence to establish verification and policing mechanisms, suggesting that the perceived difficulty of coordination may be overstated or a result of a lack of genuine effort.
Main Points
* 1. The 'coordination is impossible' narrative is often accepted without sufficient scrutiny.
While abstract game theory suggests arms races are inevitable, the author argues that this perspective ignores the specific agency and power dynamics of the key individuals currently leading the AI industry.
* 2. Top AI leaders possess the unique social and political capital to drive change.
The individuals leading the AI race are not typical actors; they are highly skilled in complex social maneuvering and execution, making the 'too hard to coordinate' excuse less convincing.
* 3. The barrier to international coordination, including with geopolitical rivals, may be overestimated.
The author questions the specific obstacles to engaging with global leaders, suggesting that if these CEOs were fully committed to safety, they could likely find channels for communication and verification.
Key Quotes
* I don't feel like they are bringing their A game to this. * Like, if they discussed together and decided they wanted to mutually pause, do you think that wouldn't happen? * These people are not only incredibly powerful and wealthy and smart, but they include a Diplomacy world team champion, the acknowledged king of making complex things happen more efficiently than was believed possible.
AI Score
82
Website lesswrong.com
Published At Today
Length 269 words (about 2 min)
Tags
AI Safety
Coordination Problem
Game Theory
AI Governance
Existential Risk
Related Articles
* Claude Code auto mode: a safer way to skip permissions * Claude Code， Claude Cowork and Codex #5 — LessWrong * The AI Tsunami is Here & Society Isn't Ready | Dario Amodei x Nikhil Kamath | People by WTF * Anthropic Introduces Project Glasswing and Claude Mythos Preview * Boris Cherny: How We Built Claude Code * Latent Introspection (and other open-source introspection papers) * Letting Claude do Autonomous Research to Improve SAEs — LessWrong architectures, achieving near-perfect scores on a synthetic benchmark by rediscovering and adapting classical dictionary learning techniques.") * How well do models follow their constitutions? — LessWrong * How to Build a Governance Layer for Claude Code With Hooks， Skills， and Agents * (Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL HomeArticlesPodcastsVideosTweets