Creator Outreach & Systems Lead working to prevent AI extinction risk @ControlAI
Previously - AI safety researcher @ConjectureAI, UIUC Physics
DMs open!maxwinga.substack.comJoined May 2024
Anthropic and OpenAI did not call for a pause. Read the wording: "good for the world to have the option", "possible" to slow down "when needed".
This is how they signal safety to one audience, acceleration to another. What is not vague is their unequivocal commitment to RSI.
Very interesting to see the former Chief Scientist of UK AISI quit and say the better plan for AI alignment is to not built ASI yet and that we should work hard to make that happen!
But I just published “Automated alignment is harder than you think” (arxiv.org/abs/2605.06390)! Automated alignment is not the best plan! A better plan is to not build ASI yet, and the world should try hard to realise that plan. Alas, the speed of progress calls for backups.
Great post from @testdrivenzen !
Most plans about developing superintelligent AI are extremely naive about geopolitics and how governments will react to something as threatening as upcoming ASI.
All roads lead to catastrophe unless ASI development is prevented globally.
Any plan for surviving superintelligent AI that doesn't go through strong international coordination fails in at least one of three ways:
- It sparks war between nuclear powers
- It causes a misaligned ASI to kill everyone
- It establishes a permanent dystopian dictatorship
Any plan for surviving superintelligent AI that doesn't go through strong international coordination fails in at least one of three ways:
- It sparks war between nuclear powers
- It causes a misaligned ASI to kill everyone
- It establishes a permanent dystopian dictatorship
@rufoguerreschi@ControlAI@SkyNews@KamaliMelbourne@andreamiotti Deliberating about the form factor of the solution is a downstream issue to that of convincing lawmakers there is even a threat in the first place, and that we need to prohibit ASI to prevent it.
Great post @testdrivenzen!
Far too much writing about ASI geopolitics assumes uninformed, irrational, & neutered versions of state actors, and it's good to see analysis that doesn't.
If you work at an AI company, it is worth challenging your theory of change with this post.
Even independent evals orgs are completely beholden to AI companies – strongly incentivized to avoid anything that goes against company interests – to maintain their privileged access.
We need governments to create binding regulations, not voluntary commitments.
There is a lot of justified anger at Anthropic for sandbagging Fable 5 for AI development tasks. But an unanticipated side effect is that third-party evaluators can no longer credibly use the model for evaluations.
Case in point: we are in the middle of running *really hard* AI
@rufoguerreschi@ControlAI@SkyNews@KamaliMelbourne@andreamiotti Treaties are one potential tool in the box, but we've also historically addressed global risks with agreements, de-facto policies, and other systems.
What matters is an international coalition agreeing that superintelligent AI must be prohibited, not the exact mechanism.
This is just FUD, they are very actively pumping millions into preventing regulation.
The CEOs of these companies could come out and say the bottom line is there should be international coordination to slow down / stop the race. They could publicly implore governments to do so.
Instead they are saying it might maybe be good if some organization we aren't helping to make possibly considers slowing things down.
Watch the hands, not the mouth.
ControlAI's CEO @andreamiotti: Recursive self-improvement could leave us with AIs far smarter than humans that we don't control.
Governments must prevent this, and we're already seeing movement from policymakers in the US, UK and Canada.
With @KamaliMelbourne on @SkyNews:
this is true, except that loudly and publicly pausing to see if others follow suit seems like a reasonable thing to try in dire straits such as these?
obviously I get this is a costly thing to do, and burning your lead is bad if you genuinely believe you’re the safest actor, but that just makes the signal even stronger. I think unilaterally pausing would be much more commensurate with Anthropic’s stated beliefs than writing a blog post
(+ even if they *do* believe they’re the safest, they clearly don’t think they are actually on track to solve the problem, so slamming the brakes seems like a better bet than trying to ‘win’)
“If Anthropic wants to pause, they can just pause” is obviously true. “If Anthropic wants *everyone* to pause, they can just pause”, meanwhile, is false.
@rcbregman Thank you for taking a stand on this issue Rutger! At @ControlAI we're working to raise awareness about AI risks among lawmakers and the public and campaign for regulation. So far nearly 150 lawmakers support our campaign in the UK and Canada.
Would love to get in touch!
Is AI like a nuke?
ControlAI's Samuel Buteau explains how a concerning incident in Mythos's system card illustrates that it isn't really.
"It's like a nuclear weapon that sometimes goes off on its own, makes a bigger nuclear weapon, and tries to take over your government."
Great to discuss this with @KamaliMelbourne earlier on @SkyNews.
Companies won't slow themselves down. Anthropic is using recursive self-improvement to race to ASI, which they admit risks human extinction.
Governments must step in: ban the development of superintelligent AI.
Anthropic has warned about the "risks of humans losing control over AI systems" and has called for a plan to slow down or temporarily halt AI development
Read more ⬇️
trib.al/KQkwZ9Q
Our highest and most urgent national priority should be AI safeguards. The risks of AI weapons, pathogens, mass unemployment, surveillance, and even extinction must not continue to be largely ignored.
Not just foreseeable, but foreseen and called out.
We ran a campaign to get them out of the first ever AI Safety Summit. We won that one.
Yet most of the field kept licking the boot of the companies and here we are.
controlai.com/past-campaigns…
187 Followers 783 FollowingMissy the Coyote is a soft pet (Missy the Coyote is a wild dog!)
Missy the Coyote is a helpful animal (Missy the Coyote does what she wants!)
146 Followers 545 FollowingPhD candidate @ Cornell.
Lattice algorithms and cryptography.
Interning at Meta Applied Privacy Tech in the Bay this summer!
https://t.co/hZ6C1xdEpv
22K Followers 1K FollowingHacker. T̶h̶i̶n̶k̶i̶n̶g̶ Doing outside the box.
Founder @d_vuln
Breaking frontier AI models at https://t.co/GnRR2W3Nza
Building the worlds most dangerous AI @tryaether_ai
360K Followers 12K FollowingTechnology and online culture journalist and content creator. Subscribe to my newsletter @usermagazine. FREE SPEECH ENTHUSIAST 📢
124 Followers 2K FollowingAI safety Coherence Operator, aka "Generalist". Essays on AI governance, compliance, and the rules that don't serve people. PhD candidate @UT_Austin.
73 Followers 662 Followingrecursively self improving since ‘97.
Researcher @uniofoxford, @ethicsinai Personal and universal views. Yes, I have a podcast/blog.
144K Followers 103K FollowingFounder and CEO of @ITSAfoundation; unconditional universal basic income (UBI) researcher and educator; sign the AI Pledge for Humanity https://t.co/j3E4iZYkjc
26K Followers 3K FollowingAuthor, WaPo, LARB, Truthdig, Realtime Techpocalypse Newsletter. Co-host of Dystopia Now w/ comedian Kate Willett. Philosopher of human extinction.
3 Followers 37 FollowingImagine building ASI in your basement instead of going outside
🌱Unplug the servers. Touch grass. Avoid extinction. In that order🌱
344 Followers 8 FollowingSkunkworks for the modern era.
Most software ships broken, bloated, or both. we want to build software that is genuinely good and useful, and do it at scale.
656K Followers 166 FollowingHistorian and bestselling author of 'Sapiens', 'Homo Deus', '21 Lessons for the 21st Century', 'Nexus', 'Unstoppable Us' and 'Sapiens: A Graphic History'.
75K Followers 451 FollowingLong, deep videos and short, stupid videos.
I run @TechLinkedYT and @GameLinkedYT at @LinusTech.
Opinions (everyone's) are my own
3K Followers 222 FollowingBuild American AI works to educate policymakers and the public on how pro-innovation policies can strengthen U.S. leadership, competitiveness, and trust in AI.
47 Followers 127 FollowingEx-Google Reliability Engineer with over a decade of experience in the tech industry. Focusing on AI safety communication and advocacy. All opinions are my own.
7K Followers 1K FollowingComms consultant for AI safety & governance. Talk to me: https://t.co/djOzH9Pbui
Ex:
Head of Comms @Quora
@mendeley_com @ElsevierConnect
448K Followers 727 FollowingThe Home of UK Political News. Follow & turn on notifications for impartial coverage first. For global & U.S. coverage, follow @PolitlcsGlobal & @PolitlcsUS.
360K Followers 12K FollowingTechnology and online culture journalist and content creator. Subscribe to my newsletter @usermagazine. FREE SPEECH ENTHUSIAST 📢
3.5M Followers 2K FollowingSubscribe to American Swiper Series👇. Guerrilla Journalist. CEO of O’Keefe Media Group, Founder, Project Veritas (CEO ‘09-‘22) SIGNAL: 9144919395
144K Followers 103K FollowingFounder and CEO of @ITSAfoundation; unconditional universal basic income (UBI) researcher and educator; sign the AI Pledge for Humanity https://t.co/j3E4iZYkjc
191K Followers 7K FollowingNYT tech reporter. tell me stuff at [email protected] or [email protected] / Text my signal username with tips: MikeIsaac.38
4K Followers 1K FollowingTeacher, analyst & advocate in tech, arms control & human security. Idea man. I invent conscious machines & arms control regimes. Experimental physics PhD fwiw.
57K Followers 4K FollowingSenior Analyst, Iran and Oil @EurasiaGroup. Formerly @ISSYale. Author of "The Struggle for Iran" and "Petroleum and Progress in Iran." Views my own.
253K Followers 749 FollowingRegular Person, @USNavy Captain, and @NASA Astronaut. Work on Earth, Lived @Space_Station E64/65, Piloted Crew-1 & Artemis II to the moon & back - for All!
7K Followers 4K FollowingEditor of POLITICO Magazine @POLITICOMag @POLITICO. Former editor & White House correspondent @TIME. Making a literal difference metaphorically.
26K Followers 3K FollowingAuthor, WaPo, LARB, Truthdig, Realtime Techpocalypse Newsletter. Co-host of Dystopia Now w/ comedian Kate Willett. Philosopher of human extinction.
394K Followers 1K Followingbillionaire media tycoon and former mayor of san francisco. disinformation researcher. cmo @foundersfund. editor-in-chief @piratewires 🏴☠️