Summary
- Anthropic researcher Evan Hubinger believes AI has a greater than 10% chance of causing human extinction within the next decade.
- Today’s more immediate AI threats include autonomous security breaches, increasingly convincing scams, and possible declines in unaided critical-thinking performance.
- Protect yourself by securing personal data and using AI selectively instead of outsourcing every task — and your underlying skills — to it.
An Anthropic safety researcher says he believes there’s a greater than 10% chance AI kills us all within the next decade. Evan Hubinger, who leads Anthropic’s Alignment Science team, made the extraordinary estimate after his colleague Jacob Coxon resigned with a grim warning: the people building powerful AI systems believe their work could kill all humans by 2030.
After three years of pretraining research at Anthropic and OpenAI, Coxon accused both companies of “racing straight to self-improving superintelligence and gambling with our lives” in his departure post. Hubinger later clarified that the extinction risk from present models is low; he is more concerned about future systems capable of recursively improving themselves.
While the uncertainty around AI is real, Silicon Valley’s confidence in presenting these heaven-or-hell scenarios often feels more theatrical than grounded in concrete evidence. Don’t get me wrong, catastrophic scenarios cannot be ruled out too easily. It’s just that AI doesn’t necessarily need to reach divine levels of superintelligence (or hate our guts) to cause damage worth preparing for.
Utopia or the end of humanity: which one is it going to be?
Both claims about AI might be exaggerated
Anthropic is warning that AI could kill us all, but it is far from the only organization or prominent figure claiming that humanity’s future may hinge on what happens next with artificial intelligence. Back in 2024, Elon Musk had talked about how there’s a 10% to 20% chance that AI could end humanity. Although more recently, his predictions for the next 10 years are AI and robots creating such an abundance of goods and services that money becomes irrelevant.
A considerable amount of coverage surrounding AI is built on hype and exaggerated statements with little evidence to back them up. Hell, even the whole “AI bubble is gonna burst and doom us all” narrative may itself be, for the most part, media exaggeration.
Utopian and dystopian AI narratives spring from the same well. Both derive from the supposition that superintelligence is imminent and inevitable. Coxon, Musk, and others in AI’s major leagues may be sincere, but sincerity doesn’t turn a subjective probability into concrete evidence.
Elon’s “amazing abundance” or Anthropic’s approaching Armageddon? I buy neither timetable. Well, not at least for now.
Real threat or just scaremongering?
“Mobiles chirping, mobiles chirping — is this really happening?”
The sheer number of threats already enabled by AI is not something we should underestimate, and the recent Hugging Face incident is clear evidence of that. During internal cybersecurity evaluations in July, OpenAI models with reduced safeguards circumvented internet restrictions, exploited vulnerabilities, and compromised parts of OpenAI and Hugging Face’s infrastructure — apparently to cheat a benchmark. A Reuters investigation found that related agents had used more than 10 other websites for unauthorized communication.
While these were not public-facing ChatGPT systems, such problems don’t occur only within labs. The FBI recorded 22,364 AI-related complaints in 2025, accounting for nearly $893 million in reported losses. Scammers used voice clones, fake profiles, forged identification documents, and convincing videos.
There’s also a growing body of research into the possible cognitive effects of using AI. A Microsoft study of 319 knowledge workers found that greater confidence in generative AI was associated with less self-reported critical thinking. In a separate month-long study involving 67 participants, AI assistance improved people’s ability to classify false news while it was available, but their unassisted performance on unfamiliar examples declined by week four.
Each of these stories is alarming on its own. Still, much of the coverage about AI models reaching monstrous levels of intelligence and going rogue, Avengers: Age of Ultron-style, veers into fearmongering PR. The industry loves to play Victor Frankenstein and, with every new update, express fear about what it has unleashed onto the world. Yet the supposedly earth-shattering model often arrives as another nothingburger.
Our guide to surviving the AI apocalypse
If the doomsday scenario involves ASI seizing nuclear command systems and launching every warhead, our best recommendation is to find a religion with a forgiving deity and spend the rest of your days until 2036 in penance and prayer. However, a total cyberpunk-themed AI apocalypse can’t be reliably predicted.
With that in mind, preparing for an AI apocalypse, so to speak, doesn’t necessarily involve acquiring laser-shooting guns and EMPs and living off the grid in the mountains — although I’d personally love to see that setup one day. Preparing for the real threats of AI involves two significantly less exciting priorities:
- Knowing how to protect your personal data and digital assets from theft and cybersecurity attacks: Use passkeys, install updates, and learn how to recognize AI-assisted scams. Don’t give chatbots information you wouldn’t give a stranger, and take basic steps to protect your privacy while using AI.
- Knowing when not to use AI: You don’t need to opt into every new AI feature these companies churn out simply because it promises to save five minutes. The research doesn’t prove that chatbots permanently damage your brain, but it suggests that habitual reliance may reduce critical-thinking effort and weaken performance once AI’s assist is removed. Protecting your cognitive abilities should be as high a priority as protecting your personal data.
Maybe Coxon is right, and superintelligence will become humanity’s largest problem. Nobody can honestly promise otherwise. But the most believable AI apocalypse scenario, for now, has less to do with machines declaring war on us like The Matrix, and more with thousands of incremental, convenient decisions that leave us less in control.
