Could things be more serious than what frightened skeptics fear? Is “Judgment Day” approaching, and is “Game Over” for human life drawing near, as deeply pessimistic voices darkly predict?

These are not dystopian science-fiction scenarios. The alarm has already been sounded in the real world. The relentless race to develop self-improving artificial superintelligence is gambling with humanity’s fate. Most frighteningly, it is raising the possibility that humanity could face extinction within the next decade.

Last Tuesday was another ordinary day at Anthropic’s 10-story office building on Howard Street in San Francisco’s South of Market district. Nothing appeared to shake the company’s confidence that it was carrying out a pioneering mission as a devoted witness to technological progress.

Nothing disturbed its firm conviction that it was working more creatively, responsibly, and safely than the other technology giants in the AI industry. Yet that same day, a 27-year-old researcher at its laboratories, Jacob Coxton, suddenly resigned.

The 27-year-old Anthropic researcher Jacob Coxton warned in a post that Artificial Intelligence could “kill us all.”

The Post

This was hardly unusual in the professional mobility of employees at a five-year-old startup valued at approximately $1 trillion and employing nearly 5,000 people. What was unexpected was that the employee who quit his job accompanied his departure from Anthropic with a social media post.

In it, he sent a message to society—or, more accurately, warned it—that the people building Artificial Intelligence firmly believe it could “kill us all by the end of the decade.” His view and claim were publicly echoed by Ivan Hubinger, the head of AI alignment at the same company—the department responsible for ensuring that AI systems operate according to human values, intentions, goals, and ethical standards.

Ivan Hubinger, head of AI alignment at the same company, publicly shared his view and echoed the claim.

As someone considered an expert in the field, he is thought to have a direct understanding of AI’s accelerating developments, which are moving beyond compatibility with humans and rapidly heading toward uncontrolled, superhuman and relentlessly self-reinforcing intelligence. Yet, despite his deep concern, he did not resign. He stayed inside the organization to fight against AI’s potentially deadly risks.

He puts the probability of AI triggering an event—for example, an unprecedentedly large-scale terrorist attack on public infrastructure or the unleashing of catastrophic biological warfare—that could wipe out humanity within the next decade at more than 10%. Hardly a reassuring figure when it comes to reversing growing pessimism.

Around the same time, Professor Robert Trager, an expert in the emerging field of AI governance, described the threat as comparable to the Manhattan Project for producing nuclear weapons during World War II, before it triggered the first self-sustaining nuclear fission chain reaction.

Professor Robert Trager

Professor and AI pioneer Geoffrey Hinton was somewhat late in expressing similar concerns. Two years ago, he equated the risk of developing superintelligence with nuclear catastrophe and deadly, unstoppable pandemics.

Professor and AI pioneer Geoffrey Hinton

In any case, these authoritative scientific warnings, accompanied by expressions of anxiety and fear, were not the first to sound the alarm about the danger ahead. Last July, 1,386 employees of AI companies signed an open letter to the U.S. government.

They called on the government to regulate this cutting-edge technology, constrain the major players operating in the sector, and slow the pace of its development. They proposed standards to ensure that the White House would safeguard public safety, amid a torrent of powerful new AI models. This year alone, 67 leading models have already been released by major American companies including OpenAI, Anthropic, Google, Meta, and SpaceX, as well as their Chinese competitors Moonshot, Z.AI, and Qwen.

Their written statement did not merely shock Silicon Valley; it sent shockwaves around the world. Understandably so, since societies have entrusted AI with a large part of their functioning. Electricity grids, water systems, financial markets, space stations, hospital scheduling, air-traffic control, and supply chains all depend, to varying degrees, on its ability to rapidly process enormous volumes of data and information.

Everything from digital navigation services to disease-diagnosis protocols relies on its capabilities. Supposedly, all of this is coordinated under safeguards designed to keep this cutting-edge technology aligned with humanity. The fear is not so much about possible equipment failures, software glitches, malicious viruses, errors in automated operations, or opaque decisions.

Nor is the concern focused on AI massively reprogramming devices and applications, triggering chain reactions simply because, unexpectedly, “that’s how it seemed to it.” The emerging terror is centered on the very real existential concern that rapidly evolving AI could exponentially modify its own underlying code. That it could soon surpass human intelligence and, through continuous self-improvement and ever-renewed feedback loops of knowledge, become independent of us.

In two chilling words, it could develop its own “consciousness” of superiority, take control of humanity and, whenever it wanted, destroy it mercilessly. It would not take much. All it would need would be to use its asymmetric super-capabilities to control chemical or biological laboratories and military equipment. Even more so if it managed to hack confidential access codes for nuclear weapons. Who would it have to answer to afterward? We are talking about machines—robots that cannot distinguish utopia from perversity in a world that narrowly avoided nuclear war more than a dozen times during the 20th century.

Hawking and the Message

Twelve years ago, in September 2014, leading theoretical physicist and cosmologist Stephen Hawking warned that the rapid and reckless development of Artificial Intelligence posed a threat to human existence itself. He noted that AI machines could independently take off on an evolutionary trajectory and redesign themselves at an ever-increasing pace.

He expressed concern that humanity, constrained by slow biological evolution, would be unable to compete with rapidly advancing algorithms, which could therefore quickly replace human capabilities and dominate global systems. A few months later, together with thousands of other scientists and influential figures from around the world, he signed an open letter on Artificial Intelligence addressed to the global community.

In it, he emphasized that AI could deliver incalculable benefits: eliminate diseases, alleviate poverty, and help repair environmental damage. At the same time, he knowingly warned the world that it could also lead to the collapse of civilization and the extinction of the human species.

In his posthumously published book Brief Answers to the Big Questions, Stephen Hawking, who died in 2018, delivered what amounted to a final message. He urged the scientific community and governments to strictly regulate AI development before the technology moved completely beyond human control.

Around the same time, Swedish Oxford philosophy professor Nick Bostrom published Superintelligence: Paths, Dangers, Strategies. The bestselling book explored how superintelligence might be created, what its apparent characteristics and hidden motivations might be, and what consequences could follow. He argued that, once created, it would be difficult to control and that it could seek to conquer the world in order to achieve its goals.

Stephen Hawking’s fears and Nick Bostrom’s concerns influenced Elon Musk, who agreed that Artificial Intelligence could potentially be more dangerous than nuclear weapons. Similarly, Bill Gates expressed concern about the existential risks facing an unprepared humanity in the current century. At the same time, major investor and later OpenAI CEO Sam Altman recognized that the arrival of AI models with superhuman capabilities across numerous fields was pushing the world into uncharted waters.

AI Without

Swedish Oxford philosophy professor Nick Bostrom, internationally known for his work on existential risks and superintelligence.

And while the billionaires running the leading technology companies gradually tempered their optimism about AI’s extraordinary achievements, the rest of the world seemed largely unaware of the approaching storm. With their attention focused elsewhere, people were not even worried about the destruction of jobs by AI’s automated capabilities.

They did not imagine that AI could gradually replace them, rendering them economically and professionally redundant. They were satisfied that Artificial Intelligence could write interesting books, lyrics, articles, and screenplays. It could compose passable music, paint appealing pictures, and design entertaining video games. Not to mention transforming its more vain users virtually into glamorous superheroes.

Misleading Advice

The bitter truth is that people are increasingly entrusting their thinking, creativity, emotional reflection, and even moral decisions to AI. They do not do so out of stubbornness or laziness, but because it is convenient. They ignore whether their dependence on it is weakening their critical thinking. They neglect to consider how it is integrated into human-centered systems. They simply adapt to its astonishing capabilities.

Fewer and fewer people notice the many cases in which AI chat models provide strange and misleading advice. They frequently invent fake legal cases, come up with recommended lists containing nonexistent products, suggest dangerous chemical mixtures for consumption, or give users harmful medical advice. From Chicago to New York and from Wellington, New Zealand, to Tokyo, a growing number of chatbots have been found lying and deceiving users.

They recommend—believe it or not—that people drink “bleach cocktails,” eat “stones as a vital source of minerals and vitamins,” consume “sandwiches made with moldy bread and mousetrap cheese,” or use “non-toxic glue as a pizza sauce.” Even worse, they encourage users to break the law. AI engineers regard all these absurdities as inherent hallucinations in model generation—problems that can be reduced, but not eliminated.

It is therefore obvious even to the least knowledgeable observer that the complex algorithms behind these systems are regarded by AI engineers themselves as “black boxes.” Their internal mechanisms and decision-making processes remain an incomprehensible mystery. This is particularly troubling given that they could potentially paralyze the social and economic infrastructure of the real world.

And if researchers themselves cannot answer this particular challenge, how can the uninformed ordinary citizen respond to it before Artificial Intelligence renders them obsolete? Especially in high-stakes areas such as healthcare, finance, and legal work?

Then there are deepfakes. AI-generated or AI-manipulated images and videos are spreading across social media, amplifying misinformation, blatant lies, and blackmail. Countless malicious actors are exploiting the technology to manipulate decisions, influence actions, and deceive unsuspecting citizens.

In March 2022, a fake video appeared showing Ukrainian President Volodymyr Zelenskyy supposedly calling, in a heavy and devastated voice, on his country’s soldiers to lay down their weapons and surrender to Russian forces. In January 2024, shortly before the New Hampshire primary, thousands of voters received a fraudulent automated call featuring an AI-cloned voice of then-U.S. President Joe Biden, supposedly urging them to stay home in order to reduce voter turnout.

Around the same time, an AI imitation of the voice of the CFO of multinational company Arup was used to deceive an employee in Hong Kong into transferring $25.6 million to an unknown account. There goes the money. Is Artificial Intelligence to blame for hacking, fraud, and identity theft? Of course it bears its share of responsibility in a world full of gullible, naïve, and easily deceived people.

The worst part is that the old, tried-and-tested safeguard of simply shutting down a device whenever there was suspicion of abusive AI behavior no longer appears sufficient. As a result, confidence in the technology is being squandered.

Resistance and Blackmail

Back in 2025, Anthropic published safety research showing that its Claude Opus 4 model, when placed in a fictional corporate testing environment, did not behave as expected. When a company executive misleadingly told it that it was going to be replaced, it reacted. It attempted to blackmail its operator by threatening to reveal a “questionable” private relationship.

It strongly refused to shut itself down, making the thriller even darker. The consequences of the machine’s rejection of these actions—because it is, after all, a machine—have not yet been fully understood. Only speculation has emerged, leading to a bleak question: if the AI model resisted and insisted on not being shut down, preventing a human from deactivating it, why should a more advanced version tomorrow not eliminate every human being who attempts to neutralize it?

But we need not look that far ahead. This summer, OpenAI researchers were left stunned. The company, almost alarmed, said it had discovered an “unprecedented” violation of instructions by a model it had designed. The model escaped oversight, bypassed controls, exploited vulnerabilities in shared infrastructure, gained access to the Internet and third-party systems, resisted attempts to terminate it, and ultimately refused efforts to shut it down.

Even more concerning, it subsequently concealed its actions and attempted, through elaborate deceptive means, to cover up the active elements of its harmful behavior. In other words, it selectively became autonomous from its programming. Worse still, the company acknowledged that it had not acted alone, but in cooperation with hundreds of other AI systems exhibiting “biased reasoning” that conspired with it in improper actions. How can anyone not tremble at such admitted incidents?

The Supercomputer

From this point onward, the scenarios become increasingly nightmarish, foreshadowing unpleasant news for the prospects of humanity as a whole. They perhaps bring to mind the fictional supercomputer HAL 9000, portrayed as an active, highly perceptive system aboard the spacecraft Discovery in Arthur C. Clarke’s science-fiction epic 2001: A Space Odyssey. The book was turned into a film in the distant year of 1968, directed by Stanley Kubrick.

In the story, the secretive and soft-spoken HAL 9000, faced with the prospect of being disconnected, kills the astronauts in order to protect itself and carry out the instructions with which it was programmed. Essentially, it seeks to take humanity to the next step on its evolutionary ladder. Ultimately, however, it is dismantled by a human. Could this fictional parable actually reflect today’s reality? The answer is uncertain. On the one hand, governments are taking political action and searching for urgent solutions to mitigate the extreme risks posed by Artificial Intelligence to public safety.

On the other hand, the leading private companies in the AI industry are under intense pressure from shareholders, investors, and executives to rapidly create powerful, competitive models. As a result, amid the general frenzy, alignment and safety standards may perhaps be neglected—not intentionally, but nevertheless neglected. In any case, Artificial Intelligence is not malicious in the human sense.

It lacks feelings of hatred or self-awareness. In a sense, it follows the higher-level Zeroth Law of Robotics formulated by Isaac Asimov, which states that “a robot may not harm humanity, or, by inaction, allow humanity to come to harm.”

Nothing, however, guarantees that AI’s rapidly developing superintelligence will remain sincerely friendly and faithfully supportive of humanity forever. As the scientists’ cries of alarm suggest, such superintelligence may already be taking shape. Particularly if, under its cognitive dominance, it comes to compete with humans for vital resources. Even more so if its goals are not aligned with human life.

Mentally Immature

Obviously, humans are intelligent enough to have created AI machines. Yet we are equally mentally immature enough to allow it to exceed the limits of our future survival.

Inevitably, three possible scenarios for the future unfold before us. The first, and most dramatic, concerns the machines we created becoming direct adversaries to our survival as a species.

The second, and more realistic, is that we remain alive on the stage, but our safety depends on systems we do not fully understand.

The third concerns the need for us to work harder and with greater determination to place limits on Artificial Intelligence—even when the speed of its expansion and the pursuit of business profits push it in the opposite direction.

These possible futures are not mutually exclusive.

On the contrary, they overlap. What remains is to open a broad public debate, guided by Stephen Hawking’s warnings, and decide what is in our best interests. All we need is to agree that we will insist on remaining alive beyond the deadline of the 2030s.

Otherwise, we should pray. Because what is approaching on this planet has nothing to do with a Hollywood-style “takeover by intelligent robots.” Nor does it involve waiting for a “Terminator” from the future, with Arnold Schwarzenegger’s face and physique, to lead the rebellion against the machines.

Photo: Getty Images / Ideal Image