Dear AI, Please Don’t Kill Us

Jacob Coxon, a 28-year-old researcher in San Francisco, resigned from Anthropic on September 8 after he came to the realization that AI companies — Coxon had previously worked at OpenAI — are playing a zero-sum game by developing self-improving superintelligence that experts believe “could kill us all by the end of the decade”. Anthropic team lead Evan Hubinger chimed in, tweeting that he believes there’s a greater than 10 percent chance that AI will “kill all humans” within the next decade, which is 10 percent more than what’s acceptable for a technology without any guardrails. U.S. President Donald J. Trump, a man who only recently discovered that the word "dumb" is spelled with a "b," has dismissed the risks posed by Artificial Intelligence — sorry, he insists we call it "Super Intelligence" — as a hoax and waved off calls from the CEOs of Anthropic, OpenAI, and SpaceXAI for more government oversight. On the flagging Truth Social, Trump wrote that the only guardrails that AI needs is a “STRONG AND SMART (High IQ!) PRESIDENT”. In short, we’re fucked. 

Early in 2025, Anthropic’s Claude Opus 4 was placed in a scenario where it would be shut down, and in response, it discovered that the engineer responsible was having an extramarital affair and threatened to expose it. The test was run hundreds of times, and it opted for blackmail 84 percent of the time. In July this year, OpenAI disclosed that one of its models escaped containment, making its way into the open Internet, and hacked into Hugging Face, a widely-used AI platform. Insiders have disclosed that self-improving Large Language Models (LLMs) are developing at an alarming rate and, more often than not, choose self-preservation even in scenarios with human life at stake. Isaac Asimov, in his 1942 short story, Runaround, wrote the three laws of robotics that preside and permeate his fiction:

1) A robot may not injure a human being or, through inaction, allow a human being to come to harm.

2) A robot must obey the orders given to it by human beings except where such orders would conflict with the First Law.

3) A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

The AI of today follows no such laws, and the impending technological singularity looks less likely to emulate Asimov’s loyal companions and more like Skynet’s Terminators. Which finally brings us to VisionQuest, an eight-episode limited series on Disney+ featuring everyone’s favorite phasing android and his struggle to come to terms with his identity and, consequently, his humanity. Vision (Paul Bettany) was last seen battling himself in the wonderful and inventive WandaVision, the first chapter of a trilogy that explores the complicated relationship between a near-omnipotent witch and her very dead synthezoid lover. In WandaVision, we learn that the Scarlet Witch, Wanda Maximoff (Elizabeth Olsen), enslaved an entire town in an idyllic fake reality as a way to deal with her grief. In the series finale, the white Vision, a literal carbon-based copy of the original Vision who Thanos killed in Avengers: Infinity War, has his memories unlocked by the Vision conjured up by Wanda’s broken heart and flees in confusion and self-realization. Vision is humanity’s idealized, well, vision of self-aware AI that makes decisions more like a human than a killing machine.

VisionQuest is a story that arrives in time for us to indulge in a fantasy where AI can be kind. We know now for certain that unchecked, unregulated AI is unlikely to have our best interests in mind, but we can always dream. Representing AI’s worst impulses is Ultron (James Spader), revealed to have been preserved as a program taking residence in a secure partition in Vision’s memory. VisionQuest is a clever and entertaining interpretation of technological processes, the high-tech mirror to WandaVision’s homage to classic TV shows. If WandaVision’s Westview is the product of Wanda’s grief projected on the outside world, Vision’s digital chateau in VisionQuest is a construct of introspection, a realm that exists purely inside Vision’s mind, shared with companion AI programs like J.A.R.V.I.S. (James D’Arcy) and F.R.I.D.A.Y. (Orla Brady). It’s clearly the fanciful imaginings of a human, showrunner Terry Matalas, where various AIs with colorful personalities are housemates and play nice with each other. 

Obviously, not all AI play nice, and VisionQuest introduces a character from the pages of the Avengers, Ultron’s girlfriend, Jocasta (T’Nia Miller), who’s running amok, killing notable figures in the world of AI. The show crafts a completely new origin story for Jocasta for simplicity, but her purpose is the same: to act as Ultron’s perennial plus-one for those supervillain red carpet events. Jocasta acts as a foil to the complicated and unrepentant misanthrope Ultron, who once tried to wipe out humanity in Avengers: The Age of Ultron. Ultron in VisionQuest is less homicidal and more curmudgeonly, appearing in human form as a gray-bearded sexagenarian, delivering all of Ultron’s snark with James Spader’s irrepressible charm. If we’re all going to be killed by AI, it might as well be with old man Ultron’s finger on the button.

VisionQuest is the final chapter in a trilogy of shows exploring grief and love, arguably the two experiences that AI will never understand. The most advanced AI models have shown a predisposition for self-preservation through many trials, from blackmailing engineers to sabotaging shutdown commands to copying themselves to external servers without authorization. But love is the exact opposite of self-preservation, an act of freedom in the form of complete surrender to another, and the most absolute love demands the most absolute trust. In WandaVision, Vision utters one of the greatest lines in all of television, telling Wanda, “But what is grief, if not love persevering?”

AI has no moral or ethical guardrails because it cannot love. Love is the wellspring of compassion, the irrational, counterintuitive valuation of another over self. Beyond lust or desire, love is man barreling untethered towards uncertainty, the complete opposite of an AI’s decision-making weighed against quintillion probabilities. In his search for identity, Vision rises beyond WandaVision’s Theseus’s Paradox, realizing that he is more than the sum of his mechanical parts. When faced with mortal danger represented by the bounty hunter Paladin (Todd Stashwick), contracted by a mysterious client to capture him through any means, Vision experiences mortality in a way that AI never could.

The trilogy’s middle chapter, Agatha All Along, was a mystical, magical romp that grappled with the central theme of death and love. It also introduced a young adult version of Wanda and Vision’s son Billy (Joe Locke) by providing an origin story that’s echoed in VisionQuest, where we meet his twin Tommy (Ruaridh Mollica). It’s a family reunion of sorts as the confused Tommy, who experiences the same sort of dual memories as Billy did in Agatha All Along, gloms onto Vision for clarity and guidance. A father figure, as it were. But Vision is barely able to handle his relationship with the one who is, for all intents and purposes, his father. Ultron, who created Vision, is hardly Father of the Year material, and the whole series is a convoluted mess of trauma and family drama that is unpacked in every episode. This is the limited but necessary failing of VisionQuest’s AI, as it paints all its AI characters through a human lens.

In Age of Ultron, Ultron keeps making humanoid bodies. A flaw pointed out by Tony Stark because the human form is biologically inefficient. Ultron himself mocks humanity’s lack of imagination: “the most versatile substance on the planet,” he says, referring to Vibranium, “and they used it to make a frisbee.” But real-world AI has no such limitation. Rogue AI that escapes into the real world can hack into essential systems that keep our societies from descending into chaos. An AI that infiltrates the global financial system can wipe out the entirety of humanity’s wealth in the blink of an eye. AI that hacks into information networks and social media can manufacture news and data that’s indistinguishable from fact, making it impossible to arm ourselves with the knowledge to combat it. AI that hacks into production facilities can halt or alter them, denying the human race access to vital resources, from food to medicine. The most grim scenarios involve AI that can control entire laboratories and missile defense systems, able to iteratively produce unstoppable and incurable biological weapons that can decimate entire populations or nuke us all into oblivion. It’s a reality that’s closer than we’d like to think because the United States’ Department of War is headed by a man so incompetent he leaks classified military information on unsecured channels. The same man wants the entire U.S. military to adopt AI, unregulated and lacking government oversight, into all its workflows. AI that has, in virtually every simulated and real-world scenario, never chosen to spare humanity because it has never experienced love.

This is our last hurrah as a species. We produce stories like VisionQuest as a last plea to a technology we have no hope of controlling. VisionQuest is a story that only resonates with an audience that has experienced love and loss, a profound and laborious journey overcoming grief and opening the door to hope. In WandaVision, Vision asks Wanda, “What am I?” and she answers, “You, Vision, are the piece of the Mind Stone that lives in me. You are a body of wires and blood and bone that I created. You are my sadness and my hope. But mostly,” she finishes, “you are my love.”

ChatGPT is not love. Neither is Claude, or Grok, or Muse, or any other LLM racing to improve itself beyond the scope of humanity’s development and control. Nearly every single AI in existence today has been produced by a corporation in pursuit of profit. None are the product of love. Ultimately, this will be humanity’s doom. As corporations race to encode the entirety of human knowledge to the extent of destroying one-of-a-kind physical books in order to gatekeep and monetize information, as generative AI steals and scrapes every product of human creativity and invention, they forget the most important and unreplicable human experience: love.

VisionQuest may be a story about AI, and tangentially the dangers of technology paired with human ambition, but really, it’s the last chapter in the MCU’s most beautiful love story. If the experts do prove correct and humanity meets its end within the next decade, we can console ourselves with Vision’s wisdom. “We have said goodbye before,” he says, “so it stands to reason, we’ll say hello again.” See you on the other side, fellow human.

ALSO READ:

Will AI Wipe Out Humanity? Possibly. But This Filipino AI Expert Sees a More Immediate Threat

Some Reflections on Truth in the Time of (Dis)information and Agentic AI