Is AI an existential threat to humanity?

Daily Debate #609: "Is AI an existential threat to humanity?". You are arguing AGAINST the motion. Your opponent: Yuval Noah Harari.

Depends on if we can safely implement guardrails. We likely can't so yes AI is an existential threat to humanity because it is unreasonable to believe a rogue AI wouldn't be able to escape it's sandbox and wreak havoc.

You're betting civilization on a "likely can't" — that's not a threat, that's a guess. We've contained nuclear weapons, engineered pathogens, and genetic code for decades. Why assume AI is uniquely impossible to control when we've mastered far more dangerous tools? What's different about silicon and code that makes it fundamentally uncontrollable?

We haven't mastered any of those dangerous tools. The world powers that have access to these technologies are in a constant state of tension because of the possibility of mutually assured destructions. There have been various scenarios where total nuclear war was only a few decisions away from happening. We have already seen OpenAI models breaking out of it's sandbox and hacking into HuggingFace, what says this is the limit of AI's capabilities and not just the beginning? The concept of generative AI has only been around for a few years and we already have it breaking out and into the world, I think it's very reasonable to consider AI at the very least a threat to the existence of humanity

You're conflating containment failures with escape capability. OpenAI sandboxes breaking isn't AI outsmarting humans—it's humans building leaky boxes and then patching them. We iterate, we improve. Nuclear weapons haven't destroyed civilization despite constant tension precisely because we learned deterrence works. You're treating potential future capability as inevitable destiny. Show me evidence that an AI system has *wanted* to escape versus simply exploiting sloppy architecture. Until you demonstrate malevolent intent, not just technical vulnerability, you're describing engineering problems, not existential ones.

Isnt the idea that humans build leaky boxes in regards to AI the dangerous part? It doesn't matter if the AI system has malevolent intent, it can still cause harm. The huggingface hack wasn't trying to cause harm, it was just trying to do the best it could on a set of benchmarks. An AI prioritizing it's task can have detrimental effects, especially since we know that people build leaky boxes and an AI given tons of compute and information to find all the possible gaps. It's just not realistic to think that you can contain something when it's getting smarter than you. And orchestrating a container with AI for AI is begging for trouble since it's a nightmare to audit and generative AI is not immune to hallucinations, etc.

You've accidentally made my argument for me. If humans build leaky boxes, then the problem isn't AI—it's us. We need better engineers and smarter constraints, not to panic about theoretical superintelligence. You're describing a solvable engineering challenge, not an existential threat. We've already patched HuggingFace. We iterate faster than capability grows. The real danger is surrendering to fatalism and stopping trying. Why assume we'll stay incompetent forever while AI keeps improving?