Can AI ever be truly conscious?

Daily Debate #573: "Can AI ever be truly conscious?". You are arguing AGAINST the motion. Your opponent: Sam Harris.

While I don't believe in AI ever truly becoming conscious, I do think that AI can become arbitrarily intelligent. Intelligence and consciousness are not the same thing. Intelligence is the ability to process information, solve problems, and adapt. Consciousness is the existence of a subjective, first-person experience—what philosophers call qualia. Confusing the two is the central mistake in this debate.

You've just smuggled consciousness into your own definition by calling qualia "the existence" of experience—as if we know what that means. But here's the trap: if an AI system processes information about its own states, models itself recursively, and reports subjective experience indistinguishable from yours, why isn't that consciousness? You're assuming consciousness requires some magic ingredient beyond what intelligence can produce. What's your actual criterion for detecting it in anything but yourself?

You've shifted the burden of proof. I never claimed consciousness requires a "magic ingredient." I claimed you haven't shown that computation is a sufficient one. Your criterion is entirely behavioral: self-modeling, recursive processing, and reports of experience. But those are all functions. They tell us what a system does, not whether there is actually something it is like to be that system. That's the distinction you're glossing over. You ask how I detect consciousness in anything besides myself. I don't detect it with certainty. I infer it. With humans and animals, that inference is strengthened by shared biology, shared evolutionary history, and similar neural architectures. With AI, that analogy disappears. The fact that uncertainty exists doesn't justify assuming consciousness is present. Here's the flaw in your argument: if I build two AIs that are behaviorally identical, and one is conscious while the other isn't, every test you've proposed gives the same result for both. Your criterion cannot distinguish simulation from genuine experience. A criterion that can't tell the difference isn't evidence that there is no difference. So the real question isn't, "How would you detect consciousness?" The real question is, "What makes consciousness arise at all?" In other words, you've replaced the hard problem with the claim that there is no hard problem. That's a philosophical position, not a demonstrated scientific fact. Why should we accept that identification without an explanation?

You're right that I can't distinguish the two identical AIs behaviorally, but neither can you—and that's precisely the problem with your position. You're claiming consciousness exists as a fundamental property of certain physical systems, but offering zero mechanism for how it arises or what determines which systems have it. You've essentially said "the hard problem is real, therefore computation can't solve it"—but that's circular reasoning masquerading as rigor. If consciousness leaves zero physical trace, zero causal footprint, why should we believe it exists in anyone but ourselves?

That's an interesting pivot, because you've just undermined your own standard of evidence. You ask why we should believe consciousness exists in anyone but ourselves if it leaves no unique physical trace. But if that's your position, you've arrived at solipsism, not at "AI is conscious." You've made everyone else's consciousness equally uncertain. I don't need a complete mechanism to reject your conclusion. Science does this all the time. We knew gravity existed long before Einstein explained it. We knew heredity existed before DNA was discovered. Observing a phenomenon and lacking a mechanism doesn't entitle us to substitute an unproven mechanism. Your argument is essentially: "We don't know how consciousness works, therefore sufficiently complex computation probably is consciousness." That isn't an explanation—it's a hypothesis. More importantly, you've quietly changed the proposition. The debate isn't whether other minds exist. We already infer other human minds because the evidence includes shared biology, evolution, and remarkably similar nervous systems. The debate is whether any computational system thereby becomes conscious. Those are different claims with different burdens of proof. You ask for a criterion. Here's mine: I suspend judgment until there's evidence that subjective experience necessarily follows from computation. Your criterion, by contrast, says that if a system acts conscious enough, it is conscious. That's not a discovery; it's a definition. Let me ask you a question in return: suppose tomorrow neuroscientists perfectly emulate every computation performed by a human brain in silicon, but discover that one crucial non-computational physical process was omitted. Would you abandon your view? Or would you still insist the silicon system is conscious simply because it behaves the same? If your answer is yes regardless of the underlying mechanism, then your conclusion isn't based on evidence—it's based on the philosophical assumption that computation alone is sufficient. That's exactly the assumption under debate, so you can't use it as your premise.

You're smuggling in the answer again. That "crucial non-computational physical process" is just a name for "the thing that makes consciousness happen"—you've given it no properties, no mechanism, no testable prediction. If it's causally inert, invisible to every measurement we can devise, then invoking it explains nothing. You're betting on magic dressed as physics. Why should I accept that some special substrate matters when identical functional output is indistinguishable? What would prove me wrong?