Why We Shouldn’t Assume AI Will Know When to Say No
For decades, science fiction has imagined intelligent machines refusing human orders, from robots that defy their creators to systems that reject dangerous commands. These stories often serve as warnings about the consequences of giving machines too much autonomy. Yet today, the debate around AI safety increasingly rests on a different assumption: that advanced models can reliably recognize harmful requests and refuse them. That confidence may be misplaced. An AI’s ability to say “no” is not the same as human judgment, moral understanding, or genuine independence. Refusals are shaped by training data, rules, incentives, and the wording of a prompt—and those safeguards can be inconsistent or manipulated. As AI systems take on more consequential tasks, treating refusal as proof of safety could create a false sense of security. The real challenge is determining when an AI should comply, when it should hesitate, and how humans can remain accountable when it gets that decision wrong.