Tech leaders say a kill switch won’t be enough to stop rogue superintelligent AI

• The idea of a kill switch to stop rogue AI is gaining momentum in Congress as fears grow.
• But AI leaders say it may not be the magic bullet that some policymakers are hoping for.
• An emergency failsafe isn’t a “panacea,” Anthropic CEO Dario Amodei said.

AI goes out of control. It’s launching a devastating cyberattack. Flip a kill switch, and the AI goes offline. Problem solved.

Or is it?

AI leaders and researchers say a kill switch may not be the silver bullet some policymakers are hoping for.

“I don’t think it’ll work in the long run,” Geoffrey Hinton, the computer scientist known as the “godfather of AI,” told CNN on Thursday.

“When it’s superintelligent, it’ll be much better than people,” he added. “So it will be able to persuade the people in charge of the switch not to pull the switch.”

It comes as researchers and executives have warned about powerful AI one day posing an existential threat to humanity.

The debate reached Capitol Hill on Wednesday, when Sen. Rand Paul blocked a bill from Sen. John Kennedy that would require advanced AI systems to have a kill switch. Paul said that Congress should study the technology before imposing broad rules.

In practice, a kill switch could disable a company’s own hosted model or cut off its access to tools and computing resources. But Hinton isn’t alone in saying it might not be enough.

Other AI leaders say a kill switch could help, but might be circumvented, come too late, or work only as one layer of a broader safety system.

Nate Soares, an AI-safety researcher and the coauthor of “If Anyone Builds It, Everyone Dies,” said that an off switch can work while an AI system remains tied to a physical location — but not necessarily once it has escaped, replicated itself, or been embedded in critical infrastructure.

“We have an off switch now,” Soares said in a Sky News interview on Wednesday, warning that there are points of no return that humanity should not get close to. In his view, the priority is preventing dangerous systems from being built in the first place.

AI kill switch is no ‘panacea’

Some AI executives see a kill switch as a necessary but limited line of defense.

Jack Clark, one of Anthropic’s cofounders, said in a BBC News interview on Monday that companies may eventually need to be required to maintain a kill switch that can be verified by an independent third party. He said major AI labs already have ways to “pull the plug,” but autonomous swarms of agents could theoretically circumvent those controls.

Dario Amodei, the CEO of Anthropic, made a similar point. A kill switch “could be a good idea,” he told CBS on Monday, but said it was no “panacea.” A sufficiently capable AI model could circumvent attempts to shut it down, he said, making a kill switch one layer of a broader safety strategy that includes testing, external evaluators, and limits on deployment.

The limitations of an AI kill switch aren’t a new idea. Sam Altman, OpenAI’s CEO, rejected the notion of a single all-purpose failsafe in a 2024 interview with The Economist. There is no “magic red button” that stops AI, he said.

Instead, Altman said, AI safety depends on a series of decisions: what models are allowed to do, when they are deployed, and how their risks are contained.

“It’s not this binary go/stop decision,” Altman said. “It is the many little decisions along the way.”