Hawking, Musk and 1,000 Experts Urged a Ban on Autonomous Weapons

Stephen Hawking and 1,000 Experts Sign Letter Warning Of Weaponized Robots

On 28 July 2015, more than 1,000 researchers in artificial intelligence and robotics published an open letter asking the world to ban offensive autonomous weapons: machines that can select and attack targets without meaningful human control. Among the best-known names attached to it were the physicist Stephen Hawking, Tesla and SpaceX chief Elon Musk, Apple co-founder Steve Wozniak, Google DeepMind co-founder Demis Hassabis and the linguist Noam Chomsky. The letter warned that a military AI arms race was within reach and that the time to stop it was before it began. Ten years on, the debate it helped launch is still unresolved, and the weapons it described are no longer purely hypothetical.

What the letter said

The letter was organised by the Future of Life Institute and presented at the International Joint Conference on Artificial Intelligence (IJCAI) in Buenos Aires. Its argument was short and direct. Autonomous weapons, the authors wrote, could be deployed within years rather than decades, and they had been called "the third revolution in warfare, after gunpowder and nuclear arms."

Unlike nuclear weapons, they noted, such systems would need no rare materials. Once a major power started mass-producing them, cheap copies would inevitably spread to black markets, terrorists, dictators and warlords. That is the reasoning behind the letter's most quoted line: "autonomous weapons will become the Kalashnikovs of tomorrow."

The signatories did not oppose AI research or every military use of it. They drew a line at weapons that pick and engage human targets on their own, and they argued that most AI scientists had no interest in building them and did not want their field tarnished by them.

Who signed, and why it mattered

The letter stood out because the warning came from inside the industry. Many of the signatories were the people building the technology, not outside critics. Alongside the famous names were thousands of lesser-known professors, engineers and graduate students; the count of AI and robotics researchers passed 1,000 at launch and grew far larger in the following months as others from science and industry added their names.

Toby Walsh, a professor of AI at the University of New South Wales and one of the organisers, told the Guardian he hoped the United Nations would take up a ban. Advocacy groups such as the Campaign to Stop Killer Robots, formed in 2013, used the letter to press governments for a treaty.

What counts as an autonomous weapon?

Part of what makes the topic slippery is definition. Many weapons already have some automatic functions. Ship-defence guns such as the Phalanx system and missile shields such as Israel's Iron Dome can detect and fire at incoming threats faster than a person could react. Sentry guns deployed along some borders can track movement on their own.

The US Department of Defense describes an autonomous weapon as one that, once activated, can select and engage targets without further human input. The concern raised by the 2015 letter is the step from defending against missiles to choosing which people to kill, with no human deciding each strike. Supporters of regulation usually talk about keeping "meaningful human control" over the use of force; critics of a ban argue that precise machines might in some situations cause fewer civilian deaths than tired or frightened soldiers.

A myth worth correcting

When the letter made headlines, some commentary claimed that a UN ban would simply hand exclusive control of such weapons to NATO countries, in the same way the UN supposedly bans nuclear weapons for some states while allowing others to keep them. That is not how international arms treaties work.

The UN does not impose bans on its own. Treaties bind only the countries that sign and ratify them. The Treaty on the Prohibition of Nuclear Weapons, adopted by 122 states at the UN in July 2017, entered into force on 22 January 2021, but only for its parties, and every nuclear-armed state, including the NATO nuclear powers, has refused to join. The older Non-Proliferation Treaty recognises five nuclear-weapon states, of which only three are NATO members. A ban on autonomous weapons would face the same basic reality: its strength would depend on which major powers agreed to it.

What happened after 2015

Governments have been discussing the issue since 2014 under the Convention on Certain Conventional Weapons in Geneva, and in 2016 they set up a Group of Governmental Experts to continue the talks. Progress has been slow because decisions there require consensus, and leading military powers, including the United States and Russia, opposed a legally binding ban when it was raised in 2019.

The issue reached the UN General Assembly in December 2023, when it adopted Resolution 78/241 by 152 votes to 4, with 11 abstentions. The resolution asked Secretary-General António Guterres to collect states' views on the humanitarian, legal, security, technological and ethical challenges raised by these systems. It opened a process; it did not ban anything.

Meanwhile, the technology has moved into real conflicts. A 2021 report by a UN panel of experts on Libya said a Turkish-made Kargu-2 drone may have attacked retreating fighters autonomously in 2020, which some analysts called the first known case of its kind, though the details remain disputed. The war in Ukraine has driven rapid development of drones with growing degrees of autonomy on both sides, and claims of fully autonomous lethal strikes there have so far not been independently confirmed.

Why the warning still matters

Stephen Hawking died in 2018, but the questions his signature helped publicise are more pressing now than in 2015. A few points help make sense of the debate:

The core disagreement is about control, not about whether militaries use AI at all. Most proposals focus on keeping a human responsible for decisions to use lethal force.
Treaties depend on participation. A ban signed only by countries that never planned to build such weapons would change little; one joined by major powers could change a great deal.
Cheap technology spreads. The letter's comparison with the Kalashnikov rifle was about cost and availability. Small drones and off-the-shelf software have since shown how quickly military tools can spread.

For readers who want to follow the issue, the most useful sources are the UN's reports on lethal autonomous weapons systems, the Convention on Certain Conventional Weapons meetings in Geneva, and organisations such as Human Rights Watch and the Future of Life Institute, which track how states vote and what they say.

Sources: GBH / AP: Researchers warn against autonomous weapons arms race (July 28, 2015); The Guardian: Musk, Wozniak and Hawking urge ban on warfare AI and autonomous weapons; Human Rights Watch: Killer robots: UN vote should spur action on treaty (January 3, 2024); Wikipedia: Lethal autonomous weapon; original commentary by John Vibes via The Anti-Media.

Related: