Military drones may have autonomously attacked humans for the first time ever last year, according to a United Nations report. While the full details of the incident, which took place in Libya, havenât been released and it is unclear if there were any casualties, the event suggests that international efforts to ban lethal autonomous weapons before they are used may already be too late.
The robot in question is a Kargu-2 quadcopter produced by STM, a Turkish firm. It is equipped with an explosive charge and can be flown manually, but in autonomous mode the drone uses on-board cameras with artificial intelligence to find and identify targets. It can then attack by flying into the target and detonating. STM claims the robot has sophisticated object and facial recognition capability. The firm didnât respond to a request for comment.
Details of the apparent attack have emerged in a report by the UN Security Councilâs Panel of Experts on Libya, . It details a civil war conflict in March 2020 between forces allied to Libyaâs Government of National Accord (GNA), which has since been dissolved, and those affiliated to Khalifa Haftar, commander of the Libyan National Army.
Advertisement
The report says that retreating Haftar forces were âhunted downâ by Kargu-2 drones operating autonomously, which were âhighly effectiveâ. âThe lethal autonomous weapons systems were programmed to attack targets without requiring data connectivity between the operator and the munition: in effect, a true âfire, forget and findâ capability,â notes the report. In other words, the drones were able to seek and attack targets without a human in the loop.
The panel says in its report that this information comes from a confidential source and declined to disclose further details to 51¶ŻÂț.
at the National Consortium for the Study of Terrorism and Responses to Terrorism in Maryland, says this appears to be the first time that autonomous drones have found and attacked humans. UN talks on restricting such lethal autonomous weapons and these weapons arenât currently restricted, with many nations including the UK arguing against an outright ban.
Kallenborn says he is concerned about the Kargu-2âs reliance on machine vision. âHow brittle is the object recognition system?â he says. âAnd how often does it misidentify targets?â While there is no public information on the droneâs AI, researchers have previously found that general machine vision algorithms can be fooled into misidentifying a turtle as a rifle.
at the United Nations Institute for Disarmament Research in Geneva, Switzerland, also voiced concerns. âWhether any such system could execute such functions reliably and to a high degree of precision under complex and dynamic conditions is a totally open question,â he says.
at Royal United Services Institute, a UK defence think tank, wonders whether the Kargu-2 drones were really in autonomous mode, because only their operators would know for sure. This highlights a challenge of legal restrictions: unlike chemical weapons or landmines, autonomous weapons cannot easily be distinguished from operator-controlled ones.
âThis does not show that autonomous weapons would be impossible to regulate,â says Holland. âBut it does show that the discussion continues to be urgent and important. The technology isnât going to wait for us.â
