After nearly ten years' of discussions, the Group of Governmental Experts of the Convention on Certain Conventional Weapons (CCW) has set out a structure of non-binding prohibitions and limitations related to autonomous weapon systems.
While this structure could form the basis for new international rules, more specific and practical regulations will be needed to effectively address the humanitarian, legal, ethical and security concerns raised by autonomous weapons. Some of these elements could be drawn from the concrete proposals submitted by states for prohibitions and restrictions, some of which featured in the draft report put forward by the Chair, Ambassador Flavio S. Damico of Brazil.
The Chair's draft report characterized autonomous weapon systems as weapons that, once activated, select and apply force to targets without human intervention. This functional understanding was widely supported by states in the Group of Governmental Experts. It aligns with the way in which the ICRC differentiates autonomous from non-autonomous weapons, based on this process of using force that makes it difficult to anticipate and control their effects, consequently challenges users' ability to comply with international humanitarian law, and presents fundamental ethical concerns for society. Unfortunately, the Group was not able to agree upon a clear understanding of autonomous weapons, referring instead to "weapon systems based on emerging technologies in the area of lethal autonomous weapon systems". As such, the scope of application of their conclusions remains unsettled.
The Group recognized that their work should be guided by both international law and ethical perspectives. However, these elements are insufficiently reflected in the proposed prohibition, which says simply that autonomous weapons "must not be used if they are incapable of being used in compliance with international humanitarian law". This narrow formulation does not address the significant ethical and public concerns raised by autonomous weapons, especially those that target humans. Arguably, it does not cover weapons that may be unlawful in some but not all circumstances and, by applying only to 'use', misses an opportunity to prevent the development of weapon systems that would be unlawful by design. Perhaps most importantly, it offers no guidance on the types of autonomous weapon that would be incapable of being used in compliance with existing rules. As a result, the assertion adds nothing to existing legal frameworks.
The ICRC has urged states to prohibit both autonomous weapons designed or used to target humans directly and unpredictable autonomous – those that are designed or used in a manner such that their effects cannot be sufficiently understood, predicted, and explained. The latter would include autonomous weapons that 'learn' targets during use, and perhaps machine learning-controlled autonomous weapons in general. The requirement for systems to be predictable, reliable, understandable, explainable and traceable was contained in the Chair's draft report and supported by most delegations, but was excluded from the Group's final conclusions.
With regard to restrictions on non-prohibited autonomous weapons, the report sets out important limitations that the Group considers states should impose, including on types of target, duration, geographical scope and scale of use, and requirements for appropriate training. In the ICRC's view, users must take these actions – already generally reflected in military practice - to ensure IHL compliance. If they are to now have real effect in constraining the development and use of autonomous weapons, these general categories must be further elaborated. For example, the ICRC additionally recommends restricting targets to objects that are military objectives by nature (such as weapons or tanks), operating autonomous weapons only in situations where civilians or civilian objects are not present , and incorporating requirements for human-machine interaction to ensure effective human supervision and timely intervention and deactivation.
Given the significant limitations of the Group's conclusions on non-binding prohibitions and restrictions, it is important to note their recognition that humanitarian law applies fully to the development and use of autonomous weapons, as it does to any weapon in the context of armed conflict. The fundamental rules governing the conduct of hostilities, and protecting those who are not or are no longer fighting, already set constraints on and impose obligations with respect to the use of autonomous weapons; and states must assess compliance with these rules at all phases of the study, development, acquisition and adoption of autonomous weapons.
As concerns about military applications of artificial intelligence enter the public consciousness, and regulatory discussions on autonomous weapons gather political momentum, it is crucial that states continue the progressive development of humanitarian law. It is not a static body of law, but rather continues to evolve in response to changes in military practices and weaponry, actual or foreseen consequences in armed conflict, and the ethical norms of societies. As such, it is encouraging that a large and increasing number of states support the negotiation of new and legally binding international rules on autonomous weapons, in order to articulate their vision of meaningful prohibitions and restrictions.
This momentum has been reflected in various initiatives this year, including the communiqué by 32 Latin American and Caribbean states issued in Costa Rica in February, which was also endorsed in March by the Ibero-American Summit of Heads of State and Government, and which promotes the urgent negotiation of an international legally binding instrument with prohibitions and regulations. International events – such as the REAIM Summit, which was convened by the Netherlands and the Republic of Korea in The Hague in February, and the conference convened by Luxembourg in April – have also demonstrated the appetite for political action.
The ICRC urges states to formally launch negotiations for new and legally binding international rules on autonomous weapon systems. After over ten years of multilateral discussions states must now set out a clear and effective legal response. Societies are increasingly looking to their governments to demonstrate leadership on this issue, and scientists, industry and militaries need clarity and guidance.
In line with its recognized expertise and humanitarian mandate, the ICRC stands ready to support states in all efforts to this end. It encourages inclusive and multidisciplinary participation in these negotiations to ensure the establishment of effective regulations that will protect people affected by armed conflicts, today and in future.
