News briefing 03 May 2024

Autonomy in weapons systems, and AI in security and defence

• Sweden: Insändare. Sverige måste ta ställning för ett förbud mot mördarrobotar/Letter to the Editor: Sweden must take a stand for a ban on killer robots: This piece in the Swedish newspaper Dagens Nyheter, written by Kerstin Bergeå, chairperson of the Swedish Peace and Arbitration Society and Malin Nilsson, Secretary General of the Swedish section of the Women’s International League for Peace and Freedom (WILPF), calls on the Swedish government to address the issue of killer robots and take a stand for stronger international law, before it is too late.
• Israel/Gaza: Two pieces on Israel’s Lavender and Habsora systems: Why human agency is still central to Israel’s AI-powered warfare: +972 Magazine (which broke the stories on Israel’s use of the Habsora and Lavender decision support systems) carries another story with further detail on these systems, noting that ‘experts in international law and AI warfare underlined to +972 that the carnage in Gaza is the result of concerted human decisions’, with intelligence units using ‘AI-powered targeting systems to rank civilians and civilian infrastructure according to their likelihood of being affiliated with militant organizations’, and ‘a concerted chain of command’ dictating ‘how these technologies are put into action’. A further piece in Opinio Juris, by war crimes researcher Christoper Elliott, says that ‘Lavender appears to be an intelligence repository which allows users to visualize persons of interest (potential targets) in a viewable “layer”’, with automation coming into play ‘when the machine populates the layer with data. In Lavender’s case, the data is people.’ Elliott further writes that ‘Gospel appears to work similarly, but for buildings’.
• Ukraine/United States: In Ukraine, New American Technology Won the Day. Until It Was Overwhelmed: The New York Times reports on Ukraine’s use of the U.S. military’s artificial intelligence tool known as Project Maven (previously covered in the 1st March 2024 news briefing here). The report notes that ‘So far the results are mixed: Generals and commanders have a new way to put a full picture of Russia’s movements and communications into one big, user-friendly picture, employing algorithms to predict where troops are moving and where attacks might happen. But the American experience in Ukraine has underscored how difficult it is to get 21st-century data into 19th-century trenches.’
• US: Army rethinks its approach to AI-enabled risks via Project Linchpin: Defense Scoop reports that the army is ‘evolving its approach to known and unknown dangers associated with deploying AI’ through Project Linchpin (featured in a previous news briefing here), and that a new ‘AI risk reduction framework’ is being developed ‘to inform all future pursuits’.
• US: Engaging China and Russia on Arms Control: An Interview With Assistant Secretary of State Mallory Stewart: In an interview with Arms Control, Assistant Secretary of State Mallory Stewart said that she sees support for a political declaration on military use of AI and support for a legally binding instrument as ‘consistent’, and said that the U.S. ‘We supported the Austrian resolution at the United Nations. We support the conversation in the UN group of governmental experts going forward on lethal autonomous weapons. The Austrian initiative, with respect to lethal autonomous weapons, says we should look at this issue more broadly. Our political declaration addresses a unique and different context, but is entirely consistent in the sense that we are building a common understanding of the potential risks of AI across the full range of military applications.’ 
• International: Military is the missing word in AI safety discussions: This article in The Financial Times, written by Marietje Schaake, international policy director at Stanford University’s Cyber Policy Center and special adviser to the European Commission, argues that ‘Making sure human rights standards and laws of armed conflict continue to protect civilians in a new age of warfare is critical. The unregulated use of AI on the battlefield cannot continue’.
• US: ‘AI-BOM’ bombs: Army backs off, will demand less detailed data from AI vendors: Breaking Defense reports on the Army backing away from initial plan to make AI contractors submit a ‘AI Bill of Materials’ which ‘how their AI was trained and what potential vulnerabilities it might have.’ Principal deputy assistant secretary Young Bang spoke to reporters and said that other attempts such as Project Linchpin and questioned the viability of choosing the ‘the most secure approach and just build all the AI it needs in-house.’ 
• US: ‘Poisoned’ data could wreck AIs in wartime, warns Army software acquisition chief: This piece in Breaking Defense talks about the risks of bad data and trojans, especially when training machine learning algorithms for military applications. Jennifer Swanson, deputy assistant secretary of the Army for Data, Engineering and Software, notes that there is still a lack of clarity in the technological capability to add ‘targeting data and strike planning, plugging into existing AI battle command projects.’ 
• International: Autonomous weapons systems in law enforcement: submission to the United Nations Secretary-General: Amnesty International have published their submission for the UN Secretary General’s report on autonomous weapons systems, focussed on the potential use of autonomous weapons for law enforcement purposes.
• International: Inside Geneva: new wars, new weapons and the Geneva Conventions: The Inside Geneva podcast by Swissinfo, hosted by Imogen Foulkes, features a conversation with Sai Borothu, researcher for Automated Decision Research, and Jean-Marc Rickli from the Geneva Centre for Security Policy. The podcast addresses crucial questions in the regulation of autonomous weapons. 

Facial recognition, biometric identification, surveillance 

• France: Paris tests AI surveillance ahead of Olympics: DW reports on the French police providing access to local transport authorities in Paris to get ‘images from more than 100 cameras’ which will then be ‘analyzed using artificial intelligence’ to run ‘intelligent and algorithm-based technology.’ 
• Israel: How Israel uses facial-recognition systems in Gaza and beyond: Matt Mahmoudi, an adviser on AI and human rights at Amnesty International, spoke to The Guardian on the report ‘Israel and Occupied Palestinian Territories: Automated Apartheid: How facial recognition fragments, segregates and controls Palestinians in the OPT.’ 
• US: US oversight body calls for more real-world biometrics testing, bias protections: This piece by Biometric Update reports on the Government Accountability Office (GAO) report to congressional committees titled ‘Biometric Identification Technologies: Considerations to Address Information Gaps and Other Stakeholder Concerns.’ With concerns including ‘exacerbation of systemic inequality, lack of transparency, biased outcomes’ among others, the report notes that despite an improvement in accuracy, challenges of ‘real-world implementations remain.’ 

AI, algorithms and autonomy in the wider world

• India: Use of Entity Resolution in India: Shining a light on how new forms of automation can deny people access to welfare: Amnesty International reports on the use of ‘Entity Resolution’, a digitised welfare technology, in India, and flags the possibility of creating barriers to access social welfare measures through the use of algorithmic decisions. The report recommends ‘testing and auditing mechanisms for algorithmic systems, transparency and explainability measures, and accountability processes and assistance for people affected by these systems.’ 
• UK: Proposed UK data protection law hit by criticism: Biometric Update reports that a group of UK parliamentarians have opposed an addition to the Data Protection and Digital Information Bill (DPDI) which would allow the Department for Work and Pensions (DWP) to ‘access to the bank accounts of individuals receiving state benefits.’ The concern raised in a letter by the parliamentarians is that the ‘power to scrutinize people’s accounts with automated systems’ could lead to ‘disastrous consequences.’ 

Other items of interest

• Online event: International Committee of the Red Cross event on artificial intelligence in military decision-making: legal and humanitarian implications: On 14th May, the ICRC will launch two reports on the use of AI in military decision-making, featuring a range of speakers. Registration is free Register here. 
• Webinar: Autonomous Weapon Systems: Webinar at the Vienna Center for Disarmament and Non-Proliferation (VCDNP): The Vienna Center for Disarmament and Non-Proliferation (VCDNP) hosted a webinar on autonomous weapon systems (AWS) on April 16, 2024. The webinar discussed ‘past and current initiatives to regulate’ autonomous weapon systems, and the speakers ‘emphasized the urgency of legally binding regulations.’ A recording of the webinar can be found here. 

Research and reports

• International: The road less travelled: ethics in the international regulatory debate on autonomous weapon systems: Alexander Blanchard, senior researcher at the Stockholm International Peace Research Institute,  writes on ethics and autonomous weapons for the ICRC humanitarian law and policy blog, noting that  ‘AWS reconfigure traditional understandings of how, where, when, and by whom decisions to use force are made, and ethical reflection is needed to help formulate good solutions to this reconfiguration.’