News Briefing 12-23 August 2024

Autonomy in weapons systems, and AI in security and defence

• Overview of submissions to the UN Secretary-General’s report on autonomous weapons systems: Our latest report provides an overview of 58 of the 91 submissions received by the Secretary-General for his report on autonomous weapons systems. The 58 submissions covered in our report were made by states, groups of states (e.g. international conference outcome documents), or international organisations or unions of states (e.g. CARICOM, the European Union, ECOWAS). Of these 58 submissions, 47 of these 58 submissions express support for some form of prohibitions and/or regulations, and 35 of the 58 submissions express clear support for the negotiation of a legally binding instrument on autonomous weapons systems. 36 of these submissions express support for a standard of human control over the use of autonomous weapons systems. 
• International: US official holds talks in Africa on responsible use of military AI: On August 21 2024, U.S. Assistant Secretary of State Mallory Stewart visited Nigeria to discuss the responsible use of ‘artificial intelligence (AI) in military operations.’ The visit aimed to enhance security cooperation with Nigeria and the regional bloc ECOWAS by focusing on aligning AI applications with international laws and addressing inherent biases in AI systems. The United States does not support the negotiation of a legally binding instrument on autonomous weapons systems, while Nigeria, and the wider ECOWAS bloc, is in support of such negotiation. The discussion highlighted the need for Africa-specific policies as the continent faces significant security challenges, including terrorism. Nigerian officials and experts stressed the benefits of AI but also emphasized the need for tailored policies and infrastructure support. The visit aligns with recent continental efforts to advance AI in Africa, as endorsed by the African Union.
• Russia: Russian defense plan kicks off separate AI development push: Last week, Russia unveiled a 10-year defense plan focusing on integrating AI into its military operations. This plan aims to enhance autonomous weapons systems and improve existing technologies, such as drones, missiles, and artillery. The AI market in Russia reached ‘$7.3 billion’ in 2023. A new budget line for defense AI is planned to support this transition. 
• International: Geneva Conventions 75th anniversary: Foundational treaties save lives and dignity, but massive humanitarian suffering shows the world must recommit: On August 22 2024, the International Committee of the Red Cross (ICRC) marked the 75th anniversary of the Geneva Conventions. ICRC President Mirjana Spoljaric urged global leaders to reaffirm their commitment to these treaties, which protect lives and dignity in warfare. Despite successes, International Humanitarian Law faces strain due to escalating global conflicts. The ICRC also cautions that ‘if algorithms are trained on lax targeting rules, civilian casualties will increase. Without new legal limits, autonomous weapons might operate with little restraint, making life-and-death decisions without human oversight.’
• International: Exclusive: Workers at Google DeepMind Push Company to Drop Military Contracts: Time reports that approximately 200 employees from Google DeepMind, the AI division of Google, urged the company to end its military contracts. They expressed concern that DeepMind’s technology might be used in military applications, which they believe contradicts Google’s ethical principles. The petition reflects growing internal unrest about the alignment of Google’s AI use with its stated commitment to avoid contributing to harmful technologies. Despite Google’s assurances that its cloud services, including the Project Nimbus contract with Israel, are not used for sensitive military purposes, DeepMind employees feel their concerns have not been adequately addressed.
• International: AUKUS partners demonstrated ‘real-time’ AI tests at US Army’s Project Convergence: Breaking Defense recently reported on tests conducted on AI-enabled drones and autonomous systems during the ‘Project Convergence Capstone 4’ exercise. The AUKUS partners (Australia, the UK, and the US) test is part of the AUKUS ‘Resilient and Autonomous Artificial Intelligence Technologies (RAAIT) program’, aimed to enhance target identification and operational efficiency. The successful integration of these technologies in real-time military environments demonstrates rapid developments in AI and autonomous systems. This effort is part of AUKUS Pillar II, focusing on advanced technologies like AI and hypersonic missiles.
• Palmer Luckey’s Defense Startup, Anduril, Raises $1.5 Billion to Produce AI-Powered Weapons: Anduril Industries, a defense tech startup founded by Palmer Luckey has secured $1.5 billion in funding to scale up production of AI-powered weapons. This investment will build ‘Arsenal,’ a high-tech manufacturing platform designed to ‘produce tens of thousands of autonomous weapons a year’. Inspired by Tesla’s manufacturing efficiency, Anduril aims to rapidly produce these systems for the U.S. defense. The company’s approach aligns with Pentagon initiatives focused on increasing the production of autonomous military systems. Previous news briefing coverage of Anduril can be found here. 
• International: ‘Killer robots’ are becoming a real threat in Africa: This article addresses the growing concern over autonomous weapons systems in Africa, focusing on the use of drones in the Sahel region. It notes that drones equipped with AI, capable of conducting surveillance and autonomous attacks, have already been deployed by militant groups, raising fears of these weapons falling into the wrong hands. The article also discusses ethical issues surrounding autonomous weapons, such as their potential to cause civilian casualties and the challenges in distinguishing between combatants and non-combatants. It calls for international legal frameworks to regulate autonomous weapons, emphasizing the need for African nations to collaborate on this issue and advocate for global standards.
• UK: L3Harris shows off new autonomous air defence prototype: L3Harris has unveiled a new autonomous air defense prototype that was successfully tested during ‘Project Convergence Capstone 4’. This system integrates long-range surveillance and electronic warfare capabilities onto an autonomous combat vehicle, designed to enhance ground-based air defense with beyond-line-of-sight (BLOS) operations. The prototype demonstrated its ability to detect and engage air threats over 26 kilometers away and showed promising results in reconnaissance, surveillance, and target acquisition. ‘Further analysis and  development’ and integration into existing defense frameworks are needed to fully assess its potential.

Facial recognition, biometric identification, surveillance

• US:The US wants to use facial recognition to identify migrant children as they age: The U.S. Department of Homeland Security (DHS) is exploring the use of facial recognition technology to ‘track the identities of migrant children’ as they age. This initiative aims to improve the accuracy of facial recognition algorithms, which struggle with the rapid changes in children’s facial features. Concerns have been raised about the privacy and ethics of collecting biometric data from minors, especially those ‘under 14’. Although DHS has denied plans to collect such data, the initiative reflects a broader trend in increasing biometric data collection and surveillance. The project seeks to address technical challenges and privacy issues but has faced criticism for potential overreach and lack of transparency.
• Global: Surveillance Watch: Surveillance Watch is ‘an interactive map revealing the intricate connections between surveillance companies, their funding sources and affiliations’, which hopes to ‘expose the enablers fueling this industry’s extensive rights violations, ensuring they cannot evade accountability for being complicit in this abuse.’ Many thanks to Savena Surana of Identity 2.0, creators of Stop Killer Robots’ ‘Automated by Design’ exhibition, for sending us this resource! (Experience a taste of the Automated by Design exhibition for yourself here)
• US: NFL’s New Facial Recognition Policy Upsets Las Vegas Police Union: The Las Vegas Police Protective Association is concerned about a new NFL policy requiring security officers at Raiders games to ‘provide photos for facial recognition.’ The union fears that this biometric data might be misused or lead to harassment. The NFL claims the system will enhance security and is working with local police to address these concerns. 
• UK: Campaigners criticise Starmer post-riot public surveillance plans: UK Prime Minister Keir Starmer has introduced the ‘National Violent Disorder Programme’ to tackle recent far-right violence by improving intelligence sharing and expanding facial recognition technology. However, civil liberties groups worry the program may lead to increased surveillance and target peaceful protesters, raising concerns about privacy and civil rights. Critics argue that the program could exacerbate issues related to personal freedoms and civil liberties. Silkie Carlo, director of Big Brother Watch, criticized the UK government’s plan to expand facial recognition technology as ‘alarming.’ She argues that it ‘threatens’ democratic freedoms rather than safeguarding them,
• UK: ‘Mass surveillance’ fears amid calls to use AI to detect DWP benefit fraud: Campaigners are expressing strong concerns about the Tony Blair Institute’s proposal to significantly increase AI use in the UK’s Department for Work and Pensions (DWP). The Institute suggests ‘AI could handle 40% of DWP tasks’, potentially saving up to £1 billion annually. However, critics fear this could lead to intrusive mass surveillance of ‘benefit claimants’ bank accounts.’

AI, algorithms and autonomy in the wider world

• International: Continental Artificial Intelligence Strategy: The African Union Executive Council endorsed the Continental AI Strategy during its 45th Ordinary Session in Accra, Ghana, on July 18-19, 2024. The strategy emphasizes Africa’s commitment to a development-focused and ethical approach to AI. It advocates for coordinated national efforts among AU Member States to address AI challenges by ‘aiming to strengthen regional and global cooperation and position Africa as a leader in inclusive and responsible AI development.’
• South Africa: South Africa publishes national AI policy framework: This piece reports on the National AI Policy released by South Africa. The policy document maps the past and present circumstances and capabilities surrounding AI, to create a futuristic plan. The policy document specifically focuses on ‘ethical AI, focusing on fairness, transparency and the mitigation of biases, with particular attention to privacy and data protection. Additionally, it addresses the importance of maintaining human oversight over AI systems and aligning AI developments with South Africa’s societal values.’ 
• International: AI’s Limitations in Clinical Decision-Making: Thiago Borges, writing for Medscape, highlights the limitations of GPT-4V, a relatively new AI tool in healthcare. While it achieved an ‘accuracy rate of 81.6%’ in diagnosing medical cases, ‘slightly outperforming doctors,’ it struggled with image interpretation and justification accuracy with a ‘27.2% error rate in image comprehension’ and ‘flawed explanations in 35.5% of cases.’ The article emphasizes that AI should support and not replace human expertise and underscores the need for further research to improve its integration into clinical practice.

Research and Reports

• Report: Towards a Two-tiered Approach to Regulation of Autonomous Weapon Systems: Identifying Pathways and Possible Elements: This report by Laura Bruun, a researcher in the SIPRI Governance of Artificial Intelligence Programme, elaborates on the ‘context and contours’ of two-tier regulation approaches to autonomous weapon systems. This report draws insights from ‘scenario exercise convened by SIPRI in January 2024—where experts explored necessary limits and requirements for hypothetical AWS use cases.’ 

Other items of interest 

• Event: Addressing the Threat of Uncrewed Systems: What Actions for the Multistakeholder Community?: This side event at the CCW GGE on LAWS, scheduled for Thursday, 5 September 2024, 15:00-16:30 CEST / 09:00-10:30 EDT. It is the third and final webinar in a series being organised by United Nations Institute for Disarmament Research (UNIDIR) and the Office for Disarmament Affairs (ODA). There is mandatory registration for the event, which can be done here, and you can reach out to Ms. Mae Ballena (mae.ballena@un.org) if you have any questions.
• Event: Dialogue on killer robots will offer faith-based lens: This is a side event at the CCW GGE on LAWS scheduled for 27 August 2024, hosted by World Council of Churches and supported by Stop Killer Robots. This interfaith dialogue will have panelists ‘representing Christianity, Islam, Soka Gakkai, and the Baha’i faiths.’ The event can also be viewed live.