News Briefing 27 May – 7 June 2024

Autonomy in weapons systems, and AI in security and defence

• U.S.: Palantir lands $480M Army contract for Maven artificial intelligence tech: The U.S. Army has awarded Palantir a $480 million contract for its ‘Maven Smart System’ prototype, which ‘uses AI generated algorithms and memory learning capabilities to scan and identify enemy systems in the Area of Responsibility (AOR)’, fusing data ‘from various Intelligence Surveillance & Reconnaissance (ISR) systems to identify areas of interests’. 
• International: We have a unique opportunity to regulate autonomous weapons: In an interview with SwissInfo, ICRC legal advisor Georgia Hinds highlights that ‘AI isn’t developing itself. Humans are choosing to develop this technology and they continue to have a choice about when and how to apply it on the battlefield’, and further notes that ‘We now have a clear majority of states in favour of legally binding rules and so we’re optimistic that this momentum can produce an effective treaty by 2026.’
• International: Fighting Drones with Drones: Learning from Ukraine on the Future of Warfare: Natasha Karner, a Phd Student at RMIT University writes for the Australian Institute of International Affairs’ Australian Outlook. She writes that despite the promise that drones would lead to more ‘more precise’ wars using technology, the reality remains that ‘drones are likely to continue to lead to civilian deaths; contribute to widespread destruction; and exacerbate humanitarian crises’, while it should ‘go without saying that moving to fully autonomous drone warfare would mean the loss of meaningful human control over life and death decisions, with unknown outcomes for civilians.’ Further, while stating the move towards complete autonomy might develop ‘innocently’, she states that ‘moving to fully autonomous drone warfare would mean the loss of meaningful human control over life and death decisions, with unknown outcomes for civilians.’
• US: Anduril Is Building Out the Pentagon’s Dream of Deadly Drone Swarms: This piece in Wired by Will Knight highlights the US Air Force’s ongoing project to commission Collaborative Combat Aircrafts (CCAs) from Anduril and general Atomics. The CCA drones are said to be ‘envisioned performing a wide range of missions, including reconnaissance, air strikes, and electronic warfare—either alone or in collaboration with aircraft piloted by a human or autonomously.’ 
• Australia: Civilian deaths from killer drones are this generation’s ‘Oppenheimer moment’: Writing on autonomous weapons in the Sydney Morning Herald, Mary Wareham,  deputy director of the Crisis, Conflict and Arms Division at Human Rights Watch,underscoring that ‘All governments, including Australia’s, have a responsibility to act; to put in place the rules to protect humanity. Autonomous weapons systems are a grave problem that can affect any country in the world, so clear, strong, global rules are crucial.’ Australia does not currently support the negotiation of a legally binding instrument on autonomous weapons. 
• Republic of Korea: Russia’s war in Ukraine has turned South Korea into a manufacturing hotspot as it makes weapons faster and cheaper than the U.S.: Fortune reports that Hanwha Aerospace Co, the Republic of Korea’s leading defense contractor, has grown its workforce to some 7,000 in the past year in order to meet ‘growing demand’ for its weapons, with more than one-third of these employees ‘focused on research and development, including efforts to develop remote and autonomous weapons.’ The Republic of Korea does not currently support the negotiation of a legally binding instrument on autonomous weapons. 

Facial recognition, biometric identification, surveillance 

• UK: ‘I was misidentified as shoplifter by facial recognition tech’: This piece from BBC News reports on a case of mistaken identity due to the use of facial recognition systems by ‘Facewatch.’ The system flagged Sara and misidentified her as a shoplifter, and later apologized to her for the misidentification. This is not an isolated incident, and multiple similar reports have also surfaced, as per the piece. 
• Uganda: Ugandan human rights lawyer’s arrest exposes use of national ID for surveillance: In a story on the arrest of a Ugandan human rights lawyer, Biometric Update highlights that ‘the implementation of biometric and digital identity (BDI) programs in Uganda has given room for surveillance and intrusion on journalism and media in the region, unveiling that journalists in the country have become targets due to the mass collection of data under the government’s biometric and digital ID programs and its ability to engage in communications surveillance.’
• Sweden: Swedish proposal tests AI Act’s live public facial recognition limits: Biometric Update reports that the Swedish police are ‘considering the use of cameras equipped with real-time facial recognition to identify suspected criminals in public spaces. The proposed system would automatically scan the faces of individuals against a database of known offenders and alert law enforcement if a match is found.’