News Briefing 18-22 March 2024

Autonomy in weapons systems, and AI in security and defence

• International: Autonomous weapons: ICRC submits recommendations to UN Secretary-General: In its submission to the UN Secretary General, the ICRC underscores that ‘existing IHL rules do not hold all the answers to the humanitarian, legal and ethical questions raised by AWS’, and recommends the negotiation of a new, legally binding instrument on autonomy in weapons systems, which should prohibit unpredictable AWS and anti-personnel AWS, and include a number of restrictions and positive obligations for all other AWS.
• Ukraine: Ukraine Rolls Out Target-Seeking Terminator Drones: For Forbes, David Hambling reports on a ‘new technology’: ‘an attack drone which locks on with machine vision and does not require a human pilot.’ However, despite the headline, the system does not select and identify a target itself; as is noted in the article, the system in question is ‘semi-automatic in the sense that the target is identified and highlighted first by a human.’ 
• Ukraine: Ukraine Prepares To Roll Out An Army Of Ground Robots: This week, Forbes reported that Ukraine is currently in the process of producing ‘multiple types of military ground robots’. Nataliia Kushnerska, Chief Operating Officer of Brave1, (a platform created by the Government of Ukraine, that aims to promote collaboration between all stakeholders of the defense tech) says ‘Our main goal is to minimize human participation on the battlefield’.
• UK: United Kingdom military is not AI-ready, procurement minister says: This piece by Breaking Defense covers comments made by James Cartlidge, Minister of Defense Procurement, at the defense committee hearing. He has indicated that while AI has been deployed in multiple ways for defense, its main use case remains ‘the ability to hoover up data to analyze it at a far greater scale.’ He also pointed to measures in ‘better management of cloud computing and data access’ while setting up a ‘failsafe watchdog’ as part of a new procurement system policy. 
• US: Army has 1 unmanned system in Replicator’s initial tranche, 3 others in mind for next go: Official: Breaking Defense reports on the possible systems that will be incorporated by the US through its Replicator program (most recently covered in our news briefing on the 16th February 2024) in the coming years. While exact information on which systems might be part of the first few iterations of the program is scarce, Head of Army Acquisition Doug Bush said that the Army has nominated an existing project which fits the requirements of the replicator program. 

Regulation and enforcement

• UK: House of Lords must act now to stop Data Bill weakening rights: The Public Law Project in the UK has raised concerns regarding the Data Protection and Digital Information Bill which has moved into Committee Stage this week. Major concerns include the ‘flipping’ of presumption against solely automated decision making, lack of ‘proper impact assessment of data processing’, and absence of a ‘legal duty to ensure that public authorities are upfront about their use of AI.’

Facial recognition, biometric identification, surveillance 

• India: Indian biometric counterterrorism database passes 1B entries: Abhishek Jadhav, writing for Biometric Update, says ‘it is important to acknowledge the privacy and ethical considerations associated with large-scale data collection and surveillance practices.’ This comes at a time where a unit under the Indian government has established a ‘comprehensive database consisting of over a billion facial images that can be used for biometric identification.’
• International: Smart Engines claims new facial recognition system uses no biometric information: Smart Engines (a software development company, specializing in artificial intelligence and computer vision software) has recently expressed that it believes that its technology ‘solves the key issue of user privacy (as defined in the AI Act) by avoiding the use of sensitive biometric information’, by instead  The company ‘believes this approach will enhance user trust, regulatory compliance and security, facilitating the widespread adoption of its technology across the European Union.’
• UK: UK struggles with biometric border crossing software:  The Person Centric Data Platform, a UK government immigration database is said to have muddled up the ‘biometric and biographical data of more than 76,000 people.’ This has led to numerous ramifications, including ‘delays in immigration processing, long lines at borders and issuing faulty identity cards’, leaving ‘immigrants to the UK who have found that their information has been swapped with strangers ‘battling to prove their identity and access their rights.’ The Information Commissioner’s Office has been tasked to look into the ‘IT incidents’.
• US: Clearview facial recognition added to US federal procurement marketplace: Clearview started ‘gaining experience in military applications with a deployment in Ukraine in 2022’, and now has ‘a facial recognition algorithm and database of 40 million images’ that are accessible to ‘national security and military organizations.’ Clearview AI has previously been criticized for privacy breaches including sharing of data with law enforcement agencies. It has also been at the receiving end of fines from the UK Information Commissioner’s office and France’s CNIL. 

AI, algorithms and autonomy in the wider world

• International: UN General Assembly calls for guidelines on AI’s risks, rewards: France 24 reports on a resolution regarding ‘safe, secure and trustworthy artificial intelligence systems’, excluding military applications of AI, in the United Nations General Assembly prepared by the United States. It ‘focuses more on the technology’s positive potential’ as well as to ‘promote, not hinder, digital transformation and equitable access.’ Simultaneously, the proposed resolution also highlights risks associated with AI including ‘eroding of human rights, reinforcing prejudices and endangering personal data protection.’ 

Research and reports

• International: State Submissions on the Legal Review of AWS and Possible Good Practices: This report in Articles of War by Lauren Sanders, Senior Research Fellow at the University of Queensland’s Law and the Future of War Project, highlights the context and challenges to legal reviews of autonomous weapon systems. The piece points to six key challenges for legal reviews of autonomous weapons including evolving capabilities of such systems and contextual operational use.