News briefing 17 Feb-3 March 2025

Autonomy in weapons systems, and AI in security and defence

• International: As Israel uses US-made AI models in war, concerns arise about tech’s role in who lives and who dies: U.S. tech companies like Microsoft and OpenAI have significantly enhanced Israel’s military capabilities in Gaza and Lebanon by providing AI tools for tracking and targeting militants. Following the October 7, 2023, Hamas attack, Israel’s reliance on these technologies surged, helping identify targets more quickly. However, the use of AI has also led to a sharp increase in civilian casualties, raising ethical concerns about its role in warfare. The Israeli military asserts that human oversight ensures compliance with international law, but AI-related errors, such as misidentifying targets, have resulted in wrongful deaths. While Microsoft and OpenAI have provided AI infrastructure, OpenAI has distanced itself from direct military partnerships, though it recently allowed national security use.
• International: The Ethics of Artificial Intelligence in Antiqua et Nova: The National Catholic Bioethics Center reports that the Roman Catholic Church has issued an important doctrinal note, Antiqua et Nova, concerning Artificial Intelligence or (AI). The note covers a number of ethical and humanitarian concerns posed by AI, and calls for an update to the Geneva conventions ‘to address this ethical issue of autonomous machine weapons that have the potential to be incredibly destructive.’ 
• US: Next iteration of Army’s ‘transforming-in-contact’ will focus on autonomy: The US Army is expanding its ‘transforming-in-contact’ initiative to focus more on robotics and autonomy as the country scales up to divisions and heavy brigades in its second phase, “transforming-in-contact 2.0.” This is reported to be an experiment with the inclusion of advanced autonomous systems, and to include unmanned ground vehicles and drones with the aim to enhance combat and logistics capabilities for the US. 
• International: AI, war and (in)humanity: the role of human emotions in military decision-making: This interesting ICRC blog post by Dr. Joanna L.D. Wilson argues for the essential role of human emotions in military decision-making, particularly when AI is used in warfare. While AI can enhance precision in tasks like target identification, Dr. Wilson stresses that ‘human judgment’, intuition, and empathy are indispensable for making ethical and morally sound decisions in complex and high-stakes situations. She highlights the importance of “meaningful human control” over AI, ensuring that humans remain accountable for military actions, especially when using lethal autonomous weapons systems. The ‘CCW GGE on LAWS has also stated that IHL compliance through the use of military AI requires “good-faith human judgement based on the assessment of the information available at the time, and the ability of the operators to make decisions and exert control over the use of force.”’
• International: Trump, Modi announce new US-India autonomous systems partnership: DefenseScoop reports that the U.S. and India have recently launched the ‘Autonomous Systems Industry Alliance (ASIA)’ to collaborate on developing and producing maritime drones and counter-drone systems. Announced by Presidents Trump and Modi, the initiative aims to strengthen regional security in the Indo-Pacific. The partnership includes a focus on advanced technologies, with Anduril Industries and Mahindra Group leading efforts in counter-drone and maritime systems. This is part of the new broader U.S.-India COMPACT partnership that aims to enhance defense cooperation across multiple domains and boost defense technology innovations.
• Havelsan unveils all-new Bulut reconnaissance and surveillance UAV: HAVELSAN, a Turkish defense and software company, has launched a new unmanned aerial vehicle (UAV) named BULUT. This new system builds upon existing systems by the company. BULUT is designed for reconnaissance and surveillance with a payload capacity of 5 Kgs and ability to be airborne for around 6 hours. It is advertised as having autonomous navigation features, and can share data over a range of 80 Kms. There is little detail to the exact features of the system, and we will monitor for additional details that surface. 
• International: The rise of autonomous warfare: a conversation with Dr Paddy Walker on his book “War Without Oversight: Why We Need Humans on the Battlefield”: In an interview with Dr. Iain Overton, Dr. Paddy Walker discusses his book War Without Oversight: Why We Need Humans on the Battlefield. Dr. Walker, an expert on autonomous weapons and military ethics, explores the challenges of AI-driven warfare and emphasizes the need for meaningful human control in autonomous weapon systems. The interview delves into the evolving role of autonomous warfare, moving from theoretical discussions to practical realities. Dr. Walker explores the technological feasibility of autonomy in real time battlegrounds, and calls for caution on excessive optimism for its incorporation. The conversation highlights the growing need for human oversight and accountability in the deployment of autonomous weapon systems.

AI, algorithms and autonomy in the wider world

• US: The Tech-Broligarch Nightmare: This captivating piece in CounterPunch by Ray Acherson, christens the technocratic totalitarian project in the US under the Trump administration as a ‘Broligarchy.’ They highlight the nexus between the silicon valley, defense industries and government and its far reaching impacts on people’s rights, surveillance, democracy and international security. They call for every one of us to take action to ‘save humanity from the destructive potential of AI.’
• US: Musk’s plan to replace government workers with AI could create chaos: The Department of Government Efficiency (DOGE) in the US is pushing for the replacement of government workers with AI systems, reports CIO. The Trump administration has already laid off around 10,000 federal workers, in pursuit of this replacement. Tech workers and industry professionals, however, are raising concerns about the shortcomings of AI systems as they are unable to ‘empathize, negotiate, or truly connect with citizens.’ Arguing that AI systems aren’t built for ‘human reasoning, context, or accountability’, the piece calls for human oversight over these systems. 

Facial recognition, biometric identification, surveillance

• Big Brother Watch condemns UK’s first use of city-wide facial recognition in Cardiff: Big Brother Watch has criticized South Wales Police’s deployment of city-wide facial recognition surveillance in Cardiff for the Six Nations rugby games. The system, designed to have capabilities and monitor key entry points, is seen as a significant expansion of police surveillance, potentially paving the way for widespread use across the UK. The group argues that this infringes on privacy rights, especially since no arrests have been made from past facial recognition deployments. 
• Mexico: New Legislation Sparks Privacy Concerns Amid Surveillance Rise: The article highlights privacy concerns regarding new security laws proposed by Mexico’s President Claudia Sheinbaum, including a ‘mandatory digital registration for mobile phone users.’ Critics argue these laws could lead to mass surveillance and government overreach, pointing to past failures of similar initiatives. The author also discusses concerns raised by personal experiences, such as content creator Daniel Giraldo’s observations of extensive surveillance in Switzerland, which sparked debates about ‘balancing safety with privacy’. Activists urge for security measures that respect privacy rights.
• New Zealand: New Zealand govt reviews privacy law amid battle over biometrics regulation: New Zealand is reviewing its privacy laws, particularly the Privacy Act 2020, to address barriers related to the use of facial recognition technology for retail crime prevention. The Ministry of Justice conducted a review in September 2024, but its findings have not been made public. This review coincides with ongoing discussions about a ‘new Privacy Code of Practice for Biometrics’, ‘which aims to regulate biometric data use more strictly. Privacy advocates are concerned that the government may be delaying the code’s adoption to enable broader use of biometric surveillance, while business groups argue the regulations may be too restrictive.
• International: Police Force Faces Backlash Over Facial Recognition Technology Linked to Surveillance in Gaza: This interesting piece also brings to the core the  concerns about Essex Police using facial recognition technology developed by Corsight ( a private Israeli company) known to be a controversial surveillance company working in Gaza and the West Bank. Critics argue that adopting such technology raises ethical concerns, particularly due to its association with human rights violations in conflict zones. Human rights advocates warn about the potential for racial bias and wrongful detentions. Essex Police defend their use of the technology, emphasizing its ‘effectiveness in public safety’, while experts urge a careful review of its ethical and legal implications. The debate also touches on broader issues around privacy and surveillance.
• US: We study mass surveillance for social control, and we see Trump laying the groundwork to ‘contain’ people of color and immigrants: This is a piece in the conversation by Dr Brittany Friedman, assistant professor in sociology at USC Dornsife College of Letters, Arts and Sciences, and Raquel Delerme, PhD candidate, Department of Sociology, University of Southern California. As the Trump administration moves towards increasing mass surveillance, Dr Friedman and Delerme highlight the role of causing moral panics unfairly targeting marginalized communities in justifying it. They warn that this would enable law enforcement agencies to ‘employ advanced military technology to gather massive amounts of personal data on people perceived as potential security threats.’