News Briefing 15-26 July 2024

Autonomy in weapons systems, and AI in security and defence

• International: AI’s ‘Oppenheimer moment’: autonomous weapons enter the battlefield: The Guardian reports on the accelerating global race to incorporate artificial intelligence and autonomous capabilities in militaries. This global race is witnessing a large influx of funds from governments, and a rapid pace of technological development by industries associated with defense production. Automated Decision Research manager Catherine Connolly spoke to The Guardian, noting that the ‘money that we’re seeing being poured into autonomous weapons and the use of things like AI targeting systems is extremely concerning.’ While industry and countries with large investments in autonomous weapon systems urge restraint in regulating and prohibiting these weapons, a number of states, civil society and academics had called for a preemptive ban on fully autonomous weapons systems. However, the term ‘preemptive’ is no longer used as we are ‘so much closer to autonomous weapons’, notes Mary Wareham, deputy director of the crisis, conflict and arms division at Human Rights Watch.
• UK/International: Why the new UK government should support a treaty on autonomous weapons: Elizabeth Minor argues that the new UK Labour government should support the global drive towards an ‘international negotiation of a treaty to regulate autonomous weapons systems.’ She aligns this proposal with the government’s emphasis on international law, alliances, and partnerships in its foreign policy. Highlighting global momentum and ethical concerns, Minor contends that such a treaty would uphold humanitarian values, address emerging challenges posed by AI, and demonstrate UK leadership in global governance. She critiques the previous government’s cautious stance and damaging rhetoric against efforts for new international law and urges action at the UN General Assembly to advance this critical initiative with utmost urgency. 
• EU: How Europe’s next-generation combat jet aims to catch the AI wave: Linus Höller, writing for Defense News raises concerns about the ethical implications of autonomous decision-making and the potential for lethal autonomy that are ongoing. The debates are centered on how much control should be delegated to AI systems in combat scenarios. The Mainland Europe Future Combat Air System (FCAS), led by Germany, France, and Spain with Belgium as an observer, aims to develop a next-generation air combat platform by the end of the decade. This ambitious project will integrate advanced AI throughout the system, including autonomous drones, known as ‘loyal wingmen’, that will work alongside manned aircraft to enhance missions through tasks like data collection and combat.
• International: Navigating New Horizons: A global foresight report on planetary health and human wellbeing: The UN Environment Programme (UNEP) released a report aimed at responding with a strategic foresight to respond to a ‘range of emerging challenges that could disrupt planetary health and wellbeing.’ The report predicts that AI enabled weapon systems could possibly cause a disruption by 2030, through their operation without ‘human oversight’ and having the ability to ‘select targets and destroy them without direct human guidance.’ 
• New Zealand: Killer robot advisory group shut down at ‘critical time’: RNZ reports that the Ministry of Foreign Affairs in New Zealand has shut down the Autonomous Weapon Systems Working Group due to ‘resourcing constraints.’ Associate Minister of Foreign Affairs Todd McClay said in a statement that ‘lack of international progress towards agreement on rules and limits’ on autonomous weapon systems was one of the reasons for the discontinuation of the working group. Dr. Tom Gregory from Auckland University expressed disappointment and stated that ‘this is a very disappointing development at the precise moment that we are seeing AI-equipped weapons being used on battlefields in various conflicts…It’s very disappointing to see the government seemingly abandon its efforts to regulate these weapons.’

Facial recognition, biometric identification, surveillance 

• UK: Concerns over facial recognition at river festival: Bedfordshire Police deployed facial recognition technology for the first time at the Bedford River Festival to locate wanted offenders and ensure public safety. This move has drawn criticism from civil liberties groups like Liberty, raising concerns about privacy and unregulated surveillance. The police emphasised adherence to strict guidelines and acknowledged past biases in the technology but highlighted improvements in reducing these biases over time. Despite social media debates, Bedford Borough Council supported the initiative, emphasizing its role in enhancing event security and public confidence.
• US: Lawmakers raise concern, vow regulation of Mall of America’s new facial recognition software: CBS News reports on the implementation of facial recognition technology at the Mall of America. This has sparked concerns among Minnesota lawmakers, including Senators Eric Lucero and Omar Fateh, over privacy and data retention issues. The technology is used for security screening purposes to match faces against a database of individuals of interest. While the mall insists it ‘does not store facial data’ of non-suspects, ‘bipartisan legislators are calling for regulatory measures to ensure adequate safeguards and transparency in the use of such technologies.’ The discussions highlight the tension between enhancing security and protecting civil liberties in public spaces.
• New Zealand: NZ government asks for help understanding facial recognition: New Zealand’s government, through the Department of Internal Affairs (DIA), has issued a request for information (RFI) to gather insights on facial recognition technology. This initiative comes amid concerns over potential biases, particularly in law enforcement applications, and seeks to understand ‘market capabilities and challenges in deploying these technologies’. The government aims to establish a long-term commitment to biometric systems for ‘identity verification and fraud prevention’, while also addressing ‘fears of discrimination against ethnic groups like the Māori and Pasifika populations.’
• India: Facial recognition and AI monitoring deployed in India’s largest railway network: Biometric Update reports that the Indian Railways, whose 14,000 trains carry an estimated 9.5 million passengers daily,  plans on incorporating AI and facial recognition to its stations. The plan includes high definition 4k cameras and facial recognition algorithms enabled with deep learning systems which can analyze ‘motion, loitering, abandoned object detection and line crossing.’ 

AI, algorithms and autonomy in the wider world

• UK: Britain’s new government aims to regulate most powerful AI models: Last week, Reuters reported on Britain’s new Labour government, led by Prime Minister Keir Starmer, has announced plans to ‘explore regulations for artificial intelligence (AI) models’ but it has not yet specified the exact laws. This cautious approach contrasts with the previous government’s efforts, led by Rishi Sunak, who focused on positioning ‘Britain as a leader in AI safety’ without immediate targeted regulations. The debate highlights differing opinions among experts: some urging swift regulation due to rapid AI integration, while others, including the government, emphasize the need for careful consideration to maintain competitive advantages and manage risks effectively.
• International: Why does the Olympics have an ‘AI agenda’ and what does it mean for the future of sport?: The article discusses the International Olympic Committee’s (IOC) adoption of AI technologies for the upcoming 2024 Paris Olympics and beyond. The IOC aims to responsibly integrate AI while preserving Olympic values, as they prepare for the games scheduled to take place in Los Angeles in (2028) and Brisbane (2032). The article emphasises the potential benefits of AI in sports while underscoring the importance of ethical implementation and data security.
• US: AI is overpowering efforts to catch child predators, experts warn: The Guardian reports on a disturbing development, where generative AI systems are being used to produce explicit images of children. This content is reportedly flooding the dark web, and the internet, leading to child protection agencies being unable to track down and rescue victims of abuse. As these AI systems have been trained on publicly available datasets, there are also instances where explicit images have been created of children who have not been abused. 
• US: Proposed California law seeks to protect public from AI catastrophes: This piece in SiliconValley.com reports on the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, authored by Sen. Scott Wiener. The proposed law aims to regulate ‘development and deployment of advanced AI models’ through the creation of a ‘new regulatory body, the Frontier Model Division.’ Opponents of the bill argue that it fundamentally misunderstands advanced AI models, though Sen. Wiener says that the bill only enforces ‘safety evaluations that these large labs have already committed to perform.’
• International: AI models collapse when trained on recursively generated data: In this article, researchers highlight a potential issue with AI systems known as “model collapse,” where ‘indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear.’ Addressing this issue requires a combination of careful training practices and ongoing access to high-quality original data.