News briefing 11-22 November 2024

Autonomy in weapons systems, and AI in security and defence

• International: First CCW meeting after UNGA resolution maintains status quo: This blog piece by Stop Killer Robots expresses concern on the lack of substantial progress at the meeting of High Contracting Parties of the CCW, which took place in Geneva from 13-15 November 2024. It says that ‘observers, international organisations, and civil society are increasingly blocked from meaningful participation’ and that this raises ‘serious concerns about the transparency and inclusivity of the CCW as a forum that can adequately respond to the real and pressing challenges posed by autonomous weapons.’ 
• Ukraine/US: Newest Replicator drones proven on battlefields of Ukraine: The article discusses the Pentagon’s Replicator program, which focuses on rapidly deploying low-cost, autonomous systems, including drones, to enhance military capabilities. These systems, such as Anduril Industries’ Ghost-X and the C-100 UAS, have been used in Ukraine to counter Russian forces. 
• Ukraine: Understanding the Military AI Ecosystem of Ukraine: The report discusses Ukraine’s rapid adoption of AI in its military, driven by the war with Russia. AI is used in supportive roles like autonomy and intelligence analysis but is still experimental. The government focuses on adopting commercial AI technologies, though there is no long-term strategy for integration. 
• Japan: Japan to conduct first artificial intelligence-driven unmanned aircraft flight in 2025: Japan is preparing for the maiden flight of an AI-equipped unmanned aircraft, the “Flying Test Bed” (FTB), in November 2025. Developed by Mitsubishi Heavy Industries, the FTB is designed to test AI capabilities for unmanned aerial vehicles (UAVs). It features a modular design for switching between combat and reconnaissance roles, with sensors for enhanced operational testing. The project is part of Japan’s broader effort to develop autonomous UAVs that can work alongside manned fighter jets, in collaboration with the U.S. Air Force.
• US: The AI Machine Gun of the Future Is Already Here: The article covers the development of “Bullfrog,” an AI-powered robotic gun system designed to counter drone threats. Created by Allen Control Systems, Bullfrog uses a 7.62mm M240 machine gun and AI software to autonomously track and destroy fast-moving drones. While currently requiring human oversight for kill decisions, Bullfrog is capable of full autonomy. This innovation is part of broader Pentagon efforts to enhance counter-drone capabilities, amidst growing concerns over drone warfare.
• US/International: The AI Startup Anthropic, Which Is Always Talking About How Ethical It Is, Just Partnered With Palantir: Anthropic, an AI company focused on safety, has partnered with defense contractor Palantir and Amazon Web Services to provide its chatbot, Claude, to U.S. military and intelligence agencies. This allows Claude to process classified data, raising ethical concerns about AI’s flaws like inaccuracies and data leaks. The move highlights growing ties between AI companies and the military, sparking criticism about prioritizing profits over safety.
• International: Meta now allows military agencies to access its AI software. It poses a moral dilemma for everybody who uses it: Meta has announced it will allow U.S. government agencies, including defense and national security entities, to use its generative AI models, Llama, despite previously prohibiting military applications. This controversial decision raises ethical concerns, as Llama powers tools in widely used platforms like Facebook, Instagram, and WhatsApp, potentially connecting civilian usage to military programs. Previous reporting on this can be accessed here.
• Finland: Poll: Finns cautiously optimistic about artificial intelligence: A new Finnish poll shows strong trust in science, though skepticism and interest in alternative treatments like homeopathy are growing due to recent crises. Finns are cautiously optimistic about AI, favoring its use in crime prevention and customer service but opposing autonomous weapons and the ‘use of AI in  journalism applications.’ According to communications professor Esa Väliverronen of Helsinki University, trust in institutions like the ‘military and universities remains stable, while confidence in political parties, the EU, and the church is declining.’
• US: Air Force researchers eye artificial intelligence (AI) and machine learning for advanced target tracking: The U.S. Air Force is seeking innovative AI and machine learning solutions for its Advanced Tracking Architecture Using what is termed the  ‘AI (ATA-AI) project’ , focusing on next-generation target tracking. The $99 million program aims to develop new algorithms for 3D data processing, signal classification, and pattern learning using diverse data sources. 

Facial recognition, biometric identification, surveillance 

• Denmark: Denmark: AI-powered welfare system fuels mass surveillance and risks discriminating against marginalized groups – report: This report by Amnesty International brings to light how ‘the Danish welfare authority, Udbetaling Danmark (UDK), risks discriminating against people with disabilities, low-income individuals, migrants, refugees, and marginalized racial groups through its use of artificial intelligence (AI) tools to flag individuals for social benefits fraud investigations.’ The report says that the UDK ‘uses a system of up to 60 algorithmic models’ which are trained on databases which include personal ‘information on residency status and movements, citizenship, place of birth, and family relationships — sensitive data points that can also serve as proxies for a person’s race, ethnicity, or sexual orientation.’ 
• Australia: Australian Privacy Commissioner’s FRT ruling could have major implications for retail: Biometric Update reports that Australian Privacy Commissioner Carly Kind has found Bunnings Group Limited ‘breached citizens’ privacy by collecting personal and sensitive information through a facial recognition technology system.’ 
• UK: Home Is the New Prison: UK’s High-Tech Digital Prison Plans Should Spark Privacy Fears For Everyone: This report in Reclaimthenet says that the UK plans on incorporating a new form of home arrest or ‘virtual prisons’ which will use multiple forms of surveillance including wearable devices. The plan, reportedly, is to ‘make people serve their serious, 5+ year sentences – at home.’ 
• For readers in the U.S: The WIRED Guide to Protecting Yourself From Government Surveillance: This piece in WIRED provides a detailed guide on how users can protect their data to ensure that increased surveillance would not lead to persecution. The piece backgrounds this guide on the recent US presidential elections, and the possible increased surveillance of ‘undocumented immigrants, Muslims, pregnant people, journalists, really anyone who doesn’t support’ them,  says Runa Sandvik, a former digital security staffer for The New York Times and the founder of the security firm Granitt. 

AI, algorithms and autonomy in the wider world

• International: Anthropic CEO Says Mandatory Safety Tests Needed for AI Models: Anthropic Chief Executive Officer Dario Amodei spoke at the AI safety summit in San Francisco hosted by the US Departments of Commerce and State. This piece by Bloomberg reports his comments on AI regulation, where he said that more explicit AI regulation is needed to replace the ‘patchwork of voluntary, self-imposed safety guidelines that major companies have agreed to.’ 
• International: AI cloning of celebrity voices outpacing the law, experts warn: The Guardian reports on the legal lacunae surrounding voice cloning, and its likely impacts on controlling fraud. Dr Dominic Lees, an expert in AI in film and television who is advising a UK parliamentary committee, said that ‘the government definitely needs to look at [voice cloning], because it’s a major issue for fraud’ as there are reports that ‘AI voice cloning scams were up 30% in the UK in the last year.’
• International: Switzerland’s AI Jesus answers questions of faith: An AI avatar of Jesus Christ has been unveiled at a small church in Lucerne, Switzerland, which has replaced the priest in the confession box, and answers questions about faith, reports DW. It has started conversations on the impact of technology on pastoral care, though there have also been criticisms against this. Peter G. Kirchshläger, Lucerne University theologist and philosopher, says that matters of faith are something humans have advantage at, and should not be delegated to machines.