News Briefing 17-21 June 2024

Autonomy in weapons systems, and AI in security and defence 

• International: Advocating for a legally binding instrument on autonomous weapons systems: which way ahead:  A new issue of the e-journal Ethics & Armed Forces focuses on AI and autonomy in weapons systems, opening with an essay by Automated Decision Research manager Catherine Connolly. The essay highlights that ‘in an area in which life and death decisions are in question, industry cannot be left to self-regulate, and nor can more powerful and militarised states be left to distract the international community with non-binding principles and declarations’.‘ The issue also includes a contribution by Andreas Bilgeri, Counsellor and disarmament expert at the Permanent Mission of Austria in Geneva, who stresses that ‘the hesitancy of a few States should not stop the rest of us from moving forward and creating a legally binding norm’
• International: Pope calls on G7 leaders to ban use of autonomous weapons: In his address to the meeting of G7 leaders last week, Pope Francis exhorted states to make ‘an effective and concrete commitment to introduce ever greater and proper human control’ over weapons systems, stating that ‘No machine should ever choose to take the life of a human being.’ Pope Francis called for states to ‘reconsider the development and use of devices like the so-called “lethal autonomous weapons” and ultimately ban their use.’ Meanwhile, an editorial in L’Osservatore Romano, the daily newspaper of the Vatican City, emphasises that ‘War is a serious violation of human dignity. Waging war while hiding behind algorithms, relying on artificial intelligence to determine targets and how to hit them, thus relieving one’s conscience because it was the machine that made the decision, is even more serious.’
• International: Time for an AI arms control agreement?: Writing in the Asia Times, Manoj Harjani, research fellow and coordinator of the Military Transformations Program at the S Rajaratnam School of International Studies in Singapore, notes that ‘In the ongoing conflict in Gaza, the alleged use of AI for targeting should raise alarm bells and motivate greater efforts towards regulation and arms control’.
• International: Rheinmetall and Anduril Industries join forces to develop the most sophisticated military C-sUAS system: The German defence company Rheinmetall has signed a memorandum of understanding with U.S. defence tech company Anduril Industries, which will see the companies working together to ‘design, develop, and produce new layered Counter small Unmanned Aerial System (C-sUAS) air defense systems.’
• UK/China: Chinese firm sought to use UK university links to access AI for possible military use: The Guardian reports that a ‘Chinese state-owned company sought to use a partnership with a leading British university in order to access AI technology for potential use in “smart military bases”.’

AI, algorithms and autonomy in the wider world

• International: AI is coming for our anger: SoftBank is working on an AI-powered system called the “EmotionCancelling Voice Conversion Engine” to mitigate the impact of customer harassment on call center workers by altering the tone of angry callers in real time. This technology aims to ease the psychological burden on operators but raises ethical questions about manipulating emotions without consent and the broader implications for human communication.
• International: Neo-Nazis Are All-In On AI: Extremists are exploiting AI technology to ‘enhance radicalization efforts, facilitate fundraising, and create weapon blueprints and bombs’. This development poses serious threats as it accelerates the spread of hate and potentially increases the danger to public safety. 

Reports

• China/International: China’s Military AI Roadblocks: PRC Perspectives on Technological Challenges to Intelligentized Warfare: The report examines Chinese perspectives on military AI development based on analyses of Chinese-language journal articles. Key findings include Chinese defense scholars acknowledging AI’s potential to enhance military capabilities vis-à-vis the U.S. Chinese experts also emphasize challenges in data management, sensor development, and inadequate testing standards. They caution against deploying unreliable AI systems due to risks of escalation and operational inefficiency in military contexts. While China sees AI as crucial for military modernization, significant hurdles remain in achieving effective integration into military operations. The report is available here

Facial recognition, biometric identification, surveillance

• US/International: Surveillance risks posed by AI among the technology’s most urgent problems: Hinton: Geoffrey Hinton, a leading AI pioneer, has shifted focus from AI’s distant existential threats to immediate concerns about its potential for pervasive surveillance, particularly under authoritarian regimes. Speaking at the Collision conference, he highlighted risks such as ‘fake videos, election manipulation, cybercrime, and socioeconomic inequality’ due to AI-driven job displacement. Hinton’s remarks reflect ongoing debates in the tech community on ethical AI development and the need for robust regulatory safeguards.
• US: California bill on police use of facial recognition passes second reading: Assembly Bill 1814 in California aims to regulate the use of facial recognition technology by law enforcement. It prohibits FRT-generated matches from being the sole basis for probable cause in arrests, searches, or warrants, requiring corroborating evidence. Critics, including those wrongly arrested due to FRT errors, argue the ‘bill doesn’t adequately address bias and unreliability concerns’.
• UK: How AI is used in UK train stations: Recent trials of AI technology in UK public transport, including Network Rail and Transport for London, have shown promising results in enhancing safety, reducing fare evasion, and improving operational efficiency. However, concerns have been raised regarding the ‘ethical implications of AI’, particularly in its ability to analyze facial expressions and predict emotions, which could lead to privacy violations and biases.
• US: The TSA’s facial recognition program could protect privacy rights. Here’s how: The recent Federal Aviation Authorization Bill passed without including a bipartisan amendment that sought to temporarily halt the TSA’s facial recognition program due to concerns over privacy rights. Proponents argue that facial recognition technology could streamline airport security by verifying identities quickly amid increasing passenger volumes and rising identity fraud. However, critics, including privacy professionals, emphasize the unique sensitivity of facial data, which could be exploited by malicious actors. They advocate for stringent safeguards to protect biometric privacy, especially as facial recognition technology evolves and becomes more integrated into everyday security and authentication processes.