News Briefing 08 November 2024

Autonomy in weapons systems, and AI in security and defence

• US: Silicon Valley defense tech can’t wait for Trump to get started: Forbes reports that ‘the impending arrival of a second Trump administration has brought renewed excitement for Silicon Valley’s ambitions to make weapons and secure the large contracts needed to support them’. The piece notes that shares in Palantir (which is heavily involved with pushing states to adopt AI for military use) ‘soared 8.61% on Wednesday’, and that Jacob Helberg, an advisor to Palantir, ‘has spent much of this year advising Trump’s campaign on AI policy.’ Silicon valley ‘defense leaders’ reportedly believe that with Elon Musk’s input, ‘Trump will upend and reallocate the $800 billion Department of Defense budget’. The piece quotes Musk’s recent tweet in which he said that ‘it is “Very important to open DoD/Intel to entrepreneurial companies like [Anduril]. Pay for outcomes, not requirements documents!”’. See here for previous ADR news briefing coverage of Palantir and coverage of Anduril.
• US: How DOD will help agencies comply with the White House’s new rules for AI in national security: DefenseScoop reports that Pentagon officials who produced its first ‘Responsible AI toolkit’ will ‘help steer multiple activities that will contribute to the early implementation of the new White House memo guiding federal agencies’ adoption of trustworthy algorithms and models for national security purposes.’
• Türkiye: Türkiye unveils armed robot dogs for military operations VIDEO: Turkish technology company Quad Robotics and defence firm Savhatek have ‘collaborated to create an armed robot dog equipped with a lightweight, remote-controlled weapon system’, and claims that they feature ‘AI-driven targeting systems’ and ‘autonomous target tracking and obstacle avoidance’.
• US: OpenAI is quietly pitching its products to the U.S. military and national security establishment: Fortune reports that Open AI ‘appears to be veering into the world of defense and military contracts’, and notes the company’s recent hiring of Palantir’s former chief information security officer to fill the same role at Open AI. 
• International: AI in Military Decision Support Systems: A Review of Developments and Debates: The Autonorms project has released a review of developments and debates on the use of AI in military decision support systems, provides a review of ‘the main developments in relation to AI DSS (at the time of writing in September 2024), focusing on specific examples of existing systems; and ‘the main debates about opportunities and challenges related to various uses of AI DSS, with a focus on issues of human-machine interaction in warfare.’
• Australia: ADF unveils robotic and autonomous weapons amid calls for stronger ethical guidelines: In an article on the further investment and development of robotic and autonomous weapons systems by the Australian Defence Forces, this article highlights comments by Australia’s High Commissioner for Human Rights, Lorraine Finlay, who noted that ‘previously Australia’s taken the position that it was premature to regulate’, but that she’s hoping that recent votes in favor of UN resolutions on autonomous weapons ‘shows a shift in that position and a recognition that the time is now to actually address these issues and make sure that those safeguards are put in place.’
• China/US: Exclusive: Chinese researchers develop AI model for military use on back of Meta’s Llama: Reuters reports that top Chinese research institutions linked to the People’s Liberation Army ‘have used Meta’s publicly available Llama model to develop an AI tool for potential military applications, according to three academic papers and analysts.’ Following this report, The Guardian reported that Meta is set to allow US national security agencies and defense contractors to also use Llama for such purposes, despite typically prohibiting the use of its open-source language model for ‘military, warfare, nuclear industries or applications, [and] espionage’. However, according to Bloomberg, ‘the company is making an exception for US agencies and contractors as well as similar national security agencies in the UK, Canada, Australia and New Zealand.’
• US/International: Considering a legally binding instrument on autonomous weapons: Charlie Trumbull, an Assistant Professor of Law at the University of South Carolina Joseph F. Rice School of Law and former Attorney-Adviser in the Office of the Legal Adviser at the U.S. Department of State, writes that ‘it appears inevitable that negotiations on an LBI [on autonomous weapons systems], such as a multilateral treaty or CCW Protocol, will take place in some forum’, that ‘negotiations on an LBI on autonomous weapons are likely to commence within the next few years’, and proffers his view that ‘the U.S. should seek an active role in shaping a seemingly inevitable LBI on autonomous weapons.’
• International: Binding Rules Urgently Needed for Killer Robots: Three french civil society organizations, l’Observatoire des armements, Human Rights Watch and Amnesty International France write this Op-ed (originally in French) to call for the urgent negotiations of a legally binding instrument on autonomous weapon systems. The piece says it is ‘highly uncertain’ whether autonomous weapons will be able to apply IHL principles of distinction and proportionality, which are ‘case-by-case decisions that require human judgment.’ 
• US: Nearly 60 militaries now endorse U.S.-launched pledge on ‘responsible AI’: During the CDAO’s Responsible AI in Defense Forum, Deputy Defense Secretary Kathleen Hicks shared that the US Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy is now backed by 60 militaries. Defensescoop reports that the countries that recently joined include Ukraine, Panama, Ethiopia, Israel, and New Zealand.
• US: How new White House AI memo impacts, and restricts, the Pentagon: Breaking Defense reports on the National Security Memorandum, signed by President Joe Biden which requires ‘human oversight, safety testing and other safeguards for many military and intelligence applications of artificial intelligence.’  While the policy does not ‘impose significant new restrictions on AI-controlled drones, munitions’ and autonomous weapons, the framework ‘restricts use of AI to forecast likely civilian casualties and collateral damage when planning a potential military strike.’ 

Facial recognition, biometric identification, surveillance

• Israel/ Palestine: Witnesses say the Israeli army is using facial recognition technology in its assault on north Gaza: Mondoweiss reports on the use of facial recognition technologies by the Israeli military to surveil and interrogate Palestinians. The piece includes testimonies from individuals who have experienced this, who describe that ‘the most terrifying and frightening moments were the moments when you stand in front of the camera to get your face scanned.’
• Ireland: Facial recognition tech could mean innocent people being misidentified as criminal suspects: The Irish Council for Civil Liberties has unveiled an art mural in Dublin highlighting how ‘Garda use of facial recognition technology could lead to innocent people being misidentified as criminal suspects, with women and people of colour at increased risk.’ Olga Cronin, Surveillance and Human Rights Senior Policy Officer with ICCL said that police ‘access to FRT would completely change Irish policing and the relationship between gardaí and the communities they serve, in particular minoritised communities.’ 
• UK: Scottish review calls for clearer standards for police in biometric data retention: Biometric Update reports on a detailed review by the Scottish Government, in partnership with the Scottish Biometrics Commissioner, examining ‘current policies surrounding the collection, retention, and destruction of biometric data such as fingerprints, DNA profiles, and custody images by Police Scotland, aiming to ensure these practices align with legal and ethical standards.’ The review highlights that there is no clear policy specifically for biometric data, and broader criminal data retention rules are presently being applied. 
• Norway: Norway reveals plans to contract 21 biometric border control gates: This piece in Biometric Update reports that the The Norwegian Police Shared Services division has allocated funds for a contract to install biometric systems at airports which ‘includes the biometric gates and biometric sensors themselves, plus software development, support, and repair and maintenance services.’ 

AI, algorithms and autonomy in the wider world

• US: Harris and Trump’s shared goal masks a fundamental AI policy divide: Alex Krasodomski, Program Director, Digital Security Program wrote this piece for Chatham House, prior to the U.S. presidential election, on the possible shift in AI policy depending on the US election results. He says that while the Biden administration focused on ‘responsibility’ and ‘anti-trust’ efforts, Trump’s policy would likely lean towards minimal regulation. 
• International: AI That Can Invent AI Is Coming. Buckle Up: This piece in Forbes analyzes the concept of ‘AI explosion’ sparked by ‘self improving AI’ as theorized by prominent thinkers such as I.J. Good, Nick Bostrom, and Leopold Aschenbrenner. Though the piece says that self improving AI at present is not a reality, it is increasingly becoming viable with large scale and multi sectoral impacts.