News briefing 25-29 March

Autonomy in weapons systems, and AI in security and defence

• International: US joins Austria, Bahrain, Canada, & Portugal to co-lead global push for safer military AI: Breaking Defense reports that delegates ‘from 60 countries met last week outside DC and picked five nations to lead a year-long effort to explore new safety guardrails for military AI and automated systems’, with the countries involved forming ‘three working groups to delve deeper into details of implementation’ of the 10 principles contained in the U.S Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy initative. As a February 2023 Human Rights Watch report on US DoD Directive 3000.09 underscores, U.S. policy on autonomous weapons is ‘out of step with widely supported international proposals for a new treaty to prohibit and regulate autonomous weapons systems’. More than 100 countries support the negotiation of a new legally binding instrument on autonomous weapons systems. 
• US: How Silicon Valley’s ‘Oppenheimer’ found lucrative trade in AI weapons: The Financial Times carries a profile of Palmer Luckey, founder of defence company Anduril Industries, in which Luckey says that ‘The thing that’s so powerful about autonomy is that you can clearly show your adversaries that you have weapons that do not cost all that much money and that don’t cost human life’. The piece notes that ‘all of Anduril’s tech is capable of operating almost entirely unmanned’, with ‘discretion as to how autonomous it is when deployed is down to users and policymakers’. Anduril’s ALTIUS loitering munition is featured here in our weapons monitor. Coverage of Anduril in previous news briefings can be found here. 
• US: The Air Force Bought a Surveillance-Focused AI Chatbot: 404Media reports that the U.S. Air Force has paid for ‘a test version of an AI-powered chatbot to assist in intelligence and surveillance tasks as part of a $1.2 million deal’. A representative of the Air Force Research Laboratory Public Affairs told 404 Media in a statement that ‘The Department of the Air Force isn’t using this program at all, nor do we have any intention of using it for targeting decisions. The program is merely at the early stages of development to determine if there is a use case for the Department of the Air Force.’
• US: Why the Pentagon wants to build thousands of easily replaceable, AI-enabled drones: In a piece on U.S. plans for drone swarms via its Replicator programme (featured in numerous past news briefings), Vox notes that while ‘both sides in the Ukraine war claim to be using artificial intelligence to improve their drones’ performance’, it is likely that ‘so far, any use has probably been limited, but the war has also accelerated development of these capabilities.’ The article also notes that ‘the shift to a greater emphasis on autonomous systems comes at a time when the military is scaling back on its human manpower.’
• South Africa: Media Statement: Parliamentary Delegation Proposes Amendments To Draft Resolutions On Peace, Security And Climate Action: Earlier this week, the Inter-Parliamentary Union (IPU) convened in Geneva, where Hon Doris Dlakude (representing the majority party in South Africa) spoke on the South African Parliament’s proposed amendments to the draft resolution on the social humanitarian impact of autonomous weapon systems and artificial intelligence. Dlakude noted that ‘the risks and consequences stemming from the proliferation of such advanced technologies have the potential to unleash profound social and humanitarian implications, and it is our responsibility as parliamentarians to address these critical challenges proactively’, and that ‘the absence of human intervention in decision-making processes poses a fundamental threat to the ethical and legal standards that underpin the principles of warfare.’ South Africa’s proposed amendments include a prohibition on autonomous weapons systems targeting humans directly. 

Facial recognition, biometric identification, surveillance

• Gaza/Israel: Israel Deploys Expansive Facial Recognition Program in Gaza: The New York Times reports that Israel is deploying a facial recognition programme in Gaza, run by Israel’s military intelligence unit. According to The New York Times, ‘the facial recognition program in Gaza grew as Israel expanded its military offensive there. Israeli soldiers entering Gaza were given cameras equipped with the technology. Soldiers also set up checkpoints along major roads that Palestinians were using to flee areas of heavy fighting, with cameras that scanned faces.’ Using technology from Corsight, a private Israeli company, and Google Photos, ‘the technologies enable Israel to pick faces out of crowds and grainy drone footage’. The New York Times notes that ‘Israel’s use of facial recognition in Gaza stands out as an application of the technology in a war’, and quotes Amnesty International researcher Matt Mahmoudi, noting the dehumanising effects of facial recognition technologies. The piece also highlights ‘instances of  false positives, or cases when a person was mistakenly identified as being connected to Hamas’.
• China/UK: Threats, fear and surveillance: how China targets students in the UK who criticise regime: According to this piece in The Guardian, in recent years there has been a high level of uneasiness concerning ‘the impact of Chinese surveillance and influence’ in several countries. The article focuses on Chinese nationals based in the UK who are raising fears of being surveilled. The Guardian interviewed ‘several students and recent graduates from China…who say they have been subjected to surveillance, harassment and other forms of intimidation by Chinese authorities for criticising the Chinese Communist party while on British soil.’
• UK: Fears Over ‘Mass Surveillance’ With Bill Changes: A UK industry group, along with a number of academics, civil liberties groups, and think tanks have raised concerns around proposed amendments to the UK’s Investigatory Powers act, including worries that proposed changes could ‘weaken safeguards around the use of bulk datasets of personal information, potentially enabling the harvesting of millions of facial images and social media data.’
• US: Revealed: a California city is training AI to spot homeless encampments: The Guardian reports that the city of San Jose has been ‘training artificial intelligence to recognize tents and cars with people living inside in what experts believe is the first experiment of its kind in the United States’, with local outreach workers worrying that ‘the technology will be used to punish and push out San Jose’s unhoused residents.’ The article further notes that ‘In addition to providing a training ground for new algorithms, San Jose’s position as a national leader on government procurement of technology means that its experiment with surveilling encampments could influence whether and how other cities adopt similar detection systems.’
• US: ‘They need to shut it down now’: Concerns raised over downtown Lakeland’s facial recognition cameras:  This piece focuses on concerns over the facial recognition technology in a Florida city, in which real-time facial recognition technology is being used in new security cameras. The piece quotes Nate Freed Wessler, Deputy Director of ACLU’s Speech Privacy and Technology Project, who said that ‘this technology used this way to track and identify people in real time is really chilling in a free society.’

AI, algorithms and autonomy in the wider world

• Africa/International: Artificial Intelligence and the integrity of African elections: The discussion here highlights a crucial matter in Africa related to the role of AI in electoral processes, given that several African States will hold elections this year. The piece asks what impact AI will have on voters and voting processes’, and urges African policymakers ‘to create a regulatory framework to inhibit the manipulation of AI to undermine election integrity, information integrity and democracy.’
• US: VP Harris says US agencies must show their AI tools aren’t harming people’s safety or rights: According to a White House announcement this week, ‘Agencies that can’t apply safeguards must cease using the AI system, unless agency leadership justifies why doing so would increase risks to safety or rights overall or would create an unacceptable impediment to critical agency operations.’