News Briefing 22-26 January 2024

Autonomy in weapons systems, and AI in security and defence

• International: Bulletin of the Atomic Scientists 2024 Doomsday clock statement: The Bulletin of the Atomic Scientists have set their doomsday clock at 90 seconds to midnight again this year, ‘because humanity continues to face an unprecedented level of danger’. The statement notes that ‘Military uses of AI are accelerating’, states that ‘lethal autonomous weapons, which identify and destroy targets without human intervention’ are of particular concern, and argues that ‘much more must be done to institute effective rules and norms, despite the daunting challenges involved in governing artificial intelligence.’ 
• Israel and Gaza: Algorithms of war: Military AI and the war in Gaza: Omar Yousef Shehabi, Acting Assistant Professor at NYU School of Law and a JSD candidate at Yale Law School, and Asaf Lubin, Associate Professor of Law at Indiana University Maurer School of Law, write on ‘the wartime use of artificial intelligence (AI) as a decision support tool in targeting’, noting that while ‘very little is publicly known about Israel’s AI programs’, the use of decision support tools ‘as manifested in Gaza puts into question whether military AI could ever be deployed in a manner protective of IHL rules.’ The authors further highlight that ‘These systems are only semi-automated, which means that there is always a human-in-the-loop’, and ask ‘But what value does this human supervision bring?’
• US: Ex-Google CEO Eric Schmidt Is Working On A Secret Military Drone Project: Forbes reports that former Google CEO Eric Schmidt has been ‘quietly building a new drone startup in the U.S. and Ukraine’. Called White Stork, the company ‘has been developing a mass-producible drone that uses artificial intelligence for visual targeting and can function in zero-comms environments created by GPS jamming.’ No further details regarding the levels of autonomy (or lack thereof) that this system might have are currently available.
• US: Pentagon says it created more hoops to jump through on AI weapons to calm fears it was ‘building killer robots in the basement’: Business Insider reports that at a Center for Strategic and International Studies (CSIS) Wadhwani Center for AI and Advanced Technologies panel on The State of DoD AI and Autonomy Policy earlier this month, Deputy Assistant Secretary of Defense for Force Development and Emerging Capabilities Dr. Michael C. Horowitz stated that the U.S. Department of Defense had updated its Directive on autonomy in weapons systems in 2023, as ‘we had ended up in a situation where outside the department, the community of interest thought that DoD was maybe building killer robots in the basement’, and that inside the DoD ‘there was a lot of confusion about what the directive actually said, with some actually thinking the directive prohibited the development of autonomous weapons with maybe particular characteristics or even in general.’ A Human Rights Watch analysis of the Directive from February 2023 is available here. 
• US: The Pentagon is already testing tomorrow’s AI-powered swarm drones, ships:  DefenseOne carries a story stating that US Department of Defense ‘breakthroughs’ in ‘experimental aerial and naval craft are paving the way for low-cost attack drones and new tactics that feature AI in key roles’, and features discussion of advances in different AI and autonomous technologies in weapons systems.
• India: Why AI in defence is a double-edged sword: An op-ed in India Today magazine by Gen. (Dr) M.M. Naravane, former Chief of Staff of the Indian Army, argues that ‘With the development of AI for military use, there is a need to have international conventions governing the use of AI on the lines of the Convention on Certain Conventional Weapons. One key feature of such a convention should be the obligation to have a human in the loop who bears the moral responsibility of the final decision.’ 

Regulation and enforcement

• France: French regulator fines Amazon for surveillance of warehouse staff: Amazon France Logistique is said to have ‘used scanning technology to monitor staff performance in packaging orders.’ French data protection authority CNIL has resorted to fining Amazon an amount of €32m ($35 million) ‘for excessive surveillance of staff at its warehouses in France.’ CNIL notes that surveillance of this nature ‘amounted to being “excessively intrusive” and a breach of the EU’s General Data Protection Regulation.’
• International: Open Letter To Coe AI Convention Negotiators: Do Not Water Down Our Rights: This week in Strasbourg, the Council of Europe is approaching ‘a finishing line of negotiations’ of the Convention on AI, which seeks to ‘protect human rights, democracy, and the rule of law.’ In a joint open letter, European civil society organisations are calling for “states to make sure the AI Convention exempts neither private companies nor ’national security’ use cases.” The letter is open and is available here should you wish to sign on.   

Facial recognition, biometric identification, surveillance 

• US: Man sues Macy’s, saying false facial recognition match led to jail assault: The Washington Post carries a report of Harvey Murphy Jnr, who ‘was sexually assaulted in jail after being falsely accused of armed robbery due to a faulty facial recognition match’ in a case that ‘further highlights the dangers of the technology’s expanding use by law enforcement.’ The man was arrested despite being over 2,000 miles away from the scene of the armed robbery on the date the robbery took place. 
• International: Is Facial Recognition a Useful Public Safety Tool or Something Sinister?: ‘New York Times technology reporter Kashmir Hill’s compelling new book, Your Face Belongs to Us’ cautions how ‘facial recognition might threaten our freedom.’ The author also delves into ‘how China and Russia are already using the technology.’ For example ‘as part of its “safe city” initiative, Russian authorities in Moscow have installed over 200,000 surveillance cameras.’
• North Korea: North Korea Using AI to Boost Surveillance, Study Claims: The Asia Financial reports that a recent study has revealed that ‘North Korea is turning to artificial intelligence and machine learning to boost its surveillance network and plan for future conflicts.’
• India/ Scotland: Chinese hardware raises biometric data security concerns in India, Scotland: In this piece by Biometric Update, it highlights two reports from India and Scotland of Chinese surveillance technology, microchips and hardware being flagged for possible data breaches and posing a risk to people’s privacy. It was found that Hikvision surveillance cameras were installed at Scotland’s University of Glasgow. Biometric systems in India at 7500 sites contain Chinese ‘microchips, which are suspected to be a conduit for data leaks from government establishments, including sensitive facilities such as military and defense offices.’ 
• India: Tender for voters’ surveillance cancelled after EC intervenes: The Hindustan Times reports the Indian ‘National Informatics Centre Services Incorporated (NICS) on Friday cancelled its tender seeking proposals from companies to provide surveillance equipment – including facial recognition tools for voters and drones – to monitor voters during the upcoming general and state elections.’ This move came after the Election Commission of India sent across a communication to  NICS flagging citizen privacy concerns which might arise from its implementation.
• International: Face recognition technology follows a long analog history of surveillance and control based on identifying physical features: Sharrona Pearl, Associate Professor of Bioethics and History, Drexel University writes for The Conversation in her book ‘Do I Know You? From Face Blindness to Super Recognition.’ She explores the history of facial recognition technology in ‘not just in the history of computing but in the history of medicine, of race, of psychology and neuroscience, and in the health humanities and politics’ to say that bias remains ‘deeply embedded into the systems and their purpose, explicitly or implicitly targeting already targeted communities.’ 
• UK: DWP signs deal for AI tech to analyse citizens’ social posts in anti-fraud drive: Public technology reports that the UK ‘Department for Work and Pensions has signed a deal for automated technology that can analyse and flag up millions of social posts made by citizens.’ 

AI, algorithms and autonomy in the wider world

• India: Multiple reports of erroneous algorithmic decisions in social welfare schemes and government services in India: Two reports by Al Jazeera highlight erroneous decisions in social welfare and government services in India which relied on algorithmic decisions. One is about the delivery of subsidized and free food to economically marginalized communities in the south Indian state of Telangana, where their platform Samagra Vedika, due to ‘faulty data and bad algorithmic decisions’ in the past five years ‘cancelled more than 1.86 million existing food security cards and rejected 142,086 fresh applications without any notice.’ Similarly, in the state of Haryana, the Old Age Samman Allowance scheme erroneously declared a living person dead while relying on algorithmic decisions. 

Research and reports:

• Africa: Strengthening Data Protection In Africa: Key Issues For Implementation: A new report from AccessNow looks at four key issues for ensuring that data protection laws are ‘implemented in a way that safeguards people’s rights’, includes case studies from Kenya, Uganda, Ghana, and South Africa, and makes a number of recommendations.