News Briefing 26 August – 6 September 2024

Autonomy in weapons systems, and AI in security and defence

• International: Killer Robots: New UN Report Urges Treaty by 2026: Human Rights Watch covers the United Nations Secretary General’s report which was mandated through a resolution in the United Nations General Assembly in December 2023. The report reflects concerns by many submissions on the inability of the CCW to ‘make progress and adopt new international law on this issue’ of autonomous weapon systems. 
• International: The problem of algorithmic bias in AI-based military decision support systems: In this ICRC blog post, Dr Ingvild Bode, Professor at the Center for War Studies, University of Southern Denmark, and Ishmael Bhila, PhD researcher at Paderborn University, working on the “Meaningful Human Control: Between Regulation and Reflexion” project, unpack the problem of algorithmic bias with reference to AI-based decision support systems (AI DSS). The piece says that the possibility of bias in data, in design and in use of AI DSS could  ‘risk spreading the impact of algorithmic bias through military decision-making processes related to the use-of-force.’ 
• International: Artificial intelligence in military decision-making: supporting humans, not replacing them: The article examines the use of AI in military decision-making, focusing on AI-based decision support systems (DSS) for decisions on the use of force. It highlights both the potential benefits, such as faster and more comprehensive decision-making, and the risks, including errors, biases, and the erosion of human judgement. The authors argue that AI should complement, not replace, human decision-making to ensure compliance with international humanitarian law. Key recommendations include conducting thorough legal reviews of AI systems, using them only in suitable contexts, and implementing measures to mitigate risks like automation bias. The article calls for ‘preserving human judgement’ and accountability in decisions that impact lives, advocating for ‘careful integration of AI in military operations’.
• Japan: Japan’s military to spend on AI, automation, perks to combat recruitment crisis: Japan’s defence ministry is tackling a recruitment crisis by investing in AI and automation, aiming to improve conditions for troops, and outsourcing certain tasks. The 2024 defence budget will rise by ‘6.9% to 8.5 trillion yen ($59 billion).’
• Japan/Israel: The Japanese Robot Controversy Lurking in Israel’s Military Supply Chain: Activists have accused Japanese robotics firm FANUC of indirectly supplying military equipment to Israel, claiming the company’s robots are used in Israeli defence manufacturing. FANUC denies these allegations, stating it conducts thorough checks to prevent sales for military purposes. The controversy underscores the challenges in regulating dual-use technologies, which can be employed for both civilian and military applications.
• International: Special operators hope AI can reduce civilian deaths in combat: U.S. special operations are examining the use of AI to ‘reduce’ civilian casualties in military conflicts. Assistant Secretary of Defense Christopher Maier argues that AI and machine learning have the potential to improve targeting and minimise civilian harm, especially in large-scale conflicts. While AI can enhance decision-making and data analysis, its effectiveness depends on the ‘parameters’ set by human operators and the quality of data. Concerns remain about AI’s unpredictability, bias, and potential misuse. The Defense Department is working on better civilian protection measures, including a ‘new centre of excellence’ dedicated to this purpose.
• Holy See: Holy See: Technology should better human life, not take it: The Holy See has reiterated Pope Francis’ call for a ban on lethal autonomous weapons systems (LAWS).  In a recent speech at the GGE on LAWS, Archbishop Ettore Balestrero emphasised that technological advancements should be used to enhance human life, not end it. The Holy See is advocating for a legally binding international agreement to regulate AI weaponization and a ‘moratorium on LAWS development and use’, arguing that human control is essential for ethical decision-making, as machines lack moral responsibility. The Archbishop drew attention to the use of military AI in theatres of war, stating that ‘it is profoundly distressing that, adding to the suffering caused by armed conflicts, the battlefields are also becoming testing grounds for more and more sophisticated weapons.’
• International: From landmines to killer robots — inside the campaigns to ban weapons of war: The ABC Radio National program “From landmines to killer robots” discusses the global efforts to ban autonomous weapons systems, featuring Mary Wareham and Daniela Gavshon of Human Rights Watch. They emphasise the need for international regulations on autonomous weapons systems to prevent their misuse and ensure human oversight in military decisions.
• International: The risks and inefficacies of AI systems in military targeting support: This ICRC blog post on AI based decision support systems (AI DSS) is by Jimena Sofía Viveros Álvarez, member of the United Nations Secretary General’s High-Level Advisory Body on AI. She says that despite claims that AI DSS would lead to improvement and accuracy, ‘biases, brittleness, hallucinations, misalignments, privacy risks and loss of control, which paired with the technology’s unpredictability, pose sundry problems for their use in target identification, selection and engagement.’ The piece further says that AI  assisted ‘targeting capabilities are unchartered waters and despite any alleged advantages, these technologies lack appropriate testing and review standards prior to their deployment.’ 
• US/China: China claims breakthroughs in autonomous vehicles: China continues to advance in autonomous vehicle technology, improving visual perception and object tracking. These advancements have both civilian and military implications. China is also becoming a major player in the field of Light Detection and Ranging systems market (LiDAR),  challenging U.S. dominance. In response, the U.S. is investigating the links between Chinese LiDAR manufacturers and the Chinese government, potentially restricting their access to U.S. markets.

Facial recognition, biometric identification, surveillance

• Pakistan: Pakistan: Authorities must be transparent about internet disruptions and surveillance tech: Amnesty International has called on Pakistani authorities to be transparent about recent internet disruptions and the use of surveillance technology, which have caused a ‘40% drop’ in internet speeds across the country. A petition challenging these disruptions has been filed in the Islamabad High Court, with the court seeking responses from the government and the Pakistan Telecommunications Authority. Amnesty warns that these actions undermine ‘freedom of expression and violate human rights’. Additionally, X (formerly Twitter) has been banned in Pakistan for six months.
• Netherlands: Clearview AI fined by Dutch agency for facial recognition database: This piece by Reuters reports on a fine imposed on Clearview AI by the Dutch data protection watchdog DPA. DPA Chairman Aleid Wolfsen said in a statement that ‘facial recognition is a highly intrusive technology, that you cannot simply unleash on anyone in the world.’ Clearview has also been fined by countries in the EU previously for its non-compliance with privacy and data protection regulations.  

AI, algorithms and autonomy in the wider world

• UK: Warnings AI tools used by government on UK public are ‘racist and biassed’: This piece by The Guardian reports that algorithmic tools being used by the central government in UK are to be published in a public registry, to address issues raised on bias, racism and lack of transparency. This move comes after multiple departments have been challenged for their use of algorithmic tools for immigration and fraud detection. Caroline Selman, a senior research fellow at the Public Law Project (PLP) had said that ‘it is in everyone’s interest that the technology which is adopted is lawful, fair and non-discriminatory.’ 
• US/ UK/ EU: US, Britain, EU to sign first international AI treaty: Reuters reports on developments regarding the Council of Europe Framework Convention on artificial intelligence and human rights, democracy, and the rule of law. The Council of Europe has opened the convention for signatures, and Britain’s justice minister, Shabana Mahmood, said in a statement that this move helps in ‘ensuring that these new technologies can be harnessed without eroding our oldest values, like human rights and the rule of law.’ 
• South Africa: South Africa’s artificial intelligence policy — a small step forward, but gaps remain: Emile Ormond writes for the Daily Maverick on South Africa’s National AI Policy Framework released earlier this month by the Department of Communication and Digital Technologies (DCDT). Ormond points to three shortfalls to this initiative, including a lack of clear methodology, unclear framework to incorporate a wide range of stakeholders and non alignment with previous national and international policy initiatives. 

Research and Reports

• Research: AI Governance in a Complex and Rapidly Changing Regulatory Landscape: A Global Perspective: This research paper by Esmat Zaidan and Imad Antoine Ibrahim delves into the societal risks and opportunities presented by artificial intelligence. It calls for a time bound response to the vacuum in international law to regulate AI technologies.