News Briefing 1-12 July 2024

Autonomy in weapons systems, and AI in security and defence

• Ukraine: A.I. Begins Ushering In an Age of Killer Robots: The New York Times carries a lengthy story on increasing autonomy in weapons systems in Ukraine,  where, according to the piece, ‘the pressure to outthink the enemy, along with huge flows of investment, donations and government contracts, has turned Ukraine into a Silicon Valley for autonomous drones and other weaponry.’ The piece details numerous systems which are in use and development in Ukraine, and highlights that most of the weapons being developed in Ukraine ‘aren’t as advanced as expensive military-grade systems made by the United States, China and Russia’. However, ‘their low cost — just thousands of dollars or less — and ready availability’, and the fact that many of these systems are ‘built with code found online and components such as hobbyist computers’, makes their development significant, with some U.S. officials ‘worried that the abilities could soon be used to carry out terrorist attacks.’ The report underscores that ‘the systems raise the stakes in an international debate about the ethical and legal ramifications of A.I. on the battlefield. Human rights groups and United Nations officials want to limit the use of autonomous weapons for fear that they may trigger a new global arms race that could spiral out of control’, and also highlights that ‘Ukrainian soldiers have raised concerns about the potential for malfunctioning autonomous drones to hit their own forces.’ The NYT further notes that ‘Groups including the International Committee of the Red Cross have pushed for legally binding rules that prohibit certain types of autonomous weapons, restrict the use of others and require a level of human control over decisions to use force’.
• International: Defense Startup Helsing Raises at €5 Billion Valuation to Expand Along NATO’s Eastern Flank: Numerous outlets report that defence AI startup Helsing AI has ‘raised €450 million ($487 million) in venture capital funding that it plans to use to expand its presence in European nations bordering Russia’. The company has tripled its ‘price tag in less than a year as global conflicts stoke a wave of private investment in military suppliers.’ This new round of investment makes Helsing ‘one of Europe’s most valuable artificial intelligence start-ups, in a strong signal of traditional venture capital investors’ growing appetite for defence tech companies’, as reported by the Financial Times prior to confirmation of the investment. Helsing ‘uses AI to process vast amounts of data generated by sensors and weapons systems to offer real-time battlefield intelligence and help militaries in their decision-making’, and ‘the company’s software is also being used to develop AI capabilities for drones in Ukraine.’ Previous news briefing coverage of Helsing AI can be found here and here.
• Japan: Military Use of AI: Japan Should Take Initiative in Making International Rules: An editorial in The Yomiuri Shimbun argues that ‘As the only country struck by atomic bombs, Japan should stress the importance of regulating inhumane weapons and take the lead in creating international rules.’
• International: We Cannot Cede Control of Weapons to Artificial Intelligence: An op-ed in Scientific American by the reporter Kate Graham-Shaw notes that ‘how we use autonomous weapons systems, the ethics behind these decisions, and whether we regulate using them are already hotly debated geopolitical topics’ and that ‘the number of meetings focusing on them, such as the conference in Vienna, has increased exponentially over the past few years’, but that despite the ‘valiant efforts’ of civil society, including Stop Killer Robots, as well as the ICRC and the UN Secretary-General, progress on international rules on autonomous weapons ‘remains decidedly lethargic’, partly because ‘nations that should be pulling their weight are not.’

Facial recognition, biometric identification, surveillance 

• Serbia: Serbia’s surveillance systems are rising amid a democratic deficit: report: Biometric Update reports that Serbian civil society groups are ‘warning that digital surveillance is growing – and the European Union’s AI Act may not be enough to prevent it’.
• Palestine/ Israel: Dutch Investor pushes for human rights safeguards to stop use of surveillance technology against Palestinians: ASN Impact Investors, a ‘major investor in the Dutch manufacturer of cameras deployed in occupied East Jerusalem’, has warned TKH Group, the camera manufacturers, that it ‘must adopt human rights due diligence policies within a year or face a termination of investment.’ Matt Mahmoudi, Amnesty International’s Interim Head of the Silicon Valley Initiative, and Researcher on AI and Human Rights, notes that ‘Investors provide the enabling conditions and resources for entities like TKH Security, a part of TKH Group, to develop and sell surveillance technologies. Their vigilance is important especially where the risk of human rights harms is heightened’, and that ‘This is particularly urgent in the wake of Israel’s escalating surveillance-enabled crackdown on freedom of movement and freedom of association and peaceful assembly. Supplying hardware or software that can be used to reinforce apartheid, which is a crime against humanity, and other human rights violations by Israel against Palestinians must not be tolerated under any circumstance.’

AI, algorithms and autonomy in the wider world

• India: Need to eliminate biases in algorithms as AI on the rise: RBI Governor Shaktikanta Das: The Governer of the Reserve Bank of India has ‘emphasised on the need to eliminate biases in algorithms as the use of artificial intelligence (AI) and machine learning (ML) is on the rise.’

Research and reports

• Global: Briefing: Gender and Human Rights in the Digital Age: This new briefing from Amnesty International details how ‘Marginalized groups, including women and LGBTI people, face threats to their human rights due to extensive and improper data collection practices that do not reflect their individual realities. Governments justify such data grab tactics as a cost-saving solution to roll-out automated systems in the public sector for benefit payments, while Big Tech companies hoard and deploy personal user data for their lucrative surveillance-based business models. This under-regulated accumulation and processing of vast amounts of data not only constitutes harmful mass surveillance but also entrenches discrimination against women and LGBTI people.’
• International: Beyond surveillance: privacy, ethics, and regulations in face recognition technology: This research paper ‘critically examines the current rules and policies governing FRT, highlighting the tensions between state and corporate interests on one hand, and individual rights and ethical considerations on the other.’ It highlights the ‘importance of placing human rights and ethics at the center of regulating this transformative technology.’