Police mass AI surveillance pilot draws rebuke from privacy experts

A controversial police trial using live facial recognition technology has sparked concerns over privacy and accuracy, according to reports from Australian media. The Western Australian Police launched a pilot program in June, making it the first law enforcement agency in the nation to deploy live AI facial recognition on the public. The system, developed by Japanese technology firm NEC, uses the Neoface m40 product to scan faces in real time and compare them against a watchlist containing approximately 4,000 individuals, including those accused of serious crimes, missing persons, and individuals deemed potentially dangerous. During the initial phase of the trial, which took place in Perth and Fremantle, the system processed over 130,000 faces within the first week. Of these, 33 alerts were generated, leading to 18 arrests. According to police statements, images of individuals not on the watchlist were automatically deleted and rendered pixelated to protect privacy. However, the system flagged one person incorrectly as being on the list, a performance slightly better than the 2023 UK trial of the same technology. Despite these claims, critics argue that the technology still carries significant risks. Melbourne Law School associate professor Jake Goldenfein highlighted that facial recognition systems often perform poorly for certain demographic groups. He noted that historical data shows these systems tend to misidentify people of color and women, particularly women of color, due to training datasets that largely consist of white and male faces. This lack of diversity in training data leads to higher error rates among underrepresented groups. Goldenfein further explained that such inaccuracies can manifest as either false positives, where individuals are wrongly identified as being on the watchlist, or false negatives, where the system fails to recognize someone who should be identified. These errors can have serious consequences, including wrongful arrests or missed opportunities to locate missing persons. While the Western Australian Police cited a 2023 study by the UK's National Physical Laboratory, which suggested no significant difference in false positives based on gender or ethnicity under controlled conditions, the study also revealed that accuracy declines when real-world factors such as lighting, skin tone, and age affect image quality. This suggests that the system's reliability may vary significantly in practical applications. The trial has drawn criticism from privacy advocates and legal experts, who question the transparency of the watchlist and the selection of trial locations. Some have raised concerns that the areas targeted might disproportionately impact First Nations communities. Additionally, the police have not released detailed statistics beyond the initial week of the trial, stating that more information will be made available in a "coordinated manner." As the pilot continues across multiple sites, ongoing scrutiny will focus on how the technology performs in diverse environments and whether the potential for bias persists. With the broader implications of AI surveillance growing, the debate over its ethical use and effectiveness is likely to intensify in the coming months.