Concerns Arise Over Accuracy of New York City's AI-Powered Shot Detection System

The New York City Comptroller has issued a pivotal report urging the New York Police Department (NYPD) to reconsider its ongoing $22 million contract for an AI-powered shot detection system. This recommendation comes after an audit by Comptroller Brad Lander’s office revealed significant performance issues with the technology provided by ShotSpotter, a product of security technology company SoundThinking.

The audit discovered that the ShotSpotter system only confirmed actual shootings a mere 13% of the time. Over sample months in 2022 and 2023, the system’s alerts translated to confirmed shootings between 8% and 20% of the time, raising serious concerns about its reliability. For instance, in June 2023, out of 940 ShotSpotter alerts responded to by the NYPD, only 122 were confirmed as actual shootings, meaning 771 alerts—approximately 82%—could not be substantiated upon police arrival, and 47 alerts were classified as unfounded.

Moreover, the report indicated a significant time investment by NYPD officers, who collectively spent 426.9 hours investigating alerts that turned out to be false. This not only impacts the efficiency of police resources but also raises questions about the value of the investment in such technology.

In response to the audit’s findings, the NYPD expressed concerns regarding public safety, suggesting that non-renewal of the ShotSpotter contract could compromise the ability of the department to respond effectively to gunfire incidents. They deemed the suggestion for further analysis a potentially premature measure.

SoundThinking, the parent company of ShotSpotter, reacted strongly to the audit. They argued that the report’s conclusion incorrectly assumes that a lack of immediate confirmation of gunfire by responding officers indicates that an alert was triggered by non-gunfire sounds. According to the company, this assumption misrepresents how the ShotSpotter technology and NYPD protocols function. They emphasized that ShotSpotter acts as a "digital witness," providing a more thorough assessment of gunfire incidents than officers may achieve during outdoor patrols.

The ShotSpotter system relies on a network of over 2,000 acoustic sensors installed on street infrastructure, such as lampposts, to detect the sound of gunfire. Sophisticated AI algorithms analyze the audio signals to pinpoint the gunfire's location through triangulation. Once an alert is recorded, the incident is forwarded to the ShotSpotter incident review center, where SoundThinking staff evaluate the sound. Confirmed incidents are communicated to the NYPD, which can then monitor the situation via CCTV and dispatch officers as needed.

Since its implementation in 2015, ShotSpotter has been a subject of scrutiny. An Associated Press investigation in 2022 highlighted critical flaws in utilizing ShotSpotter data as evidentiary support in legal proceedings. The proprietary nature of the algorithms has raised concerns, making it challenging for the public, jurors, and legal entities to scrutinize the technology fully.

Overall, this report sheds light on the pressing issues surrounding law enforcement technology, particularly the efficacy of AI systems in public safety and community trust. As stakeholders continue to debate the implications of such systems, the findings serve as a vital discussion point for their future application and development.