Biased Police Data Feeds AI's Patrol Feedback Loop — SkimNews
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Sarah Brayne found in an American Sociological Review study of the Los Angeles Police Department that the force increasingly used data to predict crime rather than only respond; skewed inputs, including greater visibility of crimes in public places and disproportionate focus on Black neighborhoods, made predictions appear impartial while feeding future police contact.
- New York University researchers found that police data from periods of unlawful or biased policing was available to train systems in 9 of the 13 US areas studied; they said company fixes were insufficient unless someone independently checked how the data was produced.
- Dr Carolyn McKay called for independent checks of predictive maps before use and stronger human oversight of state policing, arguing that systems trained on biased records cannot distinguish flawed patterns and turn them into self-fulfilling predictions.
- Dr Dimitrios Kollias said modern AI can generate outputs not explicitly present in its training data, so more capable autonomous systems require safe, reliable behavior and meaningful human control; he also said testing, improved data and objectives, and safeguards can reduce unfair outcomes.
- A 2019 Science study found that a US healthcare AI prioritizing patients by predicted care costs favored white patients, but changing its objective to predicted illness raised the Black share selected for extra support from about 18% to 47%.
- Dr Carolyn McKay proposed adding contextual data on broken streetlights, little funding and missing services so councils might direct money toward neglected neighborhoods instead of adding police patrols, while warning that place-based systems can still over-policice marginalized residents.
Why it matters: Neighborhoods already over-policed bear the cost when crime records are turned back into patrol priorities, while vendors’ technical fixes do not resolve how those records were produced. In 9 of the 13 US areas studied, training data included unlawful or biased policing, so independent checks of the data and resulting outputs directly address that harm.
Ask SkimNews
