Palantir's Predictive Policing Technology: A Case of Algorithmic Bias and Lack of Transparency

Multiple Sources
2017-2021

Research on Palantir's predictive policing systems has identified three principal ethical concerns: the unauthorized mass surveillance of personal data without public consent, the potential for algorithmic bias leading to discriminatory targeting of specific demographic groups, and the complete absence of transparency and accountability mechanisms.

RACIST FEEDBACK LOOPS

Critics such as the Stop LAPD Spying Coalition call Palantir's predictive policing practice a "racist feedback loop" because the data from which the reports are generated has the results of LAPD's racist policing embedded within it.

The system creates problematic outcomes where historical bias in policing data gets encoded into algorithmic predictions, which then justify further biased policing, creating a self-reinforcing cycle of discrimination.

NEW ORLEANS BIAS FINDINGS

Researchers found the predictive model in New Orleans replicated systemic bias against over-policed communities of color. Approximately 3,900 people were identified as high-risk for gun violence involvement, with the vast majority being Black residents of historically over-policed neighborhoods.

The system analyzed data from field interview cards, ballistics records, jailhouse calls, and scraped social media posts—all data sources known to reflect existing patterns of discriminatory law enforcement.

LAPD CHRONIC OFFENDER SCORES

The LAPD's "workup" involves software provided by Palantir that pulls data on criminal history and affiliations, and from license plate readers and social media networks, and uses it to create a "chronic offender score" for the individual.

Once someone is deemed a sufficient threat based off their score, officers send them letters and are encouraged to knock on their doors to let them know they're being monitored. Officers are also instructed to look out for opportunities to stop or arrest them.

THE CATCH-22 OF LIST REMOVAL

The only way for someone to get off a chronic offenders list for an area is to not have any interactions with the police—a sort of Catch-22 since the program is intended to flag individuals for increased police attention.

If someone is removed from the list, they aren't notified. This creates a system where the only path to freedom from surveillance requires avoiding the very police contact that the system is designed to generate.

DISPROPORTIONATE TARGETING

In Los Angeles, Palantir's software helped designate so-called "chronic offenders," disproportionately targeting minority neighborhoods, with critics finding that the system amplified racial bias.

Survey data reveals extreme concentration: just 2% of respondents reported being stopped 11-30+ times weekly, while 76% were never stopped. This dramatic disparity demonstrates how algorithmic systems concentrate enforcement in specific communities.

SECONDARY SURVEILLANCE NETWORKS

Sociologist Sarah Brayne identified how police expanded data collection beyond direct contacts. Associates of stopped individuals automatically entered the system, creating what she calls "secondary surveillance networks" based on proximity rather than suspected wrongdoing.

This means that simply knowing someone who has been stopped by police can bring you into the surveillance system, creating guilt by association encoded in algorithmic form.

LACK OF TRANSPARENCY

The research identifies the complete absence of transparency and accountability mechanisms as a central concern. In New Orleans, the program was maintained as a secret from the mayor and city council until investigative journalism exposed it.

The secrecy meant that attorneys representing defendants may not have been provided with evidence they had a right to see, violating due process rights.

PALANTIR'S RESPONSE TO BIAS CONCERNS

Palantir has argued that its technology provides audit trails and transparency features that enable oversight. However, critics note that these features are only effective if:
- They are actually used by oversight bodies
- The underlying data isn't biased
- The algorithmic logic is transparent
- There are consequences for discriminatory outcomes

FALSE POSITIVES WITHOUT ACCOUNTABILITY

Officers praised Palantir's data integration capabilities enthusiastically. However, the platform's effectiveness depended entirely on human interpretation, with engineers making subjective assumptions that could generate false positives without accountability mechanisms.

RESEARCH FINDINGS ON DISCRIMINATION

Academic studies document algorithmic bias risks and lack of transparency in Palantir's systems. The research shows that data-driven policing can create the appearance of objectivity while actually encoding and amplifying existing discriminatory patterns.

IMPACT ON COMMUNITIES OF COLOR

One resident in heavily-policed areas described police presence as "like asking me how many times do I see a bird in the day," illustrating the constant surveillance that algorithmic policing enables in predominantly Black and Latino neighborhoods.

The concentrated enforcement in these communities has effects beyond individual stops:
- Normalization of police presence
- Disruption of community life
- Economic impacts from arrest records
- Psychological harm from constant surveillance
- Erosion of trust between communities and police

CIVIL LIBERTIES NIGHTMARE

Behind closed doors, some LAPD personnel expressed doubts about the systems. One sergeant called predictive policing "worthless," while another termed person-based predictive policing "a civil liberties nightmare."

These internal concerns suggest that even some law enforcement personnel recognize the discriminatory potential of algorithmic policing systems.

CALLS FOR ALGORITHMIC ACCOUNTABILITY

Advocacy organizations call for remedies including:
- Independent algorithmic audits with public reporting
- Mandates on source transparency for training data
- Stronger data-quality standards to address bias
- Statutory limits on how datasets may be cross-referenced
- Community oversight of predictive policing systems
- Right to know when you're on a watch list
- Ability to challenge algorithmic designations

THE NEUTRALITY MYTH

The Stop LAPD Spying Coalition characterizes both Operation LASER and PredPol as enabling "decades of discriminatory and racist policing under the apparent neutrality of objective data."

This "neutrality myth" is central to how algorithmic bias operates—mathematical formulas and data analysis create an appearance of objectivity that masks the discriminatory patterns embedded in the underlying data and algorithmic design.

ONGOING DISCRIMINATION

While the LAPD ended contracts with PredPol and Operation LASER, other major technology companies continue expanding police surveillance capabilities nationwide, and Palantir continues to provide predictive policing tools to other jurisdictions.

The fundamental problems of algorithmic bias and lack of accountability remain unresolved as these systems proliferate.
