Photo Credit: ProPublica
Today, it seems as if Artificial Intelligence (AI) has shifted almost overnight from a futuristic tech concept exclusive to Sci-Fi into an everyday tool available to just about anyone with access to the internet. Moreover, it is everywhere — sorting our media feeds, managing our bank accounts, and even running our homes. But as AI becomes an increasingly regular part of our lives, it appears to also be seeping into a much higher-stakes arena: the criminal justice system.
Across California, courts now rely on algorithmic risk-assessment tools to predict "recidivism,” or how likely a defendant is to break the law again. While supporters and tech firms claim these tools mitigate human bias in the courtroom, local researchers contrarily warn they actually pose a major threat to constitutional rights.
Predictive software sits at the center of this debate. These systems scan massive amounts of data such as past arrest records and neighborhood demographics to calculate a "risk score." Judges then use these scores in consideration towards bail amounts, parole eligibility, and sentence lengths.
However, researchers at UC Berkeley’s Computational Research for Equity in the Legal System (CRELS) program detect a severe caveat to the AI model: the algorithms learn from biased, historical police data. They claim this risks creating a dangerous feedback loop:
Flawed Data: If a neighborhood has been heavily policed in the past, the AI mistakes high arrest numbers for a sign of naturally higher crime rates.
The Feedback Loop: The algorithm then sends officers right back to those same streets or slaps residents with higher risk scores, creating a self-fulfilling prophecy.
Preemptive Punishment: Legal scholars argue this shifts justice away from actual actions, punishing people instead for what a computer thinks they might do.
Photo Credit: Medium
The biggest threat to civil liberties that this software poses is that it is completely hidden from the public; private tech firms construct these algorithms and keep them confidential to remain competitive on the market. Because defense attorneys are legally blocked from accessing the AI’s code, defendants face sentencing based on mathematical formulas they cannot see, understand, or challenge. Legal experts argue this clearly violates a citizen's basic right to a fair trial.
To address this, California data scientists are building public auditing tools. These programs independently test judicial AI systems to catch unfair results or verdicts. They are already exposing instances where the software assigns higher risk scores to minority defendants than to white defendants with identical criminal profiles.
With academic research exposing these flaws, California is now a major battleground for AI regulation. Legal experts are pushing for strict state laws to force these algorithmic tools into the open during pretrial detentions and sentencing. Their proposed rules state that AI should only serve as a “non-binding advisor,” a legal term for advice, an opinion, or a recommendation that holds no legal enforcement power, to ensure that a human judge always makes the final call.
By bringing these hidden formulas into the light, California researchers hope to prove that public safety does not have to come at the expense of justice.
To add to or correct any information in this report, please contact me at alizeh.i@lead4earth.org.
0
0
Comments