AI in criminal justice should start with governance and low-risk use cases, report says

The use of artificial intelligence in criminal justice systems could be surpassing governments’ pace of establishing adequate policies and guardrails to ensure the tech is implemented accurately and responsibly, a recent report says. 

As the gap grows between the deployment and regulation of AI solutions in state and local courts, police departments, correctional facilities and other services, there is a critical need for policymakers and officials to better reign in applications that could significantly impact residents’ lives and liberties, according to a report released last month from the Council on Criminal Justice and RAND. 

“The main thing is that governance is required immediately … but yet people are adopting these technologies faster than the conversations have developed,” said Kristin Warren, an engineer at RAND who researches national security and equitable policies. In doing so, she said, policymakers must consider, “What are the real human costs here?” 

Such costs have become reality for people like Angela Lipps, now 50, who was jailed last year for a crime in a state she’d never visited. Lipps, of Elizabethton, Tennessee, was arrested last July after a facial recognition system incorrectly identified her as the suspect behind bank fraud in Fargo, North Dakota. 

Lipps was arrested by the United States Marshals at her home and served jail time in Tennessee and North Dakota until December, losing her rental home, Social Security income, health insurance and coverage, her pet dog and other personal affects, according to Lipps’s GoFundMe webpage. The charges against her were eventually dropped when evidence showed she was in Tennessee at the time of the crime. 

Lipps is pursuing civil litigation for her wrongful arrest, according to A&E. Reports of other issues that disrupt criminal justice proceedings due to AI have emerged in recent years, including legal citations increasingly containing hallucinations or other errors. 

For policymakers and decision-makers, “the primary opportunity here lies in constructing accountability infrastructure. This could include developing audit frameworks, establishing transparency requirements and creating mechanisms for challenge and redress,” the report states. 

Researchers also found that, based on academic literature, policy documents, practitioner reports and other sources, AI adoption in the criminal justice setting has largely concentrated around more high-stakes use cases, such as pretrial, sentencing and law enforcement decisions. Those findings present “an under-explored opportunity of repurposing AI capabilities to strengthen oversight and accountability, rather than expand decision-making systems,” the report reads. 

Indeed, one of several recommendations from the report for criminal justice leaders is to develop and establish specific rules and standards for using AI in criminal justice proceedings, Warren said.  

“Policymakers should establish explicit governance protections prohibiting algorithmic determinations of guilt, sentencing, prosecutorial charging and other liberty deprivations,” the report states. “Courts should be required to disclose in judicial findings when AI recommendations inform outcomes, and defense counsel should have access to information about all AI tools used in their clients’ cases.”

For example, California lawmakers this year are considering a bill that, if passed, would establish guardrails that aim to protect confidential information from being used in AI systems, make cite-checking a nondelegable task and prevent arbitrators from delegating decision-related tasks to AI.  

The Indiana Supreme Court AI Governance Committee developed guidance for courts to navigate the adoption and implementation of the technology late last year. The guidance suggests that judges in the state consider how to transparently use AI tools in court processes, how to ensure any AI tool has a human verification step and how to hold vendors accountable to their AI services and solutions. 

At the local level, the Fourth Judicial District of Montana, for example, updated its rules of practice late last year to require that the use of AI in court filings be disclosed. The new rules stipulate that users must share details like “the specific tool the party used; how the party used the tool in preparing the relevant document; and that the party certifies they have checked the accuracy of any portion of the document drafted or assisted by the tool, including all factual and procedural background, citations, and legal authority.” 

“I'm not saying that innovation should slow down or people shouldn't use these AI tools, but you have to match the speed of safeguards to the stakes of the decision being made and how much risk is involved, particularly as it pertains to someone’s freedom,” Warren said. 

Indeed, such concerns are at the center of an AI tool being tested by two California county courts. The tool aims to assist court staff with drafting orders and producing research memos, but many observers have raised flags over plans to expand its use from civil to criminal cases due to the risk of AI discrimination or bias impacting court proceedings. 

Ultimately, when adopting and implementing AI tools in criminal justice systems, state and local leaders should consider how “the criminal justice system [can] actually serve the community that they’re supposed to empower, protect and serve,” Warren said.