AI Put on Trial in ‘Life or Death’ Police Tech Clashes (1)

The latest criminal justice battle is pitting the explosion of Big Tech tools used by police against state-funded forensics offices staffed by elite public defenders.

Companies developing everything from DNA analysis to autonomous surveillance bots have found a broad customer base: roughly 18,000 police agencies eager for a crime-fighting edge. But there’s no nationally accepted standard for these tools, leaving a key question about the reliability of these technologies: Are they trustworthy enough to clear courts’ high evidence standards?

Lawyers across the country who believe their clients have been wrongly implicated by a new technology are forced to wage individual battles against companies keen to keep their intellectual property under wraps.

It’s against this backdrop that New Jersey’s Office of the Public Defender launched a forensics office this summer after dubious facial-recognition artificial intelligence put one of its clients in jail. Tamar Lerer, the state’s appellate expert and technology wonk, fought for access to the inner workings of the technology that put the client under suspicion. Because of this push the client was released with roughly time-served.

Business sends “law firms into criminal courtrooms and they’re telling us, ‘My R&D to develop this for three years is more important and precious than the liberty your client is losing,’” said Cynthia Conti-Cook.

Conti-Cook, the director of research and policy at the Surveillance Resistance Lab, is part of a nationwide network of defense lawyers, academics, technologists, and policy strategists who share data, briefs, and tactics in an effort to push back against legal tech in court. Sometimes just getting access to this data can be enough of a bargaining chip for defense lawyers to get strong plea offers from prosecutors.

“When they sent their white shoe law firms into court to say ‘trade secret,’” she said, “our attorneys weren’t ready to say, ‘No, it’s not,’ and the judges weren’t ready to say, ‘No, it’s not.”

That’s starting to change.

Inside the Black Box

New York City food truck operator Francisco Arteaga swears he’d never been to New Jersey before being charged with allegedly robbing a Garden State wire-transfer store in 2019.

AI disagreed.

New York Police Department Real Time Crime Center’s Facial Identification Section compared a surveillance footage image of the robber’s face to databases. They said Arteaga was a “possible match.”

Police included Arteaga’s image in a photo lineup. Two store workers pointed to him. He was arraigned, offered a 12-year plea, and sent to jail to await trial.

“It was confusing and weird,” he said of being implicated in the crime. “It was like, ‘to hell with it, let’s go window shopping’” to find a defendant “‘in another state.’”

Then Lerer got involved.

Tamar Lerer of the New Jersey’s Office of Public Defender and client Francisco Arteaga

Photo courtesy of Tamar Lerer

Following a multi-year fight, a state appeals court ordered prosecutors to turn over more evidence—including facial recognition system source code and error rates—a first-of-its-kind ruling.

“A lot of the answers are hidden in black box algorithms, and we can’t get a look under the hood,” without rulings like these, said Lerer, the leader of New Jersey’s forensics unit.

US Department of Homeland Security and Massachusetts Institute of Technology researchers have highlighted facial recognition reliability problems. Software creates “faceprints,” written in code, measuring minute distances between facial features. User error and aspects like lighting, facial expression, resolution and the suspect’s race can yield shoddy results.

Arteaga’s facial recognition decision followed Lerer’s seismic win in a 2021 opinion ordering disclosure of the source code for TrueAllele, which tests low levels or complex mixtures of DNA from multiple people—a type of analysis sometimes found unreliable when defense experts test companies’ source code.

“This is literally a life and death thing,” and “the reality is that defense lawyers are playing catch-up,” said Melba Pearson, Director of Prosecution Projects for Florida International University and the chair-elect of the American Bar Association’s Criminal Justice Section.

Since “the train has left the station—AI is here,” Pearson said both prosecutors and defense lawyers must work with the technology in an “equitable manner” to distinguish between reliable crime-fighting tools and junk science.

Other industries have stringent software reliability standards, said Marc Canellas, an assistant public defender in Maryland’s forensic office, who holds a doctorate degree in aerospace engineering.

“People care if something bad happens to someone driving a car,” he said. “In criminal justice software there’s no consequence to failing” in court.

Good prosecutors seek out validations for new tech, said National District Attorney Association President Summer Stephan. But law enforcement can’t ignore technological advances that help find perpetrators and exonerate the innocent.

Forensic genetic genealogy—which was used in 2018 to catch the Golden State Killer, a man who’d evaded authorities since the 1970s—is an area with immense promise, Stephan said. Her San Diego County office is at the forefront of this trend, using this technology to solve 20 murder and serial rape cases.

Joseph James DeAngelo, known as the Golden State Killer, is shown in his booking photo April 25, 2018. DeAngelo was booked on two counts of murder, but police say he may be responsible for at least 12 murders and 45 rapes in a series of attacks that began more than 40 years ago, ending abruptly in 1986.

Photographer: Sacramento County Sheriff’s Department via Getty Images

“I don’t want to deprive a victim waiting for justice, looking over their shoulder, wondering if the person who raped them is going to jump out at them from the bushes again,” said Stephan. Prosecutors must have courage “to keep moving forward and evolving” while making sure technology isn’t “trampling over the rights of someone accused of a crime.”

Crown Jewels

Pushing against the demands for information are scores of veteran lawyers tasked with protecting the intellectual property behind these technologies.

Lawyers like Foley & Lardner partner Chanley T. Howell craft agreements to protect tech companies from divulging information when their customers get dragged into court amid fierce competition among firms developing AI.

“Even with court orders we all know that leaks happen, and if our crown jewels leak we suffer extreme harm,” he said.

Litigating the sensitivity of the information a business wants protected can cost hundreds of thousands of dollars, Howell said.

Lately, judges have been bowing to the novelty and fear of AI, sometimes erring on the side of access for plaintiffs in cases where they might not grant discovery into traditional trade secrets like recipes or formulas.

“Judges are human,” he said. “So unless you get a judge that’s very AI savvy, they could be sympathetic to a plaintiff saying they need to see what’s in the black box.”

Many subpoenas are bad faith attempts to attack commercial interests, said Tom Chittum, senior vice president for analytics and forensic science with SoundThinking Inc. The company owns gunshot-tracing service ShotSpotter, which is dealing with a horde of subpoenas in gun-crime cases.

“Some defense counsel are on a quixotic adventure where they live in a lawyers’ paradise where only perfect evidence is admitted,” said Chittum. “But we don’t prosecute crimes in laboratories; we prosecute them in courtrooms.”

Conti-Cook’s national group collects contracts showing that the technology companies sometimes require prosecutor offices to oppose discovery. The contracts often include a liability shield for companies whose technology leads to wrongful convictions.

“We thought we had a tech problem,” said Conti-Cook. “We had a corporate secrecy problem.”

Making the ‘Stars Align’

Several defense lawyers told Bloomberg Law the same thing—the “stars have to be aligned” with the right facts, right lawyers, and right judge—for defendants to win a meaningful access case.

But when the opportunities come, these breakthrough opinions can help defense lawyers across the country. That’s where Lerer’s boss, New Jersey Public Defender Jennifer Sellitti, said her new unit will play a big role.

“What we’re doing now—with software affecting forensic science—is literally going to set the stage for how this evidence is used for the next decade,” Sellitti said.

Lerer plans to keep colleagues up to date on the latest technology and assist line defenders building cases with the potential to develop blockbuster decisions.

Forensics teams like Lerer’s help line defenders who often juggle 100 cases and overwhelming amounts of data, said Peter Conley, a lead assistant capital defender with the Kansas State Board of Indigents’ Defense Services. His office is also pushing to build a forensics unit.

“We’ve got cases with more than three terabytes of data,” he said. “Thousands of hours of video, dozens of copies of cellphones, jail tablets, cell tower location data, DNA, toxicology, fingerprints, ballistics—all of them rely on software and technology.”

Overlook the needle-in-the-haystack and there often aren’t do-overs.

Last month a New Jersey man lost an appeal despite the defendant finding an expert with a strong theory for why bullet-matching evidence that convicted him wasn’t reliable. The defendant’s team didn’t raise this forensics issue at trial, so the court upheld the conviction.

Despite his landmark appellate decision, Arteaga was denied access to the facial recognition discovery. After another appeal, he pleaded guilty so he could see his family after nearly four years missing his children’s birthdays behind bars.

But he takes a broader view of his role in the justice system: “Maybe this is my victory,” he said. “It got Ms. Tamar’s community one step closer.”