NY Court System Issues Rules for AI Use by Judges, Staff

The New York State court system on Friday laid out a set of rules and restrictions related to the use of AI by judges and non-judicial employees.

The interim policy, developed by the Unified Court System’s Advisory Committee on AI and the Courts, intended to establish guardrails to “ensure fairness, accountability, and security in the use of AI, particularly generative AI, by our workforce,” the document says. The rules apply for all court system-owned devices and for court-related work being performed on any other device.

Generative AI can be useful for drafting documents and summarizing long documents and datasets. But it can also produce “inaccurate, wholly fabricated, or biased outputs, and can jeopardize the security of data entered into the program,” the document says.

AI “is not designed to replace human judgment, discretion, or decision-making,” Chief Administrative Judge Joseph Zayas warned.

The court system has paid for and manages several enterprise AI products from Microsoft, GitHub, and Trados Studio. Employees can also use the free version of OpenAI’s ChatGPT, but are prohibited from using the subscription.

Approval of a generative AI product signals it’s safe to use, but that “does not necessarily mean that, for a particular task, the use of that product is suitable or appropriate,” the document warns. Judges and supervisors are allowed to prohibit their staff from using certain products.

All judges and non-judicial employees have to go through an initial training course and commit to ongoing training. Employees are prohibited from writing prompts or uploading documents that contain confidential or private information to any generative AI program not managed by the court system, even if the document is available to the public.

The court system also calls for users to review any content produced by an AI program to make sure they don’t reflect any unfair bias, stereotypes, or prejudice.

Courts across the country are grappling with the use of AI by judges, court staff, and lawyers. Earlier this week, Senate Judiciary Committee Chairman Chuck Grassley (R-Iowa) sent letters to two federal judges asking they or their staff used generative AI tools to produce rulings, after the judges issued orders that contained made-up citations.