New York counties to roll out AI assessment tool
Jason marz via Getty Images
By
Chris Teale
The new dashboard will help county tech and procurement leaders analyze and evaluate various products available on the market as they try to adopt them responsibly.
Counties in New York will soon have access to a new tool designed to help them evaluate and provide feedback on various artificial intelligence platforms in a bid to boost responsible adoption.
Officials said last week that the dashboard will offer a way for counties to evaluate the tools they use — including generative AI tools like ChatGPT, Google Gemini, Microsoft CoPilot and Claude — in a standardized 10-point scoring framework. Counties can see each other’s evaluations and procurement documents, with support from the New York State Association of Counties and Computer Aid, Inc.
It’s part of NYSAC and CAI’s efforts to develop a GovAI Trustmark program in a bid to help counties better understand AI tools and evaluate their cost, security, data protection, ownership, ease of use and risk factors. The dashboard supports that goal and will roll out publicly soon.
“There was no reason for each and every one of us to start assessing the choice [alone],” Marguerite Beirne, chief information officer for Westchester County, New York, said during the Technology Innovation Forum at the National Association of Counties’ legislative conference in Washington, D.C. last week. “It was, in my opinion, a waste of time and effort, and we all pretty much have the same goals. We’ve got the same standards that we must adhere to, so we talked about, let's make a shopping site. Let's vet them. Let's have different municipalities join in, put their feedback in the tool and rate it, and let us know their experience.”
This effort to help officials better procure AI kicked off with a survey of all 57 counties and five New York City boroughs. Those survey results found that 63% of counties are using AI in some form, with the vast majority reporting that the technology had already resulted in significant time savings for their employees. Almost one quarter — 23% — said AI was saving employees at least three hours a week.
Those surveyed reported that generative AI had helped in areas like research, drafting emails and generating ideas, as well helping with reports, presentations and policy development. But 57% said in the survey that they lack guardrails for AI use, something that has been seen elsewhere as governments wrestle with the technology’s benefits but also want to regulate how employees use it, given its inherent risks.
The survey found that the biggest barriers to adoption were worries about accuracy and confidentiality, while county leaders also expressed concerns that their employees do not have enough time to experiment with AI and be properly trained. Beirne said “getting the message to stick” on AI training and its usefulness will be a key challenge for county leaders to overcome.
And the dashboard can help procurement officials work out what is best for them, regardless of their county’s size. Meghan Cook, director of the Cyber Incident Response Team and assistant director of the Office of Counter Terrorism in the NYS Division of Homeland Security and Emergency Services, said the dashboard reflects how county IT is a “very close-knit community.”
“Everyone is a known entity who will be adding information here, and many of these people have known each other for 20 years,” Cook said. “There's building the platform, but there's also, what is your confidence level in what is being reported in there? I think this works because this community has spent so much time together that they respect each other and believe that we call each other anyway.”
In addition, Mark Lavigne, deputy director at the New York State Association of Counties, said it can help avoid any “pressure from department heads or legislators or supervisors to get this or that AI tool,” especially after they go to conferences and see those tools being demonstrated. Seeing whether they are actually being used by their peers will be vital for county leaders, he said, as well as learning if they work as intended or cause more issues than they are worth.
“Everybody can now have a place to go to look to see if that tool is being used by another county in New York State, and they're going to know who put the evaluation into that dashboard,” Lavigne said.