Free AI models won’t fix scarce public services without public infrastructure

State and local governments are being offered a compelling promise: increasingly capable artificial intelligence at a rapidly falling price. Smaller models can now run on local servers, laptops and other devices rather than depending entirely on expensive commercial cloud services.

For agencies managing Medicaid, behavioral health, child welfare, disability services, public health and aging programs, this could be significant. Locally deployed AI might reduce administrative work, limit unnecessary transmission of sensitive information and continue operating where internet connectivity is unreliable.

But inexpensive models will not solve public-sector capacity shortages by themselves. The model is only one component of the system required to produce a useful public service. The value of locally controlled AI will depend on whether governments invest in the infrastructure around it, establish enforceable limits on its authority and ensure that any time it saves is returned to the public.

AI is Arriving During a Capacity Crisis

The World Health Organization projects a global shortage of approximately 11 million health workers by 2030. U.S. public agencies and contracted providers also face persistent shortages of social workers, behavioral-health professionals, case managers, public-health workers and direct-support professionals. 

AI cannot resolve low compensation, high turnover, difficult working conditions or inadequate training pipelines. It may, however, reduce time spent searching for information, reviewing forms, preparing summaries, checking documentation and coordinating referrals.

That creates an opportunity to return scarce staff time to residents and families. It also creates a risk that efficiency gains will be absorbed through higher caseloads, reduced staffing or new reporting demands. The important question is not simply how many hours an AI tool saves. It is what government does with those hours.

The Model is Not the Operational System

Public-sector leaders frequently evaluate AI products by comparing model size, benchmark results and demonstration quality. Those measures reveal little about how a system will behave inside a county health department or state human-services agency. An operational AI system also needs a “harness”: the controls, information sources, tools and workflows surrounding the model.

The harness determines:

• Which records the system may access
• Which sources it treats as authoritative
• What actions it may take
• When human approval is required
• How uncertainty is communicated
• When a concern must be escalated
• How actions are documented and corrected

A chatbot might draft a case summary from information entered by an employee. An AI agent operating inside a harness could retrieve authorized records, identify missing information, prepare a draft and route it to the appropriate worker. That additional capability does not come from the model alone. It comes from giving the model-controlled access to government systems and authority to perform multiple steps.

For that reason, the goal should not be maximum autonomy. It should have bounded agency: enough authority to perform a defined support function, but not enough to bypass due process, professional judgment or public accountability. 

An AI system might identify an incomplete application or flag a referral for follow-up. It should not independently deny a benefit, substantiate abuse, determine a person’s eligibility or make another consequential decision without accountable human review.

Local Deployment Creates Control — and Responsibility

Running a model on government-controlled infrastructure may reduce external data transfers and provide greater control over retention, access and system updates. It may also create more predictable costs for repetitive, high-volume work. 

Local operation is not automatically private or secure. Agencies still need encryption, access management, device protection, audit logs, software maintenance, incident response and staff training. A poorly maintained local system can be more vulnerable than a well-governed cloud service.

The correct question is not whether local systems are always safer. It is whether an agency has selected the right combination of capability, control and responsibility for a specific use. Infrastructure includes more than computing equipment. 

A recent review of medical AI in low-resource settings identified unreliable connectivity, fragmented data, limited technical capacity and weak governance as major barriers. Free model weights do not repair outdated information systems or create the workforce needed to maintain them.

This creates a risk that AI will widen existing inequalities among jurisdictions. Wealthier governments may convert inexpensive models into useful capacity, while rural counties and underfunded agencies remain unable to deploy them safely. States can reduce that gap by creating shared secure infrastructure, technical-assistance centers, model-evaluation services and purchasing collaboratives. Smaller jurisdictions should not each have to design an AI governance program from the beginning.

Procurement Should Follow the Public Outcome

Before purchasing or deploying an AI system, agencies should answer five questions:

1. What public problem is the system intended to solve?
2. What information and authority does it require?
3. Which decisions remain exclusively human?
4. Who is accountable when it is wrong?
5. How will government determine whether residents benefit?

Technical accuracy is only one measure. Agencies should also track time returned to direct service, changes in caseloads, error and correction rates, staff workload, service access and differences in outcomes across communities. 

The National Institute of Standards and Technology’s AI Risk Management Framework recommends governing, mapping, measuring and managing risk throughout an AI system’s lifecycle. Public agencies should apply that principle to the whole operating environment — not merely the underlying model.

Cheap intelligence could become valuable public infrastructure. But that will happen only if governments invest in the systems surrounding it and make an explicit decision about where the benefits go. The test is not whether an agency can operate an AI model locally. It is whether doing so gives public workers more time to serve people — and whether residents experience a public service that is more available, accountable and humane.

James Lomastro is a surveyor for the international Commission on Accreditation of Rehabilitation Facilities, an advocacy associate with Dignity Alliance Massachusetts and a member of the Massachusetts Board of Registration of Nursing Home Administrators.