Breaking down silos: A practical guide for cross-agency data sharing

Government leaders are stewards of two things citizens entrust to them: public dollars and the data collected while delivering services. That stewardship carries an obligation to use both as effectively as possible. Data-sharing serves all three goals at once: it improves program outcomes, runs programs more efficiently and makes noncompliance and fraud easier to detect.

The greatest barrier to this kind of data sharing probably isn’t what you think. There are virtually never insurmountable technical blockers. Most often, it’s the pervasive misunderstanding of state and federal regulations that govern data use. Agencies tend to interpret the Health Insurance Portability and Accountability Act, the Family Educational Rights and Privacy Act, and similar regulations as prohibitive barriers. 

In reality, these are privacy-protective frameworks designed to enable sharing, not discourage it. From the Department of Health and Human Services’ own summary of the law: “the Privacy Rule's goal is to protect individuals' health information while allowing the flow of information needed to deliver and improve care.”

The regulations provide clear guidance on how to share responsibly, yet they are routinely used to justify inaction. General counsels and compliance officers default to "no" when they interpret their primary mandate as keeping agencies out of trouble. Overcoming that requires sponsorship from executive leadership who can reframe the directive: responsible data stewardship means actually using the data to improve, not just protecting it.

Delivering Value Incrementally to Build Momentum

 Agencies that share data successfully tend to start small. A single, well-scoped use case with a clearly defined outcome accomplishes two things that matter more than the immediate result.

The first is realigning risk perception. Resistance to data sharing is often rooted in an assumption that the risk is greater than it actually is. When one agency moves forward and other agencies see that it’s working (especially agencies handling more sensitive data than their own) the perceived risk recalibrates against reality. The conversation shifts from whether sharing is possible to how to do it well.

The second is reorienting leadership around mission. Agency leaders are pulled in multiple directions by day-to-day demands, resulting in thoughtful, capable leaders going months without examining whether their agency is making meaningful progress against the outcomes it exists to deliver. Has caseload trended in the right direction? Are recruitment and retention improving for the workforce? Are the populations the agency serves measurably better off? Questions like these quickly get crowded out by to-do lists.

A data-sharing initiative tied to a specific outcome forces those questions back to the surface. It requires agency leadership to articulate what success looks like in terms citizens would recognize and then to measure against it. That reorientation is often just as valuable as the analytical insights themselves.

This is why scoping matters more than scale. Each phase should deliver something concrete enough to justify continued investment on its own merits. It’s reasonable to place a measured bet on an application whose value is hypothetical but quantifiable, but pursuing work with no articulated value is not.

As early use cases prove, the technical foundations, governance practices and political support compound. Subsequent applications become cheaper to implement and easier to defend, and agencies build the muscle to ask better questions of their data over time.

When Legacy Systems Meet Modern Needs

The source systems generating most government data were never designed for decision support or cross-agency analysis. Over decades, systems, databases, programs and policies have changed, creating data discontinuities and quality issues that compound on each other. Standardization across agencies is necessary, but the challenge runs deeper than simple format differences.

While no agency has the luxury of retroactively fixing all historical data quality problems, modern machine learning and AI techniques can dramatically improve the value and usability of imperfect data. However, this shouldn't excuse continuing to design systems that ignore secondary use value. For example, the primary use of an unemployment insurance system may be processing benefit applications and paying benefits.  The same data that the system collects could secondarily help identify fraud patterns, optimize skill-up training for jobseekers, or more holistically quantify the long-term impacts of secondary education programs.

New implementations should embrace this potential from the start: clear documentation of what each system collects and how each field is defined, common standards for connecting information across systems and explicit ownership for data stewardship. This includes review processes, incident response protocols and escalation pathways for data quality and compliance issues.

Managing AI Risks and Opportunities

The same principles that have driven successful technology implementations across state governments — focus on outcomes rather than technology; target low-risk use cases first; keep humans in the loop where automation alone would be untenable; fail fast and fail small; and deliver iteratively — apply directly to AI.

The strongest AI use case candidates share a few characteristics:

1. The desired outcome is defined up front, in terms a citizen would recognize, and progress against it can be measured.
2. The work is a genuine fit for AI — finding patterns across volumes of data no human team could reasonably process — rather than a place where working human judgment is being swapped out for speed.
3. An incorrect output won’t cause irreversible damage.

A career recommendation that turns out to be a poor fit for a jobseeker is recoverable; an automated determination of eligibility for benefits, child custody, or criminal justice is not. Risk tolerance should be calibrated accordingly, and agencies should weigh the counterfactual: if the alternative is a manual process with its own well-documented inconsistencies, the question is whether AI produces better outcomes overall, not perfect ones. Contained downside use cases are reasonable places to experiment and iterate. Severe, irreversible downside use cases warrant a much higher bar regardless of the upside.

Scale also creates new exposure on privacy and fairness. AI applied to longitudinal data needs algorithmic auditing so models don't amplify bias in program delivery, including ongoing performance monitoring across demographic groups and regular review of whether AI-driven insights produce equitable outcomes. Agencies should hold AI recommendations to an explainability standard and ensure a human decision-maker validates each one before it affects a citizen.

Privacy Protection and Advanced Security

Privacy protection in longitudinal data systems requires sophisticated approaches. Effective strategies must account for the increased re-identification risks that emerge when multiple data sources are combined over time.

The goal is to create systems that enable meaningful analysis while maintaining individual privacy protections that meet or exceed regulatory requirements. Modern privacy-preserving technologies offer new possibilities, enabling analysis without compromising individual privacy. This includes federated learning approaches that allow agencies to benefit from shared insights without actually sharing raw data.

For example, three different health departments could each train a model to predict disease outbreaks using their local data. They then share only the pattern-recognition insights with each other, creating a combined model that's better at predictions than any single agency could develop alone, all without any agency ever seeing another agency's actual patient records.

Moving Forward

None of this is easy, but agencies across the country have shown it's achievable. The obligation is clear: citizens have entrusted their government with both public dollars and the data collected in delivering services. Using that data to improve outcomes, run programs more efficiently and protect against fraud is part of the job, not a separate initiative.

John Roach is the president of Resultant, a leading consulting firm shaping how local, state and federal government agencies use data.