AI-based ‘listening’ helps VA monitor vets’ mental health
Connecting state and local government leaders
To better monitor veterans’ mental health, especially during the pandemic, the Department of Veterans Affairs is relying on data and artificial intelligence-based analytics.
To better monitor veterans’ mental health, especially during the pandemic, the Department of Veterans Affairs is relying on data and artificial intelligence-based analytics.
VA is working with Medallia, a customer experience management company, on its Veterans Signals (VSignals) program. Launched at the end of last year, VSignals collects feedback from veterans, eligible dependents, caregivers and survivors receiving services, analyzes it for key words or a vocal tone that indicates mental health risk and routes alerts in real time to the appropriate responders.
Medallia worked with VA’s mental health services and the Veterans Crisis Line to establish a library of key terms and sentiments that would get flagged. For instance, “if a veteran is saying they have financial hardship or if a veteran is saying, ‘I can’t pay my rent,’ ‘I’m late on my bills’ or ‘I’m failing in my school’… it will automatically flag that issue,” said Lee Becker, solutions principal at Medallia. Additionally, the AI can identify any type of negativity in a response, even if it’s not in the existing set of crisis-related topics, he said: “AI is able to bubble up issues that the VA may have not even thought of. That was very important for COVID.”
During the pandemic, the department is using the technology to identify COVID hotspots and concerns that veterans and families might have. VA officials have increased telehealth availability and doubled-down on mental health access, Becker said.
The technology uses what he calls listening posts to collect data whenever a veteran is providing feedback around quality and safety of care. Feedback might be in written form, such as a survey, or spoken, such as a voice call. The feedback data could be structured -- such as from VSignals surveys sent after a Veterans Health Administration appointment at a medical center or clinic -- or unstructured, such as spontaneous feedback through email, SMS, phone calls or kiosks.
“There are many different ways to pull the signal,” said Becker, former chief of staff for VA’s Veterans Experience Office. “We don’t really care how it’s being provided to us, we just need to make this [service] available. ”
Medallia uses an Athena AI engine for machine learning and natural language processing to analyze text, voice – by translating it to text – and journeys, which track veterans’ progress from the time they make an appointment until service is completed.
The technology is trained to watch for and flag trigger words or angry, upset tones of voice. If it flags something indicating that a veteran may be in danger of inflicting self-harm, someone monitoring the crisis line gets the alert and immediately reaches out.
The veterans’ data is being shared essentially to the national crisis line according to permission levels set by VA. “They have the phone number, the email, all identified seamlessly,” Becker said. The system is integrated with VA’s connected data warehouse, which stores veterans’ records – a big initiative for the department, which had hundreds of databases and no common customer dataset before. It also monitors for macro-level problems across the feedback, such as complaints about a particular service at one facility. Alerts to thematic problems automatically go to agency leaders.
The technology is certified at the moderate level under the Federal Risk and Authorization Management Program, Becker said, and complies with the Health Insurance Portability and Accountability Act. As of November 2019, about 2,800 crisis alerts have gone through VA offices and more than 1,400 veterans have gotten early intervention, and as of June 2019, more than 691 suicide crises and 343 homelessness crises have been sent to VA experts for help. Those numbers are higher now, but those are the most recent publicly available statistics, Becker said, adding that 70% of the feedback coming through is complimentary.
Overall, VSignals’ goals are twofold, he said: It enables VA employees to deliver the experience they want veterans and caregivers to get, and it helps veterans by ensuring that their needs are met.
“By applying this capability, VA has proven that it improves the experience, it improves trust,” Becker said. “The reason why this is successful at the VA is because they integrated experience into every single part of the organization…. This is part of the fabric of the business,” he said. “Just like we are measuring operational measures and financial measures, we are taking into consideration the customer, the citizen, the veteran and employee.”
Medallia is in talks with the Departments of Agriculture and Health and Human Services about implementing similar systems there, Becker added.
NEXT STORY: Improving procurement through data and analytics