“As AR and VR become more accessible and easier to use, they offer a lot of great possibilities for government to explore innovative approaches,” said Ellysse Dick, a policy analyst at the Information Technology and Innovation Foundation.
From field operations to personnel training to service delivery, “there are a lot of opportunities to improve government through these immersive experiences,” she said. While state and local governments are still in the early stages of AR and VR adoption, a number of emerging use cases suggest the technology’s potential power.
In Austin, Texas, emergency personnel are using VR to train for work on a bus-sized ambulance known as the Ambus. It’s hard to train on the actual vehicle: There are only 13 in the state and they’re not readily available for exercises.
“They get about two hours or so training on the apparatus, and then we sometimes go two or three years without a disaster,” said Keith Noble, commander of homeland security and emergency management with Austin-Travis County EMS. “Now they’ve had no training for two years and all of sudden they have to respond to a hurricane.”
Trainees put on an Oculus Quest headset and can interact with a 3D-rendered prototype of the Ambus systems and equipment in order to gain familiarity with the layout and the operations of the vehicle.
“It’s hard to get 30 people all together at one time for training, and then you have to pay them overtime. With VR, they can train anytime, anywhere, at their convenience,” Noble said. VR also offers the possibility of just-in-time training. “Say we get deployed for a hurricane: It usually takes 24 hours or so to get everything together and head down to wherever we’re going. With VR, they could be putting on a headset during that time and refreshing themselves on the apparatus.”
With sponsorship from technology accelerator US Ignite and Facebook, Philadelphia is in the midst of a smart cities hackathon challenge focused on VR in support of transportation accessibility.
“We have a very diverse population here in Philadelphia, and we are actively working on trying to get more people onto public transit,” said Smart City Director Emily Yates. “The hackathon will help us to identify innovative ways to utilize augmented reality to improve accessibility to transportation for individuals with disabilities.”
People with visual impairments, for instance, might use an AR phone app to see enhanced or enlarged signage laid over an actual image of a bus depot or train station. For the physically disabled, AR could help with wayfinding, steering them toward more accessible pathways in the public transport system.
“Perhaps it’s guiding you more easily to the elevator, identifying ways that don’t require you to take the steps,” Yates said. “Or maybe it could provide you an alternative route if there’s a train station that doesn’t have accessible means available.”
While the outcomes of the challenge have yet to be seen, Yates said she is optimistic that these new tools could deliver dramatic improvements. “There’s just so much opportunity for innovation in this space,” she said.
In Montgomery County, Ohio, the child services office is using VR to train caseworkers. The aim is twofold: to help them better understand how to respond to situations, and also to address potential racial inequities.
At Accenture, Child Welfare Lead Molly Tierney has been heading up an effort to develop AR and VR experiences in support of social services. She explained there’s a natural fit here. “Virtual reality experiences are very immersive and highly realistic, and can replicate what caseworkers see in the field,” she said.
Her team has developed a virtual scenario in which a caseworker interacts with a family in its home, learning how to read the cues in human behavior and how to de-escalate tense situations. “You go to ‘visit’ a home and you sit across the table from a person who’s looking you in the eye,” she said. “You speak to them and based on what you say, that changes how they respond to you.”
A version of this helps caseworkers to recognize their own hidden biases by swapping out actors of different ethnicities and inviting participants to review how their handling of the situation varied, perhaps because of race-based perceptions.
In Raleigh and Cary, N.C., technology advocacy group RIoT is leveraging funding from US Ignite in support of a broad-based VR challenge. The effort casts a wide net, seeking ways to implement the technology in support of a range of government functions.
The challenge looks at ways to increase tourism and, in particular, to leverage the convention center. Developers also are invited to offer AR and VR solutions around inclusivity as well as workforce development.
“We are helping to crowdsource potential solutions from entrepreneurs, startups, small businesses,” said RIoT Executive Director Tom Snyder, who predicts augmentation will have a far-reaching effect on civic life.
“We’re going to see impacts not just across these statement areas, but across every aspect of how we interact in our communities,” he said. “When we can augment more and we can do it more and more in real time, we’re going to be able to solve problems that just weren’t solvable before.”
Across these municipalities, those who are pursuing augmented solutions describe a range of potential benefits.
For first responders, AR and VR could expose trainees to more diverse scenarios, Noble said. He also likes the fact that emergency personnel can train on hazardous situations without being exposed to actual hazards.
“We want to use this for the high-acuity, low-frequency incidents that it’s hard to train for in real life, things like hazardous materials or active shooter incidents,” he said. “There are things like that, that hopefully we don’t respond to very often, but we still have to be prepared for.”
Yates said she is intrigued by the possibility of being able to open up new modes of communication.
“The visual approach is a very compelling tool for getting people to engage, versus just talking,” she said. “There are people who can’t do it with just words, who can better understand the pros and cons if you give it to them in pictures.”
For Tierney, one big potential advantage to augmentation lies in its geographic reach — the ability to deliver training anytime, anywhere. That could be a game-changer in social services, where training budgets may be lean, she said.
“If you are running a statewide child welfare agency and you have a training in one part of the state, people are going to have to travel across the state to get to it,” she said. “This offers an alternative.”
She also expressed enthusiasm about the unique ways in which VR might help individuals in state government to tackle their own implicit biases. She pointed to her team’s social service example, where individuals get to gauge their own reactions to people of different ethnicities.
“It gives them opportunity to see — minute to minute, transactionally — how bias is actually manifesting itself,” she said. “That is enormously important. If we don’t find new ways to practice our way of being, we will just keep playing like we have always been playing. This creates an opportunity to practice it a little differently, and then when we’re out in the world, we might actually change how we react.”
In terms of outcomes, some have expressed concern that virtual training in a simulated space may not carry the same weight as a more conventional hands-on or classroom experience. While it’s probably too soon to say for sure, some initial results suggest VR and AR experiences can in fact drive meaningful outcomes.
Noble has done tests, training a cadre on a simulated Ambus using AR tools, while another group trained using VR scenarios and a third learned via traditional training.
“We found that the VR and AR groups greatly improved their memory on where items were in the bus. Their retention of all the things that we wanted them to learn was far greater than the ‘traditional’ group,” he said.
“We think it’s mainly because they were able to train any time they wanted,” he said. “And they were able to do it multiple times in VR and AR.”
For state and local entities looking to adopt augmentation, either for training or for citizen services, those in the trenches suggest a number of best practices.
As a still-emerging technology, there are not yet a lot of set standards around AR and VR platforms. Snyder suggested it will be important for government organizations to seek out solutions that share some common ground, in order to make them both broadly applicable and also cost effective.
“It’s not wise for government to jump in on a solution that is truly only working in your community,” he said. “It’s important to be open, to look for things that can maybe pilot in one location and then can scale into other communities. We don’t want every single town in America running a completely different technology stack.”
Civic problems tend to ignore jurisdictions. “Traffic, pollution, storm water, those don’t stop at the county line,” he said. “When we’re developing solutions for these different things, it is important to have interoperability. We should do this in ways that are open source, so they can be replicated in other places.”
For those looking to implement pilots, Noble said, it’s important to have leadership on board. To that end, he said, IT will need to come to the table with metrics of success.
“It’s a new technology, I don’t know how well-proven it is. One of the biggest hurdles that we’re going to face with VR and AR is actually proving that it works just as well as ‘real’ training, or better,” he said. “We did beta tests with our cadets where we proved that their memory retention and time-on-task greatly improved with VR and AR. That went a long way with our leadership in showing that this is not just a toy or a video game.”
As government looks to augmentation as a way to deploy training across broad geographies, Dick said, IT leaders will need to think about how personnel in the field are accessing those tools.
“Especially if you’re talking about something that requires wireless VR, you might not be able to train people who are in a rural area with very low bandwidth available to them,” she said.
“VR is great, but you have to make sure it’s usable for everyone that you’re asking to train on it,” she explained. “You need to make sure that there are alternatives available if people can’t access it, whether it’s due to bandwidth and infrastructure or physical ability.”
In terms of implementation, it may be helpful to support augmented learning with a degree of human contact, ideally in the form of follow-up encounters.
“Let’s say 15 individuals have each gone through the headset experience,” Tierney said. “Then they’re invited into a seminar where they get to talk together and learn the differences about what you saw and what I saw, what each of us noticed. We can begin to unpack our thinking together.”
Overall, these early implementers agree, AR and VR together hold great promise across a range of potential civic uses.
“We as humans have always tried to augment ourselves. When our vision starts to go, we put on glasses. If our hearing starts to go, we get a hearing aid,” Snyder said.
“We want to use technology to make ourselves better,” he said. “AR and VR are going to be transformational in bringing clarity, in bringing real-time information to us, right when it’s needed. It is going to affect every single aspect of government.”