Perhaps the best practice that I saw at the PTI event — and the most outstanding local government technology project I have seen in the past year — comes from the “Monty” project in Montgomery County, Md.
The presentation team from Montgomery County was led by Shayna Taqi. Taqi serves as Montgomery County's chief change officer within the Department of Technology and Enterprise Business Solutions (TEBS). She has over 20 years of experience in leading strategic change management initiatives. Her division is responsible for deploying methodologies to increase understanding and adoption of large-scale innovation and technology implementations. Taqi currently leads the county’s first generative AI (GenAI) project. This technology has expanded Montgomery County's customer service presence, allowing residents and visitors to easily obtain information on county services from a virtual chat agent on the county's 311 website.
Taqi received her B.A. from American University and a Certified Public Manager designation from the Metropolitan Washington Council of Governments, Regional Executive Development Program.
After the PTI event, I reached out to Shayna to interview her for this blog. My goal was to share the Monty project with the nation and the world.
Dan Lohrmann (DL): Tell us how the Monty project started. What did Monty 1.0 include back at the beginning?
Shayna Taqi (ST): Our initial website chatbot, Monty 1.0, was launched in January 2021 to help the county’s 311 team meet increased resident service demands during the COVID-19 pandemic. The Monty 1.0 chatbot used Zammo.ai artificial intelligence technology and multi-turn conversation diagramming in English and Spanish to fulfill resident information requests pertaining to the two dozen or so most popular 311 topic areas.
DL: Coming out of the pandemic, how did the vision change?
ST: During the COVID-19 pandemic, Montgomery County residents were experiencing high wait times to reach a MC311 customer service representative. We knew we needed to provide an additional communication channel as soon as possible since many of these requests were time-sensitive in nature, such as rental and financial assistance. Within just a few weeks, the Monty 1.0 chatbot was configured and went live on the MC311 website by a small project team within TEBS, and through partnerships with Microsoft and Zammo.ai. Monty 1.0 included the top services residents were calling in for. We also changed the launch options on the telephony system to route urgent calls quickly. Within a two-month period from launch, hold times dropped from 5 minutes to 2 minutes, and call abandonment (hang-ups) dropped from 25 percent to 11 percent.
Despite successfully alleviating some of the increased service demands placed on the county’s 311, resident feedback associated with the Monty 1.0 chatbot was often not always positive, since it did not address all questions a customer service representative could answer. The effort to expand Monty's database of knowledge was manual and required advanced technological proficiency, which limited 311’s ability to rapidly develop and maintain a knowledgeable chatbot. As a result, residents commented that the chatbot was too limited in scope and unable to comprehend basic phrases and requests. As a direct result of our resident’s feedback, the county’s Department of Technology and Enterprise Business Solutions initiated a proof of concept in May 2023 to evaluate the use of generative artificial intelligence technologies as a solution to these problems.
DL: What does Monty 2.0 offer today? How many languages can you support for residents?
ST: The Monty 2.0 chatbot uses generative artificial intelligence technologies to facilitate lifelike conversations with residents; absolutely no multi-turn diagramming is required. The chatbot is capable of instantaneously answering information requests related to over 3,000 311 topic areas — a hundredfold increase from the chatbot’s previous iteration. Conversations are supported in over 140 languages, including seven of the most popular languages spoken in Montgomery County (English, Spanish, French, Mandarin Chinese, Korean, Vietnamese and Amharic).
DL: What GenAI and other technologies are used to build Monty?
ST: The Monty 2.0 chatbot is underpinned by a synapse pipeline (a grouping of activities that perform data-driven workflows) that feeds 311 knowledge base information to the county’s Microsoft Azure platform. There, the 311 knowledge base information is transformed and indexed for use by Microsoft’s GenAI Cognitive Search service. The chatbot application, workflows and front-end user experiences are all facilitated using Zammo.ai tools.
In short, a chatted request is first received and transmitted to the Microsoft Cognitive Search service via Zammo.ai workflow. The Microsoft Cognitive Search service ranks and selects indexed 311 knowledge base information, in accordance with the chatted request. This information is then transmitted to OpenAI via Zammo.ai workflow, where it is composed into a coherent reply using ChatGPT GenAI technology. The reply is then transmitted to Zammo.ai and delivered to the resident, completing the request cycle. The entire request/response process is facilitated in a manner of seconds.
DL: How can Monty be accessed? Describe the main benefits that Montgomery County residents receive by using this service?
ST: The Monty 2.0 chatbot is accessible via the county’s 311 website. The chatbot allows Montgomery County residents to instantly procure headline information pertaining to thousands of county services in dozens of languages without the hassle of weeding through pages of information or dialing up a customer service representative. Residents simply type or voice chat their query within the chat window and Monty 2.0 does the rest. The Monty 2.0 chatbot excels at answering clear and concise requests, allowing the county’s 311 customer service representatives to research and problem-solve more complex resident issues.
DL: What has been the user reaction/feedback so far?
ST: Reception of the Monty 2.0 chatbot has been generally favorable since its launch in March 2024, per analytics collected by the county via Zammo.ai feedback features. Resident feedback is automatically solicited via thumbs up/down buttons and comment forms as part of every user interaction with the chatbot and Monty 2.0 chatbot usage has more than doubled that of Monty 1.0, when comparing year-over-year dates. In general, residents have appreciated the chatbot’s improved scope and functionality compared to the Monty 1.0 chatbot; resident feedback has also been critical in identifying previously undetected defects and other feature requests. TEBS continues to learn from resident feedback and iterate enhancements to the Monty 2.0 chatbot, including improvements to prompts, workflows, knowledge base information and other back-end components, to reduce the rates of incorrect and unanswered resident queries.
DL: Can you provide a few more details on resident feedback?
ST: Resident feedback has generally centered on three areas: improvements to our knowledge base, improvements to the chatbot user experience, and requests for new content and features. Resident feedback has been crucial in helping our team pinpoint errors and defects present within the knowledge base, including broken links, outdated content and other related issues. We are also leveraging resident feedback to guide our prompt engineering back-end efforts and improve the overall performance of the chatbot's large language model infrastructure. We also continue to maintain a backlog of resident enhancement requests to inform future iterations of the Monty chatbot.
The most common enhancement requests we receive involve requests for functionality directly within the chat window itself — for instance, submitting county service request forms via the chatbot, transferring conversations from the chatbot to a 311 customer service representative, and receiving address-specific service information.
Incorporating other service levels, such as 911, state and federal assistance, may be explored in the future as Montgomery County continually looks for ways to improve service delivery for our residents. We are also in the preliminary stages of revamping the county’s web pages.
DL: When your team presented at the recent PTI AI and Cyber Summit in Washington, D.C., I was impressed with how you brought your team along and let them present a piece of the project. They clearly loved their jobs and were proud of what they are doing. How did you create such a positive culture? Do you have a mix of public- and private-sector staff?
Gail Roper, Chief Information Officer, Program Sponsor
Shayna Taqi, Project Lead
Manu Daniel, Technical Project Manager
Michael Zanfardino, Functional Project Manager
Tushar Parekh, Developer
Sri Harsha Kotagiri, Developer
Colin Cox, Product Owner
Skyler Grubbs, Change Management Lead
Lauren Der, Communication and Marketing Lead
ST: The Monty 2.0 chatbot was developed entirely in-house by TEBS, with close support from our Microsoft and Zammo.ai vendors. Our project team developed a special rapport over the course of 10 months, from inception of the initial proof of concept to deployment of the chatbot. The Monty 2.0 chatbot project represented one of many competing projects for members of the project team, who were periodically borrowed from several divisions throughout TEBS to lend expertise in areas such as project management, business analysis and software development. As such, we adopted an agile, sprint-based approach to the project to best maximize our team’s productivity.
Our sprints were intimate and involved hours of close cooperation between multiple team members to design, develop and vet chatbot components. This vetting ultimately included upward of a dozen functional testing rounds and the facilitation of a resident focus group to inform our end product. Through it all, team members routinely wore multiple hats and alternated roles in accordance with the needs and phases of the project. We continuously learned from our missteps together as a group and were never afraid to “fail forward” in the pursuit of innovations. This fusion of diverse thought and skills and openness to constructive feedback drove the project’s success — and still shows in the close camaraderie present amongst our team.
DL: What is next for Monty? Where do you hope to go in the next year or two?
ST: Our project team is currently working on several major enhancements to the Monty 2.0 chatbot, including the incorporation of county GIS features to better assist residents with user-specific information requests, such as the location of voting centers, status of snow removal activities and other related items. We are also working closely with our Microsoft and Zammo.ai vendors to improve the chatbot’s overall performance through back-end component upgrades. These upgrades include the installation of Microsoft Azure Prompt Flow, a development tool used to assess and improve the accuracy of the prompts guiding the Monty 2.0 chatbot’s GenAI components. Our nascent product ownership team also continues to evaluate UI/UX improvements and other knowledge base enhancements to comprehensively address resident-submitted defects and increase overall adoption of the chatbot. Lastly, we aim to parlay our project team’s hard-earned expertise to tackle additional GenAI projects within the county sphere.
From the beginning of this project until now, we’ve realized the value of continuous improvement and leveraging the operational efficiencies we made during the COVID-19 pandemic. We are extremely proud of the team’s ability to rapidly support our residents while modernizing our 311 technology and more holistically enhancing Montgomery County’s customer service delivery model and capabilities.
