Apr 05, 2022

The Importance of Continuous Improvement in Conversational AI

There's no denying the power of Conversational AI. In a 2021 study conducted by IBM, 99% of companies reported an increase in customer satisfaction due to using conversational AI solutions like virtual agents. Leaders went on to see higher first contact resolution, containment, and better intent recognition. That has dramatically reduced call abandonment rates, re-routes, and operational costs while enhancing the overall customer experience – often in a matter of days to improve outcomes immediately.  

But it's not enough to deploy Conversational AI. Enterprises can't assume these bots are set-it and forget-it solutions. Conversational AI requires continuous improvement and optimization to learn and improve as a self-learning system over time. If you have a conversational AI system in place or are considering one for your organization, here are some critical steps to ensure you continue improving its efficiency over time…

Monitor and measure effectiveness continuously.

Without measuring performance with honest user conversations, you'll never know if your bot works for your customers. To measure effectiveness, you must establish success metrics and monitor them regularly. There are many metrics you can consider, including:

• Engagement rate: The percentage of total sessions that are considered engaged. An engaged session is when the user creates a topic (as opposed to the bot prompting a question or suggesting a topic), or the session ends in escalation to a live agent. You can further analyze the customer journey depending on whether the session is resolved (either by the bot or the human agent if escalated), further escalated to a supervisor, or abandoned.
• Abandon rate: the percentage of engaged sessions that are abandoned. This means the session was neither resolved nor escalated.
• Resolution rate: the percentage of resolved engaged sessions, indicated by the user acknowledging that their question has been answered or the issue has been resolved. By asking the customer to rate the session, you can measure their satisfaction score (CSAT).

Analyze transcripts and get user feedback

Collecting and analyzing transcripts of customer-VA conversations is a great way to gain valuable insights from interaction data for making critical system improvements. You can more easily identify issues customers are going through and where your bots are falling behind to improve their performance. Analyzing transcripts will also help you better understand what information your customers want so you can retrain your bot to surface it faster. You can explore a sampling of chats at a more granular level to better understand specific issues. Or you can look across all chat sessions to see what percentage of overall chats were positive, neutral, or negative. The same analysis can be conducted for conversations customers have with your conversational IVR.

You can also conduct surveys across channels to better assess your customers' satisfaction with your bot's performance. Voice surveys are beneficial, letting customers say what they're thinking in their own words versus conforming to a one-size-fits-all questionnaire. You'll get more robust insights from your customers and likely better response rates.

Iterate through train and test cycles to improve AI accuracy

A robust conversational system is built by multiple iterations, training, and testing cycles combined with ongoing monitoring and tuning. After all, there's no such thing as a single testing phase. A general rule of thumb is to use 80% of your customer data for training and 20% for testing. Testing can include regression testing (identifying flaws in the conversation flow), NLP testing (improving your VA understanding), E2E testing (verifying the end-user experience), voice testing (understanding users on voice channels), performance testing (ensuring your VA is responsive under high load), and security testing (making your bot secure). The goal isn't just to deploy conversational AI. It's to continuously improve the solution by identifying defects and introducing new functionality to create more positive and memorable user experiences.

According to Gartner, 85% of AI projects fail – many never reach deployment, while others are abandoned after failing to meet expectations. This is precisely where a specialist like Servion can help:

1. Domain expertise to help identify use cases and business objectives: Most businesses rush into deploying a conversational AI solution without clearly identifying the business problems that the AI solution should be addressing. Servion, with its 25+ years of customer experience and expertise, helps identify good use cases for conversational AI and determine the KPIs for measuring success.
2. Partnership with leading Conversational AI platform providers: Servion has established partnerships with leading AI technology companies like AWS, Google, and Nuance. This allows us to recommend the best conversational AI platform that suits the customer's requirements.
3. Readily available pool of resources: Not every business is equipped with the skilled resources required to build and maintain AI projects. Servion can tap into its talent pool that has implemented numerous conversational AI projects across several platforms.

Our clients have improved customer experience by deploying AI-powered self-service solutions and saving millions of dollars by deflecting calls to a low-cost self-service channel. Connect with our team to get a personalized demo.

Servion has helped some of the world's most prominent brands design and deploy AI-powered conversational systems.