How to Develop Chatbots With Real-Time Sentiment Analysis

In recent years, however, sentiment analysis has evolved to include real-time narrative mapping that allows the chatbot to look at the important words in a sentence and assign them a relative value of positive, neutral, or negative, giving the bot an understanding of the entire tenor of the conversation. For example, if a caller says, “I really liked your article from last week, and I’m looking forward to the next one,” that would get a 90 percent positive sentiment rating. From this initial rating, the chatbot can utilize complex webs of narrative mapping to determine the best way to respond to and dialogue with the caller.

While this kind of mapping technology may seem like a thing of the future, it’s very real—and it’s helping to enhance customer communications in businesses today. Here’s a look at how it all works.

Capturing Unique Comments

When creating chatbots that can analyze the sentiment of a conversation in real time, it’s important to consider the use case. In financial services, for example, a chatbot can mine answers to questions from a limited number of predefined responses. To answer unique questions on the fly, chatbots require natural language processing (NLP) that can actually understand normal human speech. But how exactly can this be done?

To map out the narrative decision tree that chatbots can support, developers rigorously test their chatbots, all their functions, and the user experience by expanding the chatbot vocabulary. For example, tools such as API.AI, IBM Watson, and Microsoft Bot Framework can instill more lifelike comprehension in chatbots—which is especially important for use cases like customer service and commerce that regularly handle customer inquiries.

This type of NLP support has already helped some leading companies—like Lyft—capture rich, free-form customer feedback that ordinary closed-domain chatbots could not. For sentiment analysis to actually work for these companies, the chatbot cannot be left to decipher the decision tree on its own; humans and machines have to work together to use NLP to interpret the sentiment and continually refine the chatbot vocabulary.

Collaborating with the Bot

Long before PCs, mobile phones, and the internet became accessible everywhere, something like narrative mapping was functioning in interactive voice-response (IVR) systems. IVR technology gives callers a decision tree of available options. If a customer wants to check the balance due on a utility bill or make a payment, for instance, these are clearly definable choices an IVR can support. But if the customer wants to ask about an undefined choice, such as why charges are higher this month, they can quickly become frustrated by the technology’s lack of understanding.

In contrast, a chatbot powered by NLP can support analysis of sentiment that can identify customer dissatisfaction. From there, the call can be connected to a live contact center agent for more hands-on and personalized engagement. Once the contact center agent answers, the chatbot can continue to help in whisper mode, providing background information on the caller’s location (down to the street or ZIP code), prior transactions, and other information. In this case, the chatbot keeps working to improve the customer experience, even if only in the background.

The Right Language for Sentiment Analysis

Even though chatbots and analysis of sentiment are powered by artificial intelligence (AI), the narrative mapping—or topic mapping—that forms the basis of the decision tree behind the chatbots has to be created manually. A subject-matter expert must rank topics by primary keyword, additional keywords, and excluded keywords, which requires a specific language understood by both the expert and the machine.

With its built-in functionality in the Natural Language Toolkit (NLTK), Python is a popular language for narrative mapping. It takes the taxonomy-based approach, identifying topics and subtopics and attributing sentiment accordingly. And since Python NLTK was first released in 2001 — five years ahead of a Java-based competitor — it got a head start and was applied to many use cases and supported by a wide range of open-source libraries. This Swiss-Army-knife capability makes Python NLTK a good choice not only for simple chatbots but also for advanced machine learning.

Whatever the future holds, whether chatbots are used solo to answer phone queues on IVR or to back up contact center agents in whisper mode, the sentiment analysis use case will no doubt be part of the solution. And this part will only increase as machine learning and AI make chatbots better at their jobs over time.