Our last blog described how organizations can improve their Net Promoter ScoreTM by analyzing verbatim responses. In fact, today organizations have a wealth of information in various text data sources such as customer complaints, reviews and social media conversations. Most customer centric organizations deploy text analytics tools to glean actionable insights. But, still if you ask any analyst that “How often were you stuck with text analytics tools, and had to massage the data manually?” the answer would most often be, “Almost always.” And the underlying reason for this is the complicated and contextual nature of text analytics itself.
For example, an airline that used a leading text analytics tool to mine customer complaints had “upgrade to business class” as a key theme. However, this theme is not specific enough to drive any Action. It is unclear whether the customers complained about “upgrades that were cancelled” or “inability to use miles for upgrades” or were simply saying that the “upgrades are expensive”. Each of these specific themes would demand different action and it is very difficult for a software to extract insights at this level of specificity.
Also, accuracy is often a concern with text analytics tools as they can’t adequately interpret the context and nuance of human language. For example, as Andrew Wilson rightly noted in this article , most of today’s text analytics tools would classify the following statement as a negative comment about the Scion: “With the supercharger included on my Scion, it is one bad machine,” not being able to recognize the colloquial use of the word “bad” to actually mean “good”.
While specificity is needed to make the themes actionable, accuracy is needed to act with confidence (especially when the action is at a micro level). At the current rate of Natural Language Processing (NLP) technology advancement, it will be decades before technology evolves to the level of human interpretation. As a result, leading technology companies like Google and IBM have increasingly relied on humans to train, evaluate, edit or correct an algorithm’s work, as explained in this article in the New York times.
Organizations need to understand where NLP works best and where it doesn’t so that human review can be used to complement NLP. Because human review becomes expensive and time consuming as the scale increases, it is very important to optimize the combination of NLP and human review so that it is scalable & cost effective.