About a 2 min. read
There has long been a reliance on the voice of the consumer to develop deep market research insights and guide decision making. A recent explosion in the volume of behavioral data means we often know the ‘what’ but using open-ends to understand the ‘why’ is as important as ever. The prevailing opinion for years among most researchers was that human interpretation and coding of these comments was the gold standard. Even though it was labor intensive, error prone, and costly, only humans could sift through expressions and other jargon associated with brands and interpret conversational text to provide accurate coding…right?
Well, not necessarily.
Human coding certainly has its advantages, but it has important challenges too. Humans come with biases of their own. Multiple trained coders can take the same data set and will rarely produce the same underlying code frame. In addition, comments and codes are often interpreted differently from one another. Human coding is manual, takes a lot of time, and for studies with larger sample sizes, often means coding only a random sample of the comments, which is not optimal. Therefore, the most time-consuming parts of the survey for participants are not always being used in the analysis. Also, for a research agency, the ebb and flow of coding demand means it is difficult to right-size a coding team. This often leads to using contractors, which introduces further bias and inconsistency. These limitations don’t mean that open-end comments are any less valuable, but they were begging for a better alternative to analyze them.
The development of advanced Natural Language Processing (NLP) algorithms and major increases in computing power have been game changers. Early attempts at leveraging technology to code were limited to a simple word search and did not always capture the broader idea that was intended in the text. This did not yield the required level of accuracy or precision. With recent improvements in language models, computers now group words with similar meanings and can eliminate common words to quickly identify the main themes in each comment.
At CMB, we have developed our own custom solution, themeAI, to code open ends into relevant categories using a combination of Artificial Intelligence (AI) and Human Intelligence (HI):
There are many benefits to this approach and our clients across many industries including Financial Services, Entertainment and Travel and Leisure have used themeAI and are responding favorably and coming back for more.