It’s Time to Convert: The Path from Human Coding to a New Way

There has long been a reliance on the voice of the consumer to develop deep market research insights and guide decision making. A recent explosion in the volume of behavioral data means we often know the ‘what’ but using open-ends to understand the ‘why’ is as important as ever. The prevailing opinion for years among most researchers was that human interpretation and coding of these comments was the gold standard. Even though it was labor intensive, error prone, and costly, only humans could sift through expressions and other jargon associated with brands and interpret conversational text to provide accurate coding…right?

Well, not necessarily.

Human coding certainly has its advantages, but it has important challenges too. Humans come with biases of their own. Multiple trained coders can take the same data set and will rarely produce the same underlying code frame. In addition, comments and codes are often interpreted differently from one another. Human coding is manual, takes a lot of time, and for studies with larger sample sizes, often means coding only a random sample of the comments, which is not optimal. Therefore, the most time-consuming parts of the survey for participants are not always being used in the analysis. Also, for a research agency, the ebb and flow of coding demand means it is difficult to right-size a coding team. This often leads to using contractors, which introduces further bias and inconsistency. These limitations don’t mean that open-end comments are any less valuable, but they were begging for a better alternative to analyze them.

The development of advanced Natural Language Processing (NLP) algorithms and major increases in computing power have been game changers. Early attempts at leveraging technology to code were limited to a simple word search and did not always capture the broader idea that was intended in the text. This did not yield the required level of accuracy or precision. With recent improvements in language models, computers now group words with similar meanings and can eliminate common words to quickly identify the main themes in each comment.

At CMB, we have developed our own custom solution, themeAI, to code open ends into relevant categories using a combination of Artificial Intelligence (AI) and Human Intelligence (HI):

• We leverage best in class NLP algorithms to sift through thousands of comments to identify common themes. No longer is there a need to staff up or down to meet volume demands with AI doing its job of analyzing large volumes of text quickly
• These algorithms are impartial, avoiding the inherent biases and fatigue of human coders, but they are not perfect…yet. Advancements in language models continue and we stay abreast and incorporate new functionality as it becomes available
• We complement this with HI, using CMB’s deep industry knowledge and hands-on project approach to analyze, refine, and direct the computer-generated themes to ensure they address key project objectives. This is all custom to each project and not forced to fit an existing framework

There are many benefits to this approach and our clients across many industries including Financial Services, Entertainment and Travel and Leisure have used themeAI and are responding favorably and coming back for more.

To learn more about how themeAI can be a differentiator for you, please contact info@cmbinfo.com or contact us here.