Why Winning the World Series May Not be the Right Goal
I’ve worked in research for over 20 years. During that time, I’ve seen countless brand trackers and brand funnels, and talked through a lot of brand drivers. I’ve had the pleasure of leading them from the client side and the brand side. I’ve also experienced “every” KPIs (e.g., Unaided Awareness, Consideration, NPS, Ad Recall, Key Attributes).
Inevitably, each year clients ask me, “what should our goal be for [fill-in-the-KPI] for [fill-in-the-time-period]?” Unfortunately, it depends.
I’ll use a baseball analogy to explain. I’m a lifelong Chicago Cubs fan. If you asked me what the Cubs’ goal should be for the year, it’d vary widely depending on a lot of things (just like setting KPI goals). In 2023, I never would have said “win the World Series” for my beloved Cubs. Maybe in 2017, but not 2023.
So, how do you set KPIs? Here’s my guide for market research tracking studies:
- Reflect on the current team. Some brand metrics are more stable and slower moving than others. Is the metric relationship-oriented? Relationship metrics move and react slowly (e.g., a consumer’s likelihood to consider or recommend a brand doesn’t just change overnight). Transactional metrics can support bigger moves, things that might take one touch or exposure (e.g., if a consumer has seen an ad or recognizes a logo). It’s safer to be more aggressive on one side than the other. So, make sure you reflect on your current team before finalizing your goal.
- Look at last year’s win total. Rarely are you setting KPI goals in a vacuum. How much have you seen the KPI move in the past 3 years? How about the related attributes driving it? If your brand is in a well-established category (e.g., CPG, athletic wear, travel), with a brand that’s been in the market for years, you’ve likely seen stable, slow moving KPIs. Looking at last year’s “win total” can help. Historic improvement vs. the prior time periods, can give a realistic sense of the realistic range of movement in your category and brand.
- Know the General Manager’s plan. Review the plan for next year and reflect on how different it’ll be from the prior year. What is your brand planning for the future? Will marketing, product, and/or spending be meaningfully different than prior years? This can level-set expectations. If you have game changing launches planned for next year, while the prior year had none, that’s a recipe for bigger movements. Planning to spend $10M on a new campaign? That’s great, but less meaningful if last year was also $10M or if the competition is spending $50M. So, basically, you need to know how aggressive your GM will be in the next year!
- Use sabermetrics. Many tracking programs include driver models. These models often stop at attribute prioritization (e.g., coefficients, importance scores). It’s worth taking the extra step to simulate their impact on your dependent (KPI) variables. At CMB, we often quantify the lift required of drivers to equate to a 1-point lift in the KPI. Those “sabermetrics” contextualize what’s feasible. If it takes a 10-point lift in a driver to equate to 1–point lifts in your KPI – you may not want a KPI goal that’s any bigger than that!
- Consider what “winning” means for your brand. I often get asked, “at 95% confidence interval, what lift does it take to flag as statistically significant?” That threshold sets a very stringent criteria to flag. You need either a huge sample size (which costs big dollars) to flag minor changes. Or you have to live with a sizeable range (e.g., +/- 5% points). Often, lower confidence intervals can still provide “enough” confidence that the KPI improved with smaller thresholds, smaller sample! What is a win? And what are you comfortable with?
If this sparked your interest, don’t hesitate to reach out.