Transform Data’s Impact: Pick The Right Success KPI!

Your analysis provides clear data that the campaign was a (glorious) failure. It could not be clearer. The KPI you chose for your brand campaign was Trust, it had a pre-set target of +5. The post-campaign analysis that compares performance across Test & Control cells shows that Trust did not move at all. (Suspiciously, there […]

The post Transform Data’s Impact: Pick The Right Success KPI! appeared first on Occam’s Razor by Avinash Kaushik.

Your analysis provides clear data that the campaign was a (glorious) failure.

It could not be clearer.

The KPI you chose for your brand campaign was Trust, it had a pre-set target of +5. The post-campaign analysis that compares performance across Test & Control cells shows that Trust did not move at all. (Suspiciously, there are indications that in a handful of Test DMAs it might have gone down!)

Every so often, the story is just as simple as that.

You do the best you can with a marketing campaign (creative, audience, targeting, channels, media plan elements like duration, reach, frequency, media delivery quality elements like AVOC, Viewability, etc.), and sometimes the dice does not roll your way when you measure impact.

You would be surprised to know just how frequently the cause for failure is things that have nothing to do with the elements I mentioned above.  In future Premium editions we’ll cover a bunch of these causes, today I want to cover one cause that is in your control but often a root cause of failure:

Judging a fish by its ability to climb a tree!

AKA: You picked the wrong KPI for the campaign.

[Note 1: I’m going to use the phrase Success KPI a lot. To ensure clear focus, clear postmortems and clear accountability, I recommend identifying one single solitary metric as the Success KPI for the initiative. You can measure seven additional metrics – say for diagnostic purposes -, but there has to be just one Success KPI. Close accountability escape hatches.]

[Note 2: Although the guidance in this article applies to companies/analytics teams of all sizes, it applies in particular to larger companies and large agencies. It is there that the largest potential for mischief exists. It is also there, with an army of brilliant Analysts, that the highest potential for good exists.]

[Note 3: This article, part 1 of 2, was originally published as an edition of my newsletter The Marketing < > Analytics Intersect. In part 1, below, we’ll sharpen our skills in being able to recognize the problem, and five of the twelve rules for success. If you are TMAI Premium member, check your inbox for TMAI #313 for part 2 with the remaining rules and additional guidance. If you can’t find it, just email me. Merci.]

Be sure to save the summary visual at the end for implementing it in your company/agency.

The Subtle Art of Picking Bad KPIs.

Example 1.

Let’s say I work at Instagram, specifically in the Reels team. We want Reels to, say, crush TikTok. The team runs a $250 mil multi-platform campaign to increase Awareness of Reels. The campaign Success KPI was chosen to be: Incremental Reels Videos Created.

Good campaign. Bad Success KPI.

If you truly build Awareness creative, then judge success using the KPI Awareness. No?

Fish swim.

[Yes, long-term success of Reels will only come from Instagram users uploading Reels, but that was not the problem the creative was solving for. If the goal was Incremental Reels Videos Created, you would build an entirely different creative, you would target the campaign, potentially, to a different audience, you might create a different media plan, you would… run a different campaign.]

Creating a performance Success KPI for a brand campaign is a particularly common, and heartbreaking, mistake. Sophisticated brand measurement is hard. It feels simpler to pick what’s easy to measure, but you are going to make the fish feel bad when you judge its ability to climb trees AND you don't accomplish the desired outcome.

Example 2.

Let’s say I ran the campaign mentioned at the top of this email for my employer American Express.

If you look at Brand Trackers published by numerous industry sources, it becomes apparent in two minutes that American Express does not have a Trust problem. Americans trust American Express in massive quantities.

If you run a trust campaign for American Express, that campaign is going to fail. You are solving a problem that’s not a problem.

Bad Success KPI because of, technically speaking, high baselines.

Example 3.

Your new, Extremely Senior Leader is obsessed about doing Marketing that makes people fall in love with our brand. So, they conceive of a multi-million dollar Social campaign and demand the success KPI be: Brand Adoration.

[A KPI like that instinctively makes Analysts cringe because what’s Brand Adoration anyway. What does that even mean? Do we just make something up? If we do, how would we ever know if we did something meaningful, how we are doing compared to competitors/industry, what kind of creative/media do we even use to build “Brand Adoration,” and what are the core drivers of Brand Adoration, and if you don’t know, what are you actually doing spending all this money? I am going to set all this aside for a future TMAI Premium editions!]

You’ll measure that KPI using a question (or five) that will be presented in both the Test & Control cells. Will anyone who is not an employee of your company or in your Team's orbit even understand what the question is?

Let’s say, you ask Do you adore PayPal? Will the responding human know how to process this question?

Let’s say, you try an even more clever trick and ask PayPal is my preferred choice for financial transactions of a personal nature, and I would never use any other service, choose Yes or No.

Would the responding human understand that you are measuring brand adoration and give you a valid answer?

This is a bad Success KPI because no responding human can understand what you are asking – then the signal you accumulate to assess the campaign success or failure is a false signal.

And, it is the analytics person/team/agency's mistake.

Example 4.

A little grab bag for you…

When you are trying to drive long-term profit, picking Conversion Rate as a Success KPI for a campaign would be a mistake.

For your Display Advertising campaigns, picking any Success KPI close to buying (ex: Revenue) usually is a mistake. (Assisted Conversions – over a 30 or 90-day period, depending on your business – might be better.)

Anointing Conversion Rate (or dare I say even Revenue) as the Success KPI for your Email newsletter is a double mistake. It will cause your team to use newsletters in the spirit of pushy spam, and it will stop newsletters from truly becoming a strategically valuable owned asset, as Email is magnificent at See and Think, not so much Do.

I could keep going on. I have a hundred thousand more stories of judging a fish by its ability to climb a tree.

12 Rules for Picking the Right Success KPI.

While there is enough responsibility to spread around, I rest accountability for this common mistake on the Analyst/s. Marketers, CMOs, Finance peeps should know the implications of picking an imprecise Success KPI, but the Analyst is the expert and, hence, I expect them to take the lead.

To help you do that, here are 12 rules I codified for our team to use when we pick the Success KPI for a campaign. Each of these rules helps address a common error, collectively they also help you/leaders think through the campaign strategy, consider if they are solving the right problem, and so much more beyond just the KPI.

Ready to be A LOT MORE influential in your company?

Here are 12 rules brilliant companies use for picking the right Success KPI (and do Marketing that matters):

1. Is it an industry standard KPI?

It sounds like bad news that I’m saying you are not a special snowflake, that your campaign/tactic/magnificently brilliant idea is not so very incredibly unique that you need to make up a Metric to measure its success.

When you use an industry standard KPI, you have access to standards and benchmarks – providing you the super cool benefit of being able to assess your own performance in a much bigger context. This choice also comes with guidance on best practices for measuring this KPI – so that you don’t have to invent a methodology/technique that has no benefit of the industry’s collective wisdom.

Bonus: If you use an industry standard KPI, very often you’ll get access to research related to drives of that KPI that your Creative, Media and Strategy teams will kill for. If they know the drivers, they can internalize at a deeper layer what it takes to drive success.

Try not to make up a KPI, try not to make up the formula/question/methodology for a Success KPI. On that note…

2. (If it is a made up metric:) Is the KPI definition clear and understandable by a non-employee (aka consumer)?

For brand marketing, you and I assess success using a question we ask consumers.

When we make up our own metrics, the questions come from our best expertise, they might then get changed by a non-expert (Director of Marketing, CEO) because they like the sound of a particular word or phrase. But, phrased like that… Only your Director, and five people who say yes to everything the Director says, actually understand the question and answer choices. People taking the survey are super confused or putting their own interpretation on what you are asking. Now, their answers are suspect and – regardless of if their campaign results are indicated as a Big Success or Big Failure by the data – the measurement is imprecise.

Non-employees – aka normal people – need to be able to clearly and quickly understand what you are asking in your brand measurement surveys. Both the question AND the answer choices.

For performance marketing, you can see this confusion practiced when you create compound metrics. I bet your CMO dashboard has Social Engagement on it – only you understand what that metric actually is, and the convoluted formula ensures no one will ever know why Social Engagement went up or down. Not a good success KPI.

3. Is the Success KPI a business metric or a third-order driver metric?

You might have noticed above that I’m a fan of understanding the drivers of success (driver metrics) and not just the Big Thing we are trying to move (success KPI).

But, there is a special type mistake I see often made: The driver metric is chosen as the Success KPI.

An example of this is choosing Conversion Rate – certainly a driver metric – as the Success KPI vs. Profit. Yes, perhaps Profit will go up if you have a higher Conversion Rate, but the team could just use coupons or targeting low-value customers to drive the Conversion Rate and Profit will never go up.

Another example of this is that we want to influence Trust in our company, and we end up picking Product Quality as the Success KPI. Yes, Product Quality will improve Trust over time, but the coefficient is probably petite.

To correctly identify the impact of your campaign, pick the business outcome you want as the Success KPI and not one of the many driver metrics.

4. Is the Success KPI the goal set in the creative brief?

The creative is the ad we see on TV or TikTok, it is the lines of text in your Bing ad, and it is the (hopefully not annoying) image, text, animation, call to action, in your Display ad currently running in the Sacramento Bee.

Creative teams love big challenges and are motivated by solving existential issues. Hence, when Marketers / Leaders write creative briefs, they end up briefing the team for Big Things.

Make the world believe we are as good as Apple in quality… We are trying to get customers to think we are an innovative company… Our goal is to have the world believe that we are a force for good when it comes to climate change… The campaign investment is meant to help shift the perception that we are committed to our customers in the long run!

These are all fantastic things to shoot for (if your reality matches these aspirations).

The challenge occurs when the Success KPI for all of the above campaigns is set as In-Store Sales. Or, Lifetime value. Or, Most Valuable Brand in the world.

When there is a conflict between what the creative brief is (what the ads are being built for) and the measured Success KPI, the latter is an extremely poor choice because it will invariably show failure.

Brief the creative team for an outcome that actually matters to the business, and then set that exact same outcome as the Success KPI. Clear alignment between input and output.

5. Does the KPI have headroom?

I love this one. Not only as a great rule, but also to force Marketers to be clever.

What’s headroom?

Let’s consider this brand question: Is Apple an innovative company?

The answer: Yes (68%).

That is a very high baseline. If 68% of the people think anything positive of a company, there is likely no one else left in the world to persuade.

[In the case of Apple, there are a fair number of people who love to dislike Apple. That further means, purely from a measurement perspective, no headroom.]

You cannot move an unmovable metric.

No matter how much money you spend.

Even IF the campaign had great creative, it was well delivered, on the right channels, with optimal reach and frequency. The campaign will look like a failure. And, it was not the Marketing team’s fault.

Before you pick a Success KPI, do a bit of research to understand headroom. If you have less than six or eight points, don’t solve that problem (because data is indicating that it is not a problem!).

Pick something else. Unaided Awareness of Apple Tags is just 12 points. Solve that problem. Lots of headroom!

[Note: The concept of headroom applies to performance marketing as well. You might be maxed out for the audience you can reach in a particular channel. You already have max possible Click Share on Google. There might not be any more new customers to entice across the East Coast of the US. Etc. Assess headroom available across your performance Success KPIs as well.]

​​​​ Premium subscribers will recognize assessment of headroom as another clever manifestation of the win before you spend Minerva (Pre-Flight) Check outlined in TMAI #273.]

Scoring Success KPIs.

It would not surprise you to learn that smart teams codify their thinking (frameworks FTW!), and implement a process that ensures that thinking is applied 1. at scale 2. at the right moment, and 3. is understood by all.

That’s the real success to winning influence with data. To make it easier for teams I've led to implement the rules for success KPIs framework, we use the following checklist (with part 1 rules)…


[Click image above for a higher resolution version. It is pretty easy to type it all up in Excel, but if you need an Excel version, just email me.]

A thoughtful assessment, upfront. Simple and clear to all the cross-functional teams involved (and not just the Analytics team).

Rules 1 through 8 are mandatory, all of them have to be met for a KPI to be anointed a success KPI. The scoring in light blue row above. Rules 9 through 12 are for Analysis Ninjas, those who want to go above and beyond, those who do not leave things to chance, those looking for coming as close to guaranteeing success as possible. The scoring is in the darker blue row.

The KPI candidate with the best score wins! :)

In a future blog post, we can cover the process to put in place to ensure this happens at scale in your company/non-profit.

Bottom line.

Measuring the wrong thing should be the last reason to get a false signal of the impact of a campaign. False positive or false negative.

Measuring the right thing, and ensuring there is a process and framework in place to discuss that up front, ensuring every good and bad dimension of thinking can be put on the table up front, is a gift of immeasurable proportions to your employer/client.

Pick the right Success KPI.

It won’t guarantee campaign success, it will ensure that you’ll know when success occurs that it is real, and when failure occurs, there are clear lessons to learn for doing better in the future.

Pick the right Success KPI.

How good is your team, your agency, at ensuring that you are picking success KPIs that deliver in-depth insights, and optimal accountability? Please share via comments below. Merci.

[Quick reminder: If you are a TMAI Premium subscriber, part 2, with rules six through twelve and bonus content, is in your inbox. If you can’t find it, just email me.]

The post Transform Data's Impact: Pick The Right Success KPI! appeared first on Occam's Razor by Avinash Kaushik.

Winning With Data: Say No To Insights, Yes To Out-of-sights!

If there is one thing the universe agrees on, it is that you should just provide data… You should provide INSIGHTS!!! In the 807,150 (!) words I’ve written on this blog thus far, at least 400,000 have been dedicated to helping you find insights. In posts about advanced segmentation, in posts about how to build […]

The post Winning With Data: Say No To Insights, Yes To Out-of-sights! appeared first on Occam’s Razor by Avinash Kaushik.

If there is one thing the universe agrees on, it is that you should just provide data… You should provide INSIGHTS!!!

In the 807,150 (!) words I’ve written on this blog thus far, at least 400,000 have been dedicated to helping you find insights.

In posts about advanced segmentation, in posts about how to build strategic dashboards that don’t suck, in encouraging you to reimagine how you pick metrics to obsess about using the magnificent Impact Matrix, and on and on and on.

Go for insights!

The Problem.

In time, I've come to hate the word insights.

In our world – marketing research and analytics – that word has come to represent data puking.

It has come to represent telling people, with dozens of reports or eighty slides, that water is wet.

I've observed, during my work across the world, when we deliver insights, we mostly deliver to our audiences things in-sight – things they can already see!

As in, the blue line is 20% above the red line. I CAN SEE THAT! Or, life-time value of California purchasers is 3x when compared to those who reside in Georgia. Oh, please, I can also see that on the table with my eyes.

This, unsurprisingly, ends up being a massive waste of your incredible talent, and an insult to the intelligence of our audience (the people who pay your salary).

This blog post was originally published as an edition of my newsletter TMAI Premium. It is published 50x/year, and shares bleeding-edge thinking about Marketing, Analytics, and Leadership. You can sign up here – all revenues are donated to charity.

The Fix.

The last time I changed jobs, I wanted to change the aspiration of what our talented team and I should shoot for.

Instead of insights, I coined a new phrase for a new start: Out Of Sights!

Our aim would be to provide out-of-sights – things people can't see.

As in, the blue line is 20% above the red line because our biggest competitor launched a new product and priced it 10% below our best product. You are explaining the performance. BOOM!

It is such a small play on words. But, my goal was to provide each peer this pause-worthy moment when they bring the results of their work: OMG, is this really an out-of-sight?

The influence on our culture, on our actions, on our audacity was profoundly dramatic. Turns out, cultural transformations can start with a word. :)

Every time we were done with our analysis, we now had a higher standard to shoot for. We challenged each other by saying, are you sure that is an out-of-sight?

Four Attributes of Out-of-Sights.

While we got the spirit right away, scaling understanding requires a common language we could share.

Our team was up to the challenge, and off to the whiteboard we went to systematically approach the problem of creating a simple framework to identify what’s an out-of-site.

This is when I feel super blessed to work with such smart people. At the end of a couple of whiteboard sessions, we came up with our criterion that every out-of-sight had to meet…

1. Novel: New and surprising.

Is this truly new (data, source, research type)? If it has been provided before, what was the impact? Why do we need to provide it again?

What do we have here that we, or the audience, never knew before? Is this just the result of a fishing expedition?

Could the audience – your peers, boss, public – get this data from anyone else? What makes you so special if you provide it?

Novel is a tough, high, standard. It eliminated 90% of the data puking and made work so much fun for the Analyst.

2. Actionable: Expressed with a clear implication for the audience.

What could the audience do as a result of your findings?

What specific current or future campaigns, activities, internal or external business strategies are being influenced by the out-of-sight?

If you don't have an identified what to do, whose job is it? Does that person know it is their job to figure out what to do based on the data?

Would it be wiser to work with your peers who make decisions, who understand strategy, to come up with actions to take based on data before you call your data actionable?

Actionable is the single most crucial element if you deal with data. It requires knowledge well beyond the data – requiring an understanding of the business, a robust set of cross-functional relationships, and an ability to persuasively influence.

3. Credible: Data source – tool, people, entity – is respected by the audience.

What steps have you taken to identify the soundness of the source?

Do you understand the limitations of how the data is collected? Have you noted the assumptions for sharing with the audience?

Does it pass the foundational 24 filters of skepticism? For example, is it simply a correlation or have you teased out causality? [Premium subscribers see: TMAI #298: Smart Statistical Significance Reporting.]

Is there room for an alternative explanation? If so, find it.

I have the following ask of the analytics team:

We have to be the biggest enemy of our work. We have to ask hard questions. We have to poke at every corner. We have to seek alternative explanations. It seems harsh, but we are probably the most analytically savvy individuals who will look at this data. After that, it is our business peers who will typically have less analytical knowledge than us. So. Be our work’s best enemy. It builds credibility.

Credibility is very helpful. Be the biggest enemy of your analytical work.

4. Relative: Expressed in context.

Is the out-of-sight expressed so that there's no doubt as magnitude or urgency?

Context derived from:

opportunity size,
possible impact,
strategic altitude,
dimension criticality,
cause-effect ratio,
and so much more, including benchmarks

[Premium members also see TMAI #263.]

The relative attribute helps speed up understanding. It ensures your out of sights really sink in.

Every finding from your rigorous data analysis has to meet the above-mentioned four attributes – N-A-C-R –, before it can be called an out-of-sight.

When you set yourself on the quest for out-of-sights, you set a standard for yourself, for your team, for your data, that will result in everything you discover being meaningful and material.

Your Out-of-Sights Jumpstart Guide.

You will find your own way to discovering out-of-sights for your company. Still, I want to help a little bit. I would like to give you a list of questions that will increase chances that you will bump into more out-of-sights.

Questions provide context, questions lead to relationships, questions expand your horizon, questions enhance your business savvy, and in doing all that, and more, questions provide that magical missing ingredient: Purpose.

And, knowing purpose increases the chances you’ll discover out-of-sights that qualify as such with all four attributes: Novel, Actionable, Credible, and Relative.

Here’s a helpful list:

1. How can I improve revenue by 15 percent in the next three months from our website?

2. What are the most productive inbound traffic streams, and which sources are we missing?

3. Have we become better at allowing our customers to solve their problems via self-help on the website, rather than our customers feeling like they have to call us?

4. What is the impact of our website on our phone channel?

5. How can I increase the number of customer evangelists by leveraging our website?

6. What are the most influential buckets of content on our website?

7. If we could only do one thing to increase revenue on our website, what would it be?

8. What is the incremental impact of our display ad campaigns?

9. Are we building brand value via activity on our website?

10. Do fully featured trials or interactive demos work better on the website?

11. What are the top five problems our customers face across our digital channels?

12. What is the cost for us to earn $1.00 on our website?

13. What is the effect of our mobile paid search strategy on our offline sales?

14. How much does the lifetime value of a customer increase if we can convert them into a 7-day active user of our mobile app?

15. What would the impact on unaided recall of our brand if we shut down all our Facebook efforts?

Not an exhaustive list by any means, just a representation of the kinds of questions I strongly believe your talent and emphatically answer.

You answer these questions not with insights (things people can already see), but with out-of-sights (things people can’t see).


When you check if the results of your analysis pass against all four N-A-C-R attributes, chances are 90% of what you do today needs to die. Because telling people water is wet, week after week, results in only sadness.

Don't deliver sadness, deliver meaning and impact. Make your job fun!

What percentage of your team's insights this week met all four of the out-of-sights criterion?

Happy out-of-sighting!

The Premium edition of my weekly newsletter shares solutions to your real-world Marketing & Analytics challenges, while unpacking future-looking opportunities. Subscribe here, and help raise money for charity.

The post Winning With Data: Say No To Insights, Yes To Out-of-sights! appeared first on Occam's Razor by Avinash Kaushik.

The Most Important Business KPIs. (Spoiler: Not Conversion Rate!)

I was reading a paper by a respected industry body that started by flagging head fake KPIs. I love that moniker, head fake. Likes. Sentiment/Comments. Shares. Yada, yada, yada. This is great. We can all use head fake metrics to calling out useless activity metrics. [I would add other head fake KPIs to the list: […]

The post The Most Important Business KPIs. (Spoiler: Not Conversion Rate!) appeared first on Occam’s Razor by Avinash Kaushik.

I was reading a paper by a respected industry body that started by flagging head fake KPIs. I love that moniker, head fake.

Likes. Sentiment/Comments. Shares. Yada, yada, yada.

This is great. We can all use head fake metrics to calling out useless activity metrics.

[I would add other head fake KPIs to the list: Impressions. Reach. CPM. Cost Per View. Others of the same ilk. None of them are KPIs, most barely qualify to be a metric because of the profoundly questionable measurement behind them.]

The respected industry body quickly pivoted to lamenting their findings that demonstrate eight of the top 12 KPIs being used to measure media effectiveness are exposure-counting KPIs.

A very good lament.

But, then they then quickly pivot to making the case that the Most Important KPIs for Media are ROAS, Exposed ROAS, “Direct Online Sales Conversions from Site Visit” (what?!), Conversion Rate, IVT Rate (invalid traffic rate), etc.

Wait a minute.


Most important KPI?

No siree, Bob! No way.

Take IVT as an example. It is such a niche obsession.

Consider that Display advertising is a tiny part of your budget. A tiny part of that tiny part is likely invalid. It is not a leap to suggest that it is a big distraction from what's important to anoint this barely-a-metric as a KPI.  Oh, and if your display traffic was so stuffed with invalid traffic that it is a burning platform requiring executive attention… Any outcome KPI you are measuring (even something basic as Conversion Rate) would have told you that already!

Conversion Rate obviously is a fine metric. Occasionally, I might call it a KPI, but I have never anointed it as the Most Important KPI.

In my experience, Most Important KPIs are those that are tied to money going into your bank account.

The paper from the respected body made me open PowerPoint and create a visual that would make the case for never identifying Conversion Rate or ROAS the Most Important KPI in your company / practice of analytics.

We expect greatness from our work, let’s focus on great KPIs.

This blog post was originally published as an edition of my newsletter TMAI Premium. It is published 50x/year, and shares bleeding-edge thinking about Marketing, Analytics, and Leadership. You can sign up here. All revenues from your subscription are donated to charity.


The Money In-Out Continuum | Intro.

When I think  of importance, I have five elements in mind.

Let’s identify them first.

The Money Making Continuum

To make money, you have to spend money. The law of God.

That’s the red box on your left.

Revenue is what the customer will pay for a product or a service. It is a range above because some products and services you sell for more, others for less.

Media Costs is the amount you have to spend on advertising (a category that also includes your Owned and Earned efforts – after all SEO, Email, Organic Social all cost money).

Hopefully, you spend less on acquiring the order than the revenue you earned. Hopefully. :)

Obviously, whatever you sell is not free to you.

Cost of Goods Sold (CoGS) is the amount it costs you to manufacture the product or the service.

As an example, revenue from selling an iPhone is approx. $1,099 and the CoGS is approx. $490. (Source: Investopedia.)

But. Wait. $609 is not all Profit. There’s more to account for.

Fully Loaded Costs (FLCo) contains the costs associated with salaries of human and robotic employees, agency fees, depreciation associated with building, free doughnuts on Fridays for all employees, credit card processing fees, discounts, and the long laundry lists of things that goes into producing the product/service that you sold to earn revenue.

I’ve represented FLCo (I’m pronouncing that as flock, what do you think?) as a smallish bar above, I don’t need to stress just how big it can be. Hence, crucial to measure and account for.

$$$ – something close to Profit – is the money left over that will go into your bank account.

Money at last. Money at last. Thank God almighty, we have money at last!


The Money In-Out Continuum | KPIs.

Now that we have a common understanding of the elements that form the money in-out continuum, we can layer in what it is that we understand when we measure every day metrics — and the ones anointed Most Important KPIs by the respected industry group.

Let's lay out the depth of what each KPI measures on our continuum.

The Money Making Continuum | KPIs

Conversion Rate is a fine metric. A junior Analyst – even a budding reporting-focused new hire – should be watching it.

But. As illustrated above:

1. It is very, very, very far from the green, and2. It does not have any sense of what it cost you to get that conversion!

You can, literally, go bankrupt increasing your Conversion Rate.

(Hence, at the very minimum, pair up Conversion Rate with Average Order Value to get an initial whiff of doom.)

Conversion Rate is not a Most Important KPI.

Return on Ad Spend (ROAS) is an ok metric.

It is typically computed by dividing the Revenue from Advertising by the Cost of Advertising (a.k.a. Media Costs). You times that by 100, and you get a ROAS %.

ROAS only sucks less. It remains very, very, very far away from the green. Additionally, by aggregating products/services into lumpy groups, it can give a misleading sense of success.

[Disclosure: I profoundly dislike ROAS — even hate it — for, among other reasons, driving a disproportionate amount of obsession with ONLY Paid Media by CMOs when Paid Media typically delivers a minority of the incremental business revenue. Bonus Read: Attribution is not incrementality.]

Gross Profit is revenue minus Media Costs minus CoGS.

Now, you have yourself a KPI! Not yet the Most Important KPI, but a KPI nonetheless.

In the past, I’ve recommended using Custom Metrics in tools like Google Analytics to compute Gross Profit. You can do this using an aggregate % number that you can lop off for CoGS. At the very minimum, your Traffic Sources report does not have to stop at Revenue (misleading much?).

With Google’s Data Studio, you can actually bring item level CoGS in and easily compute Gross Profit for every single order you get.

It. Will. Change. Your. Life.

Net Profit then is revenue minus Media Costs minus CoGS minus FLCo.

Finally, you have something super cool.

You can work with your Finance team to get FLCo. You’ll get a different number for your Owned, Earned, Paid media strategies. You’ll have a number that’ll accommodate for a sale that might have happened on your website vs. retail store vs. placed on website but picked up on retail, etc.

You can build this into Google’s Data Studio if you like. Or, the Business Intelligence tool of choice used by your company.

Net Profit totally qualifies for the Most Important KPI tag.

It helps identify how much money you created that is going into the bank, and what it is that you did exactly to create that money.

Yep. Understanding that will deliver a transformative impact on your business.

I’ll go out on a limb and say that it will also shock your CMO.

The Money In-Out Continuum | The Problem.

I say this with some confidence that none of your reports for digital, and barely any reports for the entire business, currently report on either one of the above two Most Important KPIs.


Simple. You are using Adobe Analytics or Google Analytics or some such tool, and they have no built-in concept of 1. Media Costs 2. CoGS, and 3. FLCo.

Sure, if you connect Google Analytics to your Google Ads account, #1 becomes easy. You have Media Costs. But, in addition to Google, you are advertising on a ton of other channels and getting all those costs is a pain – even when possible.

Obviously, digital analytics tools have no concept of #2 (CoGS) or #3 (FLCo).

You are stuck making poor business decisions, in the best case scenario, at stage two of the Stages of Savvy.

This is not enough.

The Money Making Continuum | Stages of Savvy

To build a strategy to address this gap in your analytics strategy…  My recommendation is to break out of the limitations that your digital analytics tools, and shift to your business intelligence tools (start with exhausting the features Data Studio provides you with for the magnificent cost of zero dollars – lower FLCo!).

Recognize that Analysis Ninjas live at stage 3, and they truly come into their own when they get to stage 4.

Is this true for you? Does your analytical output include Net Profit?

By a staggering coincidence, Analysis Ninjas who live in stages 3 and 4 also have long, productive, well-compensated careers! Because getting there is hard, AND it requires building out a wide array of cross-functional relationships (always crucial when it comes to annual performance reviews!).


The Money In-Out Continuum | The Most Important KPI.

Obviously, the most important KPI is the one you are not measuring.

Customer Lifetime Value (CLV) is the sum of Net Profit earned from a customer over the duration they are your customer.

Say I buy the Pixel 1 phone from Google, and Google makes $50 Net Profit from that sale.

Then, I buy the Pixel 2, Pixel 3a, and Pixel 4a. Google makes $60, $60, and $60 Net Profit (they save on advertising costs to me, which translates into higher profit).

Then, for reasons related to innovativeness, I switch to Samsung and buy a Z Flip 3 (great phone!).

The Money Making Continuum | Customer Lifetime Value

My CLV for Google is: 50+60+60+60 = $230.

I originally converted to buying a Pixel 1 after typing best android phone into Bing.

Analytics tools, configured right, with analysis done by Stage 4 Analysts, will show a Net Profit of $50 driven by Bing.

Except, it is $230.

Cool, right?

So. Why don’t we all calculate CLV every day and every night, and then some more of it on the weekend?

Because it is hard.

Go all the way back up and reflect on why is it that we are satisfied with Conversion Rate or ROAS vs. Gross Profit?

Because it is easy.

It is so hard to get to Gross and Net Profit.

Then, to be able to keep track of that same person (me, in the above Pixel example). Then, wait for me to churn so that you get my CLV. Oh, and remember to have systems interconnected enough to keep track of every touchpoint with me to ensure you attribute accurately.

It is hard.

Of course, you don’t have to do the computation for every individual. You can do it by micro-segments (like type people, same geo, age groups, products, etc. etc.). You can do it in aggregate.

Sadly, none of these is easy.

Hence. You don’t do it.

No matter what CLV zealots will tell you.

If they make you feel bad. Don’t feel bad.

My advice is twofold:

1. Keep your primary quest to get to stage 4 (Net Profit) because the quality of your insights will improve by 10x.2. (If you don’t have it already) Create a long-term plan to understand the lifetime value of a customer for your company.

Execute that advice in that order, and you'll get to the global maxima faster.

As you contemplate your strategy for #2 above, my dear friend David Hughes helped write one of my favorite posts on this blog: Excellent Analytics Tip #17: Calculate Customer Lifetime Value.

Read it. Internalize the recommendations. Download the detailed lifetime value model included in the post, and jumpstart your journey.


Bottom Line.

It is unlikely that any of you reading this blog on advanced analytics is measuring a head fake metric. You realize the futility already.

I also believe that you and I can do more to move beyond stage 1 and stage 2 of the stages of savvy. And, I hope I’ve encouraged you to do that today. It is so worth it.

I believe almost all of us can do more to be on a CLV journey — but not at the cost of losing focus in stages 3 and stage 4.

Let’s get to it!

As always, it is your turn now.

Via comments, please share your critique, reflections, tips and your KPI lessons from the front lines of trying to drive material business impact. What do you disagree with above? What has been the hardest nut for you to crack in your career?

The Premium edition of my weekly newsletter shares solutions to your real-world challenges while unpacking future-looking opportunities. Subscribe here.


The post The Most Important Business KPIs. (Spoiler: Not Conversion Rate!) appeared first on Occam's Razor by Avinash Kaushik.

Increase Analytics Influence: Leverage Predictive Metrics!

Almost all metrics you currently use have one common thread: They are almost all backward-looking. If you want to deepen the influence of data in your organization – and your personal influence – 30% of your analytics efforts should be centered around the use of forward-looking metrics. Predictive metrics! But first, let’s take a small […]

The post Increase Analytics Influence: Leverage Predictive Metrics! appeared first on Occam’s Razor by Avinash Kaushik.

Almost all metrics you currently use have one common thread: They are almost all backward-looking.

If you want to deepen the influence of data in your organization – and your personal influence – 30% of your analytics efforts should be centered around the use of forward-looking metrics.

Predictive metrics!

But first, let's take a small step back. What is a metric?

Here’s the definition of a metric from my first book:

A metric is a number.

Simple enough.

Conversion Rate. Number of Users. Bounce Rate. All metrics.

[Note: Bounce Rate has been banished from Google Analytics 4 and replaced with a compound metric  called Engaged Sessionsthe number of sessions that lasted 10 seconds or longer, or had 1 or more conversion events or 2 or more page views.]

The three metrics above are backward-looking. They are telling us what happened in the past. You'll recognize now that that is true for almost everything you are reporting (if not everything).

But, who does not want to see the future?

Yes. I see your hand up.

The problem is that the future is hard to predict. What’s the quote… No one went broke predicting the past. :)

Why use Predictive Metrics?

As Analysts, we convert data into insights every day. Awesome. Only some of those insights get transformed into action – for any number of reasons (your influence, quality of insights, incomplete stories, etc. etc.). Sad face.

One of the most effective ways of ensuring your insights will be converted into high-impact business actions is to predict the future.

Consider this insight derived from data:

The Conversion Rate from our Email campaigns is 4.5%, 2x of Google Search.

Now consider this one:

The Conversion Rate from our Email campaign is 4.5%, 2x of Google Search.

Our analysis suggests you can move from six email campaigns per year to nine email campaigns per year.

Finally consider this one:

The Conversion Rate from our Email campaign is 4.5%, 2x of Google Search.

Our analysis suggests you can move from six email campaigns per year to nine email campaigns per year.

We predict it will lead to an additional $3 mil in incremental revenue.

The predicted metric is New Incremental Revenue. Not just that, you used sophisticated math to identify how much of the predicted Revenue will be incremental.

Which of these three scenarios ensures that your insight will be actioned?

Yep. The one with the Predictive Metric.

Becaues it is hard, really hard, to ignore your advice when you come bearing $3 mil in incremental revenue!

Starting your Predictive Metrics journey: Easy Peasy Lemon Squeezy.

In a delightfully wonderful development, every analytics tool worth its salt is adding Predictive Metrics to its arsenal. Both as a way to differentiate themselves with their own take on this capability, and to bring something incredibly valuable to businesses of all types/sizes.

In Google Analytics, an early predicted metric was: Conversion Probability.

Simply put, Conversion Probability determines a User’s likelihood to convert during the next 30 days!

I was so excited when it first came out.

Google Analtyics in this instance is analyzing all first-party data for everyone, identifying patterns of behavior that lead to conversions, now looking at everyone who did not convert, and on your behalf giving a score of 0 (no chance of conversion) to 100 (very high chance of conversion).

Phew! That’s a lot of work. :)

What’s particularly exciting is that Conversion Probability is computed for individual Users.

You can access the report easily in GA: Audience > Behavior > Conversion Probability.


An obvious use of this predicted behavior is to do a remarketing campaign focusing on people who might need a nudge to convert, 7,233 in the above case.

But, there are additional uses of this data in order to identify the effectiveness of your campaigns.

For example, here is the source of traffic sorted by Average Conversion Probability


In addition to understanding Conversion Rate (last column) you can now also consider how many Users arrived via that channel who are likely to convert over the next 30 days.

Perhaps more delightfully you can use this for segmentation. Example: Create a segment for Conversion Probability > 50%, apply it to your fav reports like the content ones.

There is so much more you can explore.

[TMAI Premium subscribers, to ensure you are knocking it out of the park, be sure to review the A, B, O clusters of actionable recommendations in #238: The OG of Analytics – Segmentation! If you can’t find it, just emial me.]

Bonus Tip: I cannot recommend enough that you get access to the Google Merchandise Store Google Analytics account. It is a fully working, well-implemented real GA data for an actual business. Access is free. So great for learning. The screenshot above is from that account.

Threee Awesome New Predictive Metrics!

With everything turning over for the exciting world of Google Analytics 4 you get a bit more to add to your predictive metrics arsenal.

Conversion probability is being EOLed with GA 4, but worry not as you get a like-type replacement: Purchase Probability

The probability that a user who was active in the last 28 days will log a specific conversion event within the next 7 days.

Currently, purchase/ecommerce_purchase and  in_app_purchase events are supported.

You can do all of the same things as we discussed above for Conversion probability.

To help you get closer to your Finance team – you really need to be BFFs with them! – you also get a predictive metric that they will love: Revenue Prediction

The revenue expected from all purchase conversions within the next 28 days from a user who was active in the last 28 days.

You can let your imagination roam wild as to what you can do with this power.

Might I suggest you start by looking at this prediction and then brainstorm with your Marketing team how you can overcome the shortfall in revenue! Not just using Paid strategies, but Earned and Owned as well.

Obviously in the rare case the Revenue Prediction is higher than target, you all can cash in your vacation days and visit Cancun. (Wait. Skip Cancun. That brand’s tainted. :)

There’s one more predicted metric that I’ve always been excited about: Churn Probability.

The probability that a user who was active on your app or site within the last 7 days will not be active within the next 7 days.

What’s that quote? It costs 5000x more to acquire a new User than to retain the one you already have? I might be exaggerating a tad bit.

For mobile app/game developers in particular (or for content sites, or any entity for whom recency/frequency is a do or die proposition). Churn is a constant obsession and now you can proactively get churn probability. Make it a core part of your analytical strategy to understand Behavior, Sources, Users, who are more/less likely to churn and action the insights.

GA 4 does not simply hand you these metrics willy-nilly. The algorithms require  a certain number of Users, Conversions etc., in order to ensure they are doing sound computations on your behalf.

These three predictive metrics illustrate the power that forward-looking computations hold for you. There are no limits to how far you can take these approaches to help your company not only look backwards (you’ll be stuck with this 70% of the time) but also take a peek into the future (aim to spend 30% of your time here).

And please consider segmenting Purchase Probability, Revenue Probability and Churn Probability!

Bonus Tip: If you would like to migrate to the free version of Google Analytics 4 to take advantage of the above delicious predictive metrics, here’s a helpful article.

Predictive Metrics Nirvana – An Example.

For a Marketing Analyst, few things come close to nirvana in terms of forward-looking predictions from sophisticated analysis than to help set the entire budget for the year including allocation of that budget across channels based on diminishing returns curves and future opportunity and predict: Sales, Cost Per Sale, and Brand Lift.

Here’s how that looks from our team’s analytics practice…


Obviously, all these cells have numbers in them. You’ll understand that sharing them with you would be a career-limiting move on my part. :)

I can say that there are thirteen different element sets that go into this analysis (product launches, competitor behavior, past analysis of effectiveness and efficiency, underlying marketing media plan, upcoming industry changes, and a lot, lot, lot, of data).

Supercool – aka superhard – elements include being able to tie Brand Marketing to short, medium, long-term Sales.

Forward-looking allocations are based on simulations that can take all of the above, to answer low, medium, high-risk plans – from which our senior leader gets to choose the one she believes aligns with her strategic vision.

[Note: Strictly speaking what we are doing above is closer to Predictive Modeling, even though we have a bunch of Predictive Metrics. Potato – Potahto.]

I share our work as a way to invite your feedback on what we can do better and in the hope that if you are starting your Predicted Metrics practice, that it might serve as a north star.

From experience, I can tell you that if you ever felt you as an Analyst don’t have influence, that your organization ignores data, then there is nothing like Predicted Metrics to deepen your influence and impact on the business.

When people use faith to decide future strategy, the one thing they are missing is any semblance of what impact their faith-based strategy will have. The last three rows above are how you stand out.


The Danger in Predicting the Future.

You are going to be wrong.

A lot, initially. Then less over time as you get better and better and predicting the future.

(Machine Learning comes in handy there as it can ingest so much more complexity and spit out scenarios we simply can’t imagine.)

But, you will never be exactly right. The world is complicated.

This does not scare me for two reasons, I urge you to consider them:

1. Very few companies drove straight looking out of the rear view mirror. But, that is exactly what you spend time trying to do every single day.

2. Who is righter than you? The modern corporation mostly runs of faith. You are going to use data, usually a boat-load of it. It is usually far better than faith. And, when you are wrong, you can factually go back and update your models (faith usually is not open to being upgraded).

So. Don’t be scared.

Every time you are wrong, it is an opportunity to learn and be more right in the future – even if perfection will always be out of reach.

Bottom Line.

My hypothesis is that you are not spending a lot of time on predictive metrics and predictive modeling. Change this.

It is a great way to contribute materially to your company. It is a great way to invest in your personal learning and growth. It is a fantastic way to ensure your career is future-proof.

Live in the future – at least some of the time – as an Analyst/Marketer.

I’ll see you there. :)

As always, it is your turn now.

Please share your critique, reflections, tips and your lessons from projects that shift your company from only backwards looking metrics to foward looking metrics that predict the future.

The post Increase Analytics Influence: Leverage Predictive Metrics! appeared first on Occam's Razor by Avinash Kaushik.