Not running experiments? You’re falling behind.

Back in 1975, Kodak had just developed a digital camera—the first of its kind. But the product was dropped for…Read blog postabout:Not running experiments? You’re falling behind.
The post Not running experiments? You’re falling behind. ap…

Back in 1975, Kodak had just developed a digital camera—the first of its kind. But the product was dropped for...Read blog postabout:Not running experiments? You’re falling behind.

The post Not running experiments? You’re falling behind. appeared first on WiderFunnel Conversion Optimization.

Responses to Negative Data: Four Senior Leadership Archetypes.

Not everything a company does works out. (That is different from everything that a company is doing not working out. :)) If you are in the data business – my bread, butter and tofu – you often carry the burden of being the bearer of bad news. The conversion rate is down 30% at launch. […]

The post Responses to Negative Data: Four Senior Leadership Archetypes. appeared first on Occam’s Razor by Avinash Kaushik.

Not everything a company does works out.

(That is different from everything that a company is doing not working out. :))

If you are in the data business – my bread, butter and tofu – you often carry the burden of being the bearer of bad news.

The conversion rate is down 30% at launch.

The goal was to deliver a 30% increase in revenue, the team delivered 1.7953%.

During 2019, our Net Promotion Score has dropped 15 points.

The average length of our video ads is 30 seconds, less than 10% of the audiences watches beyond 5 seconds and 90% is exposed to less than 1 second.

Our Market Share in the 2-ton truck market shrunk by 1.5% (= -$3 bil).

Negative data.

Accurately collected. Intelligently analyzed. Factually presented.

Sadly still, negative data to the person/team receiving it.

Why be hurtin’?

A decade ago, data people delivered a lot less bad news because so little could be measured with any degree of confidence.

In 2019, we can measure the crap out of so much. Even with the limitations of tools, government regulations, and the astonishing fragmentation of everything (attention, devices, consumption sources, identities and more).

Companies have also evolved to be significantly more complex beings, who have to do so much more than what they did 50 years ago. Think of all P&G had to do to sell soap 50 years ago, think of everything they are doing now. Add on top of that, where P&G could sell soap, the purposes it could sell soap for, and imagine both of those things now. It is a lot of stuff!

When you do that much stuff, and you can measure almost everything… The result is that our ecosystem of data people are returning a lot more negative data when measuring performance of Marketing, Sales, User Experience and Customer Service.

Let me repeat this one more time: It is not that companies have slowly over the last decade started to suck more (well, maybe some). It is that we are able to analyze and identify bad performance with greater accuracy.

While that change has occurred, two things have obstinately stayed the same:

1. Company cultures are rarely open to hearing anything negative.

2. The top leaders in your company grew up, succeeded, and were promoted during the era of no data (and hence a ton less negative news). They are not natively wired to receive data-delivered reality checks.

The combination means no red carpet for negative data. It is not hard to see that a modern large corporation is likely missing out on the benefits of all they should know about their business through data. It also results in a depressing existence for data people.

Short-term, this let’s not listen to the negative data strategy sometimes actually works. No one is telling the Emperor he is naked, and the Emperor is delighted everyone loves his clothes so much.

Long-term…. : (

Four Negative Data Leadership Archetypes.

The solutions to this big opportunity have many dimensions.

I want to focus on the massive “make or break” dimension: #2 above, with Extremely Senior Leaders (ESLs).

Through their words and actions, ESLs can quash data's learn to improve spirit, or they can nurture that spirit and deliver a transformative, positive impact on the company's culture + profit.

In my work with clients around the world (all continents except Antarctica), I’ve encountered a whole host of reactions to when I have shared negative data with ESLs. From the patterns in those reactions, I’ve developed four archetypes of leaders.

While people are never starkly black and white, they typically have a dominant archetype – the one they most frequently demonstrate.

I’ve also observed cultural implications that each leader-type ends up creating from their reactions to negative data. For three of the four, I’ve seen individuals successfully navigate the leader-type – never without scars though – and individuals emotionally burn-out due the environment the leader creates.

Today, I want share the four leader archetypes with negative data as the lens. Included below is my accumulated wisdom, with the hope that you’ll fall in navigate category and not the crushed one.

Ready?

Archetype #1: The Bubble King (/Queen).

Archetype #1: How they react:

This individual lives in a bubble, so their reaction to any data is… Nothing. Data, unless it is super-positive, never makes it to them. Bubble Kings are comfortable making decisions that sound good – decisions just as likely to be informed by their long experience as the quest for shiny objects.

Bubble Kings most commonly reside in organizations where there is little to no accountability (or misplaced accountability, ex: celebration of vanity metrics).

Their most common reaction to negative data, if it makes it through, is to try to discredit it by asking analytically-nonsensical questions: What are the p-values of your multi-channel attribution model applied to performance of my strategy?

Archetype #1: Cultural implications:

In small or medium sized companies, Bubble Kings (/Queens) have short reigns. Not all that hard to imagine why – you don’t listen to data, ignore reality checks, and the transparently oriented accountability loop ensures there is nowhere to hide.

In large companies, or teams with massive budgets, Bubble Kings (/Queens) have long reigns. The accountability loops are larger, less transparent, and the natural large-company multi-layer organization complexity does not help. Typically a change in the C-Suite layer above will transition them out of the company (fresh eyes, sunshine … call it what you will).

However, while they still reign – since feelings matter more than data – sycophantic behavior is common and often encouraged.

Archetype #1: How to deal with them, their org:

Data will never play any impactful role on strategy. Since Bubble Kings live in a, well, bubble, you can often form relationships and influence at lower levels in a Bubble King’s org, and you can have a positive influence on tactics. Absolutely take advantage of it.

If you want to get promoted, give up the quest to identify factual real-world performance and focus instead on proving that the Bubble King's decisions deliver excellent results. Don't compromise on your ethics. But on this blog and in my newsletter I’ve shared enough strategies you should not use to slant data – use them.

[Bonus Reads: A Great Analyst's Best Friends: Skepticism & Wisdom! & TMAI #154: Irrationality, Cognitive Bias, and Us.]

Archetype #2: The Attacker.

Archetype #2: How they react:

They attack.

They attack the data. They attack your knowledge. They attack your intent. They bring up that one time in 2013 when your analysis missed an important assumption. They attack your personal attributes.

In the face of factual negative data related to their decisions, they will counter-attack. At times, harshly. Sometimes they counter-attack, in a twist of irony, by trying to drown you in enormous detail and minutiae.

You will be branded Ms. Bad News or Mr. Not A Team Player or some such ugly moniker.

Archetype #2: Cultural implications:

In extremely senior positions, Attackers fuel the creation of a culture where no bad news ever filters through. When business performance is non-positive, every employee, at every level, will work super-duper hard to look at every dimension of data to find any semblance of good news (no matter how small). Only this good news will make it to the top (Attacker ESL).

A typical example: The entire house is on fire but the analysis of that situation will focus on the one unsinged rose in the font lawn and how beautiful the rose is.

Attackers lead can last for a surprisingly long time in an organization, for a whole host of strategic reasons (as I’m confident you’ve observed as well).

Archetype #2: How to deal with them, their org:

If you are a data person and you are in a small organization lead by an Attacker, you need to update your resume and find a way out. There is no hope for your career (or emotional positivity).

If you are a data person and you are in a large organization lead by an Attacker, also update your resume. If, for any number of valid reasons, you are stuck there my advice is to focus your analytical efforts exclusively on the Attacker's biggest fears. It might take a little bit of effort to discover them, but it is so worth it. Even an Attacker has a point at which their instinct for self-preservation kicks in, in those rare (often hidden) situations they’ll be open to negative data.

And this is key: If you can provide solutions and not just data, you might even become a trusted adviser. This will do nothing to advance your acquired negative branding of Ms. Bad News, nor will this change the broader team/company culture… But…You'll have an impact with data, providing a pretty decent existence in an Attacker created culture while you look for a way out.

Archetype #3: The Rationalizer.

Archetype #3: How they react:

Their trigger instinct in face of factual negative data is to make excuses. To provide context. To identify circumstances to blame. To poke holes in the data/methodology (regardless of the Rationalizer’s analytical competence). To create enough uncertainty to fuzzy up any negative – or remotely negative – data.

If we were stack rank the four types, the Rationalizer would come on top as the most undesirable leader (often corrosive for the institution).

You might think it would be the Attacker. It is the Rationalizer because their approach to dealing with negative data is not as overtly corrosive. A Rationalizer subtly sows doubt. They dilute the analysis with non-facts. They force the inclusion of non-related nonsense in the quest to paint a fuller picture. At their worst, they commonly turn diamonds into coal.

Archetype #3: Cultural implications:

Everything data people do to highlight reality, to bring truth to the fore, to identify positive solutions from negative data, will be discounted, buried, and compete for impact with faith. Questionable analysis and slanted views will have equal footing with the most factual and intelligent analysis.

When people say "this team’s culture runs on BS," they are describing an organizations run by a Rationalizer.

Status quo will rule the day in such organizations, unless there is a big external force that creates change. The operative instinct is to maintain mediocrity with just enough reality massaging flowing upstream to ensure existing mediocrity is not utterly obvious.

One identifying attribute of Rationalizer organizations is the overwhelming abundance of data pukes. Why? Data pukes do nothing to make an organization intelligent, while providing the feeling of competence and productive output.

Archetype #3: How to deal with them, their org:

Rationalizer org’s are hardest to deal with because you are not obviously being ignored (Bubble King) nor are you being openly challenged (Attacker). You are just constantly being undercut to the point where the data represents a watered down version of an adjacent reality.

If you are a data person full of courage and determination, find the largest element of the business' strategy and unpack the power of strategic analysis to present factual data. Lead with as many things as you can find that are going right, then follow that with the most material two things that are factually not going right. Present the collection directly to the Rationalizer if you can.

On that note… Since the Rationalizer is an ESL (Extremely Senior Leader), it is quite possible that you have to work through many layers of people in-between you, my peer data person, and them. In a Rationalizer’s culture, every layer you go through will instinctively take the material two negative news and will try to kill it or fuzzify/massage it. In these cases, if you can make cosmetic changes to pass each layer, do so. Don't give up on the core of the positive and negative stories.

When you are in the presence of a Rationalizer, bring overwhelming analytical competence – there is no better way to deal with their reactions (see above). A Rationalizer never gives up trying to rationalize every small bit of negative data, persistence is a virtue that’ll come in handy.

Keep in close contact with your soul. At some point you’ll find it is sapping, it’ll be your clue that you need to find it a different professional environment.

Archetype #4: The Curious One.

Archetype #4: How they react:

In face of negative data, the Curious One asks you questions to understand the why behind what you are presenting.

If a period has elapsed where the data person has demonstrated competence, the Curious One does not question the analytical approach of data collection methodologies (they trust you to have applied fanatical quality control). The Curious One demonstrates, well, curiosity about what biases might be in the data or what assumptions you might have made.

They have two critical attributes: 1. They demonstrate open mindedness in the face of negative data. 2. Their posture is not to instinctively blame (backwards looking), but rather the posture is to identify and fix (forwards looking).

Archetype #4: Cultural implications:

Due to the demonstrated behavior at the top, open mindedness is usually encouraged in organizations led by Curious Ones.

Negative data is never a delightful experience, but the trust fostered amongst senior leaders results in a lot more truth telling, and is as good of a welcome mat as will ever be provided to the data people.

It might seem odd that in such a positive posture to negative data that there is still accountability, but incredibly in my experience it exists in spades in such organizations. It flows down from the clear measurable goals, an empowered data organization, and a close and direct partnership with different leadership levels (VPs, Directors, Sr. Managers).

Archetype #4: How to deal with them, their org:

Pinch yourself every day.

Don't take your position for granted. Invest in self-learning every week – even couple hours a week – to ensure you can keep pace with the demands for sophisticated analysis which will be expected at an agile pace.

You know my Care-Do-Impact model for analysis and storytelling. Organizations led by the Curious Ones are the very best places for you to slowly migrate your sophistication in Do and Impact. This, in turn, means that your demonstrated sophistication will open up new career options, for example becoming a business line leader or moving on to the strategy side of the house. Joyous outcomes for you, your company, and your company's customers.

Two Inspiring Examples | Curious One Archetype.

Paul Polman.

One person who demonstrated Curious One behavior to me was the recently ex-Global CEO of Unilever, Paul Polman. I had an opportunity to spend time with him and his leadership team. My role was to be a challenger, to share stories about what Unilever did well and focus on the challenges faced by vividly demonstrating things they did not do well. The latter part of the story qualifies as negative data.

It would be normal CEO behavior to be defensive, to pick the story apart, to make excuses. But, no. Through his words and actions to me and his brand CEOs, Mr. Polman demonstrated every attribute of the Curious One. This opened mind share required to re-imagine the future.

As with inspiring leaders, there is a whole ton lot more as to why I admire Mr. Polman.

Alan Mulally.

Mr. Mulally’s stewardship of Boeing and Ford is legendary. I want to share one story that Mr. Mulally shared with us in a meeting (and in his book).

When he got to Ford he instilled the same colored charts approach to identify what's working and what needs more attention. All the charts Mr. Mulally got were color coded green (#everythingisawesome). The problem was that Ford was on track to lose $17 billion dollars that year. Ford's culture was such that business leaders would hide problems, therefore making issues "disappear." Mr. Mulally set a different tone of honesty and looking at negative data as an opportunity to improve/change/fix (classic Curious One approach). In next meetings, things slowly started to turn Red… and Yellow… and some real Green.

There is a lot more to Mr. Mulally's turnaround of Ford. Eleven principles actually (buy the book!). But in this anecdote you can see the central reason that I adore him, and the leadership skills that can turn even the most intractable business problems in some of the largest companies on the planet.

A Plan for Action.

Humans are complicated beings.

No individual is just one black and white type.

Yet, humans, at least professionally, tend to demonstrate a dominant type. It is what they are natively comfortable with.

With that in mind, a suggestion for a plan of action…

Leader.

Introspective is in order. Assume you are doing this only for your own selfish benefit reasons, no one else has to know.*

Take a quiet moment.

Reflect on what your dominant type is: Bubble King or Attacker or Rationalizer or Curious one.

Once you do that, consider the impact that your leadership posture is having on your team, on your data people, on the ability of negative data (or negative anything) to help stop/rethink strategy, and indeed on the corporation.

The global maxima is that you consider a personal shift towards exploring the benefits of evolving to become the Curious One type (if you are not there already).

An incentive is that at some point in a long career, one does tend to reflect on the personal impact of one's professional accomplishments. In that moment, on that day/week/month/rest of your life, realizing the heart-breaking impact you delivered by being a Bubble King, Attacker or Rationalizer does deliver a heavy emotional burden and a personal crisis. So. Not. Worth it.

There is a meme that people can't really change who they are. You'd be surprised how untrue that is.

* Your team already knows what type you are. You might as well be honest with yourself for all the benefits that will come.

Individual.

Self-reflection is in order for you as well.

Set some quiet time aside so that you can consider the how they react and cultural implications demonstrated by the leader who has the biggest influence your personal work. (Sometimes this is your direct manager. Other times it is someone a few levels above yours.)

In your mind only, assign the archetype (BK, A, R, CO) to the leader. That act will bring clarity as you ask yourself these three questions I recommend…

What is your behavior in response to that dominant leader?

Is it as suggested in the how to deal with it section of each leader type?

What will it take for you to change your behavior to optimally deal with the situation you are in?

Make a specific plan.

Act on it. Life is short.

It is always better to be on a path chosen after careful self-reflection and planning, even if you find yourself in an undesirable situation. It might not deliver world peace, but it will reduce your emotional burden.

Bottom-line.

We've used leader reaction to negative data as a vehicle to discuss creating an optimal professional path for ourselves (as leaders or as individuals).

The framework I've shared, how they react, cultural implication, and how to deal with it, can be applied to multiple dimensions of our professional ecosystem. Give it a try.

If you are a leader, if you have a choice, be the Curious One. Here are a recap of the benefits: 1. Lighter personal emotional burden. 2. Cultures where the goal is not blame, it is making progress. 3. People who love you (yes, love, in a workplace!) and will help you deliver transformative results – in good times and bad.

So. Worth. It.

Good luck!

As always, it is your turn now.

What archetype identifies the most influential leader in your organization? If you are a leader, what archetype is reflective of your impact? If you’ve successfully worked inside organizations lead by a Bubble King (/Queen), Attacker, Rationalizer… What worked, what did not work? Have your seen a leader transform into a different archetype – do to an HR-induced or personal induced change? What worked, what did not work? Is there an archetype you would have created, if we are looking through the lens of negative data?

Please share your reflections, critique, culture-shifting strategies, and tips for individuals or leaders via comments.

Thank you.

The post Responses to Negative Data: Four Senior Leadership Archetypes. appeared first on Occam's Razor by Avinash Kaushik.

New Features in Illuminate: Impact Analysis, Enhanced Filters, Updated Dashboard & More

Since we launched Illuminate back in May, our team has been working around the clock to develop even more features to help optimization teams better organize experiments, report performance and maximize impact. Today, we’re excited to share a few of these with you. What’s new in Illuminate? Show impact and determine priority Use our new Impact […]

The post New Features in Illuminate: Impact Analysis, Enhanced Filters, Updated Dashboard & More appeared first on Brooks Bell.

Since we launched Illuminate back in May, our team has been working around the clock to develop even more features to help optimization teams better organize experiments, report performance and maximize impact. Today, we’re excited to share a few of these with you.

What’s new in Illuminate?

Show impact and determine priority

Use our new Impact Analysis to show the overall impact of your tests by page type and identify where you should be focusing your testing efforts.

Sort and filter by what matters most

Filter your tests by 15 attributes including target audience, page type, start and end date, KPIs, revenue impact and more. Not seeing what you need? Add your own using our new custom tagging feature.

Keep sight of the bigger picture

Our new dashboard view enables you to view your program’s overall performance or view performance by a specific team or line of business.

+ a new tiled layout

If you love a good masonry layout (á la Pinterest), then you’re going to love our updated experiment view. Easily switch between a basic list of your experiments or a super slick-looking tiled layout.

Many of these features were developed in response to feedback from our beta users, bringing more of Brooks Bell’s advanced experimentation methodologies directly into the software.

“With Illuminate, you’re not just getting another test repository,” said Suzi Tripp, Senior Director of Innovative Solutions at Brooks Bell. “You’re getting 15 years of experimentation expertise and proven frameworks to help you do more, and do it better.”

Interested in learning more about illuminate? Learn more on our website or schedule a demo using the form below.

The post New Features in Illuminate: Impact Analysis, Enhanced Filters, Updated Dashboard & More appeared first on Brooks Bell.

Video Series: Conquer Your Biggest Testing Challenges

Here at Brooks Bell, we work with clients that are at varying stages of maturity when it comes to experimentation. Despite the differences in these partnerships, you might be surprised to learn that regardless of whether we’re working with a new or established testing program, they all face common enemies: pressure to deliver results; inefficient […]

The post Video Series: Conquer Your Biggest Testing Challenges appeared first on Brooks Bell.

Here at Brooks Bell, we work with clients that are at varying stages of maturity when it comes to experimentation. Despite the differences in these partnerships, you might be surprised to learn that regardless of whether we’re working with a new or established testing program, they all face common enemies: pressure to deliver results; inefficient processes; a lack of understanding and support for testing; and difficulty iterating on and applying learnings from test results.

In this four-part video series, you’ll hear from Suzi Tripp, our Sr. Director of Innovative Solutions, Jonathan Hildebrand, Sr. Director of Design & UX, and Claire Schmitt, VP of Strategic Consulting and Solutions at Brooks Bell. They’ll discuss tips and tricks for addressing these challenges. You’ll also get insight into best practices for organizing your testing program, developing smarter tests, showcasing your results and obtaining insights about your customers.

Check out the first video below, or watch the full series by filling out the form at the bottom of this post.

Part 1: Storing and Learning from Past Tests

Fill out the form below to view the other three videos, covering:

  • Collaborative Ideation / Strategizing Better Tests
  • Communicating Testing Insights Up The Ladder
  • Retaining and Growing Testing Budget

The post Video Series: Conquer Your Biggest Testing Challenges appeared first on Brooks Bell.