Unlocking the True Power of Testing & Other Takeaways from Brooks Bell’s Interview With Ambition Data

Recently, our Founder and CEO, Brooks Bell, sat down with Allison Hartsoe, host of the Customer Equity Accelerator—a podcast produced by Ambition Data. Listen to the full podcast or read on for a few highlights from their conversation:  On what inspired her to build an experimentation consultancy… Originally, Brooks founded Brooks Bell Inc. in 2003 as […]

The post Unlocking the True Power of Testing & Other Takeaways from Brooks Bell’s Interview With Ambition Data appeared first on Brooks Bell.

Recently, our Founder and CEO, Brooks Bell, sat down with Allison Hartsoe, host of the Customer Equity Accelerator—a podcast produced by Ambition Data. Listen to the full podcast or read on for a few highlights from their conversation:

On what inspired her to build an experimentation consultancy…

Originally, Brooks founded Brooks Bell Inc. in 2003 as a website development agency. After working with a few local clients, a chance introduction led to her first major experimentation client, AOL.

Today, you might think of AOL as one of the [now-extinct] internet dinosaurs, but even back in the early 2000s, the media giant was facing its fair share of challenges. According to one story by Time Magazine, despite having 34 million members in 2002, AOL was battling slowing subscriber growth, falling ad revenue and exorbitant operational costs. 

So, the company turned to experimentation. “AOL had the right environment to build a testing culture,” said Brooks. “They had a closed technology environment, their own analytics platform, and their data was clean and connected.”

Back then, AOL relied on pop-ups to drive new subscriptions. Working with Brooks, the company issued a challenge: design a new subscription pop-up that would beat the control experience. And so, drawing from her background in design and psychology, she did—and then she did it again, and again, and again.

But that was just the start. As other large companies began to rely more on the digital space to drive their business, Brooks saw an opportunity to help them tap into the power of experimentation.

“We realized that no one was testing!” said Brooks. “No other large companies had the data, culture and processes in place to test. So we set out to help them build the data fidelity and really recreate what we saw at AOL in those early years.”

On the difference between optimization and experimentation…

It’s one of the more common questions we get: “Brooks Bell is an experimentation consultancy. What’s that? What’s the difference between experimentation and optimization?” As Brooks explains it, it all comes down to science.

By definition, experimentation is the application of the scientific method to determine something. And while optimization is one potential outcome of an experiment, true experimentation requires running tests without a prescriptive outcome or application.

To put it simply – you’re testing to learn. And as long as your results are statistically significant, there is always something to be learned from experiments—even those with flat or negative results.

On how to unlock the real power of experimentation…

Today, in the age of Amazon, a customer-centric experience is critical. But for some established companies, this requires a bigger paradigm shift in culture and processes.  

“Customer-centricity requires rethinking metrics, the type of data you collect, how teams are organized, how teams are incentivized, how you communicate and also your core values,” said Brooks.

The true power of experimentation lies in its ability to align your customer needs with your company’s strategic goals and your program’s agenda. Furthermore, you can use experimentation to learn new things about your customers in a scientific way.

“Having statistically-sound customer insights can totally change how you organize your store, how you train your team, and how you structure your website,” said Brooks. “This is where testing programs can really drive change.”

To that end, we recently celebrated the launch of Illuminate, our customer insights software for testing teams and executives. Illuminate not only provides a place to store, share and learn from your experiments, but also a means to develop impactful customer insights.

“We launched Illuminate to provide a repository of great test examples, to learn from each other, and to build a library of great test case studies,” said Brooks.  This is because outside of the testing program, any key learnings from an experiment can get lost within the data. Illuminate solves this by encouraging deeper thinking about customers, their needs, preferences, and behaviors. 

Learn more about Brooks Bell’s experimentation consulting services. 

The post Unlocking the True Power of Testing & Other Takeaways from Brooks Bell’s Interview With Ambition Data appeared first on Brooks Bell.

How to hire your testing unicorn (without using magic)

When I was running my own testing program, I was in desperate need of an associate to help me manage my small (but mighty!) team. My single associate and I were launching tests left and right and we were unable to do anything other than focus on the day-to-day of the program. A job description […]

The post How to hire your testing unicorn (without using magic) appeared first on Brooks Bell.

When I was running my own testing program, I was in desperate need of an associate to help me manage my small (but mighty!) team. My single associate and I were launching tests left and right and we were unable to do anything other than focus on the day-to-day of the program.

A job description had been posted and the company’s recruiters were doing everything they could to find the right hire.

I remember reaching out to an old friend of mine to see if she knew anyone who might fit the role. I told her that I was looking (simply) for a data-driven individual with stellar communication skills and the ability to manage several complicated web projects at one time.

“Oh,” she said. “So you’re looking for a unicorn.”

“No, Susan… I’m looking for a Testing Specialist.”

Now, I don’t want to be too dramatic here, but this unicorn revelation did rock my world a bit. (It also made me want a bowl of rainbow sorbet with sprinkles… but I digress.)

When I finally overcame this existential testing crisis, I realized that I believed, deep down, that testing unicorns did exist. But I also knew that due to magic (obviously), I might never find one.

There were three main things I was looking for in my unicorn:

  1. Strong analytics skills and the ability to develop advanced data-driven recommendations
  2. Amazing communication skills – for helping stakeholders understand and action off of that data
  3. Organized and efficient project management skills for planning and managing the execution of test strategies

First, I had to assess which skills I already had on my team.

I took a look at my own skills and the skills of the team I had in place. To be honest, I’m much better at talking about analytics than I am at sitting behind a desk and doing a deep dive into the numbers.

My personal strength is in the communication realm of testing and my associate was an awesome project manager. So, it became pretty clear to me that there was a need for a strong analyst on our team.

Then, I had to decide what was teachable.

This is where things get controversial. Because teachable skills can really depend on the skills of the trainee, the trainee’s willingness to learn, and the skills of the trainer.

I did a quick poll here at Brooks Bell to see which skills my colleagues believe is the toughest to teach.

As you can see, many people here believe that good communication skills are hard to coach. And during my search for a Testing Specialist, I felt the same way.

I was pretty confident that I would be able to help my next hire become a better analyst or project manager, but I wasn’t so sure I could teach someone to communicate well in a stakeholder-facing role.

Finally, I had to decide if I could tweak my program structure

Depending on my next hire’s strengths, there were a few scenarios that I had to consider in order to structure my program without a unicorn. Here are a few examples:

If I decided to hire a strong analyst with weak communication skills

In this scenario, I would consider making this Testing Specialist role a non-stakeholder facing role. Because this person would not be project managing or communicating directly with stakeholders, they would be solely dedicated to analytics and free up the rest of the team’s time to focus on project management and stakeholder communication.

If I decided to hire a strong project manager with weak analytics skills

Because I believed that analytics skills were teachable, this associate could focus on project management in the beginning and slowly take on analytics work when they were ready.

If I decided to hire a strong communicator with weak project management skills

In this scenario, I would start by putting this associate in a stakeholder-facing role focused on analytics. After some time, I would begin training him or her on project management.

The magical lesson I learned

When I first approached this seemingly impossible task of hiring my next Testing Specialist, I was discouraged by the reality that I wanted so many specific skills in one individual.

But the truth is, Experimentation and Optimization is still a very niche industry, so finding a single person with so many abilities is going to continue to be tough for a while. That’s why I recommend first looking at the structure of your team, and then deciding which skills you feel comfortable teaching.

And always remember this: Testing unicorns do exist, sometimes we just have to help them find their wings.

Are you a testing unicorn looking for your next big challenge? Check out our monthly “who’s hiring” post for open positions in testing and personalization at top companies.


About the Author:

Sam Baker has eight years of experience running experimentation and digital analytics programs for major e-commerce brands. As a consultant at Brooks Bell, she helps global brands build and grow their testing programs.

In addition to her role at Brooks Bell, Sam is also an accomplished career coach, providing guidance to ambitious women looking to land their dream careers. Originally from Indiana, Sam now lives in Raleigh, North Carolina with her husband and her dog.

The post How to hire your testing unicorn (without using magic) appeared first on Brooks Bell.