New Features in Illuminate: Impact Analysis, Enhanced Filters, Updated Dashboard & More

Since we launched Illuminate back in May, our team has been working around the clock to develop even more features to help optimization teams better organize experiments, report performance and maximize impact. Today, we’re excited to share a few of these with you. What’s new in Illuminate? Show impact and determine priority Use our new Impact […]

The post New Features in Illuminate: Impact Analysis, Enhanced Filters, Updated Dashboard & More appeared first on Brooks Bell.

Since we launched Illuminate back in May, our team has been working around the clock to develop even more features to help optimization teams better organize experiments, report performance and maximize impact. Today, we’re excited to share a few of these with you.

What’s new in Illuminate?

Show impact and determine priority

Use our new Impact Analysis to show the overall impact of your tests by page type and identify where you should be focusing your testing efforts.

Sort and filter by what matters most

Filter your tests by 15 attributes including target audience, page type, start and end date, KPIs, revenue impact and more. Not seeing what you need? Add your own using our new custom tagging feature.

Keep sight of the bigger picture

Our new dashboard view enables you to view your program’s overall performance or view performance by a specific team or line of business.

+ a new tiled layout

If you love a good masonry layout (á la Pinterest), then you’re going to love our updated experiment view. Easily switch between a basic list of your experiments or a super slick-looking tiled layout.

Many of these features were developed in response to feedback from our beta users, bringing more of Brooks Bell’s advanced experimentation methodologies directly into the software.

“With Illuminate, you’re not just getting another test repository,” said Suzi Tripp, Senior Director of Innovative Solutions at Brooks Bell. “You’re getting 15 years of experimentation expertise and proven frameworks to help you do more, and do it better.”

Interested in learning more about illuminate? Learn more on our website or schedule a demo using the form below.

The post New Features in Illuminate: Impact Analysis, Enhanced Filters, Updated Dashboard & More appeared first on Brooks Bell.

Built to Wow: An Introduction to Launching Personalization At Your Company

The promise of personalization is enticing: a complete 1-to-1 experience for every customer, driven by every detail and data point about that person: who they are, their interests, needs and history. Their customer experience is completely optimized to deliver the right content at the right time, influencing brand engagement, purchase activity and “wow”-worthy customer experiences. […]

The post Built to Wow: An Introduction to Launching Personalization At Your Company appeared first on Brooks Bell.

The promise of personalization is enticing: a complete 1-to-1 experience for every customer, driven by every detail and data point about that person: who they are, their interests, needs and history. Their customer experience is completely optimized to deliver the right content at the right time, influencing brand engagement, purchase activity and “wow”-worthy customer experiences.

For years, this vision has been a pipedream among marketers, product managers and customer experience professionals. Many clients come to us wanting to “do personalization” but face significant challenges in doing so.

Part of this is due to the fact that “personalization” is so ill-defined.

At Brooks Bell, we define personalization as any experience that is delivered to a user based on known data about that person. By that definition, personalization exists on a spectrum: it can be one-to-few, one-to-many, or one-to-one. In the digital environment, product recommendations, customized search results and even segmented experiences are all considered examples of personalization.

But while many companies are already implementing these experiences, there’s still an overwhelming sense that many brands have yet to arrive in terms of personalization.


Got a bunch of burning questions about personalization? Submit them using the form below.

We’ll use this information to make sure we cover these topics in our upcoming posts.


A 2018 study of 300 marketers by Evergage and Researchscape International found that 98% of respondents believe personalization helps advance customer relationships, but only 12% were “very” or “extremely” satisfied with the level of personalization in their marketing efforts.

This is because (not unlike experimentation) personalization is a business strategy that should evolve in order to deliver long-term value. And while it’s true that many brands already have the ability to do personalization, they’ve also found that elevating and scaling a personalization program is difficult, costly and, frankly, can feel pretty darn impossible.

So, how to do this? In addition to the fundamentals for a standard optimization program, there are three critical working components that need to be established for personalization:

  • Technology: you need top-notch tools to centralize user profiles and deliver personalized experiences;
  • Data: personalization requires a clean, unified view of relevant customer attributes, and
  • Strategy: you need research and planning to purposefully and effectively launch, scale and benefit from personalization.

In the coming weeks, we’re going to break down personalization further by each of these components. We’ll outline the best practices, advice, strategies and tips to go from scrappy to smart when it comes to introducing and scaling personalization at your organization.

Struggling to execute a scalable personalization strategy? We can help. Contact us to schedule a free 30-minute consultation.

The post Built to Wow: An Introduction to Launching Personalization At Your Company appeared first on Brooks Bell.

What Happens When Data Meets Creative (and How to Make it a Reality at Your Company)

There are quite a few people out there that just don’t *get* creative. They don’t understand the way in which we work or make decisions. And, indeed, creative teams are known to be cost centers rather than revenue generators. To certain execs, creatives are simply the sneaker-wearing hipsters who are brought in to make things […]

The post What Happens When Data Meets Creative (and How to Make it a Reality at Your Company) appeared first on Brooks Bell.

There are quite a few people out there that just don’t *get* creative. They don’t understand the way in which we work or make decisions. And, indeed, creative teams are known to be cost centers rather than revenue generators. To certain execs, creatives are simply the sneaker-wearing hipsters who are brought in to make things look pretty or sound good.

While this is a far cry from reality, it’s also not that hard to understand why. As creatives, we understand the value of good creative work. Proving that value, however, can be difficult. So here are a few tips for proving the ROI of your creative team and incorporating data within your creative process.

Tip #1: Know and Speak the Language of Business.

Smart creative work requires an objective-based approach. Objective-based creative is driven by data—often in the form of user feedback, website analytics, and strategic business goals. As a designer or copywriter, your job is to gather and digest this data and apply it to your work.  

When pitching your concepts to your stakeholders, most aren’t going to accept work that just “looks” better. It’s important that you are able to articulate the business problem, your target audience and the objective-based reasoning behind your decisions. This ensures that your work is influenced by hard data and research, rather than just design preferences.

On the other side, it’s important to train your stakeholders in the art of objective-based feedback. That is, feedback in the context of whether or not your work is effective in addressing the objective at hand. Doing this takes time, practice and a lot of patience, but the payoff is huge. Your executives will feel more confident after seeing that your creative team is aligned and hyper-focused on providing measurable value.

Tip #2: Use Testing to Eradicate B.S. in the Creative Process.

Brooks Bell was founded on the idea that you can eliminate creative guesswork by applying the scientific method. But at many companies, creative and UX teams rarely engage with testing teams. While this might make sense from the perspective of your org chart, few realize just how much collaboration between these functions could positively impact a business.

A few years back, my team and I were brought in to work with one of our retail clients. Looking at their website data, our analysts realized that a large majority of people were abandoning the express checkout form for the full checkout form. This seemed counterintuitive to us: less friction is always better, right? Why would anyone prefer to fill out the long form!?

In order to develop a strategy to test, we needed more data—so we turned to user research. We polled a select group of users about their purchasing experience and uncovered some potential reasons for their behavior.

We discovered that many users preferred to use alternative or saved payment methods, yet the account login and gift card payment options were only available in the full checkout experience. We ran a test adding these options to the express checkout flow, which resulted in a 5% lift. When implemented, this test translated to a $5M increase in revenue.

The impact of this was significant—and not just from a revenue perspective. Through this process, we were able to identify other areas where users could be experiencing anxiety. It also prevented us from over-designing in the future. For this company’s customers, a simple and clear message and a less cluttered experience were enough to quell their anxiety.

For data-starved creatives, these types of insights can be extremely valuable and can greatly influence the company’s overall design aesthetic.

Tip #3: Be Sure You Recruit Relevant User Groups for Discovery Research

This tip is for you if—upon presenting the results of your user research—you’ve ever been asked “why did you talk to [audience group]?” or the alternative: ”why didn’t you talk to [audience group]?”

Sure, conducting guerilla research on random mall-goers or your coworkers at lunchtime will get you basic usability feedback. But if you want actionable insights, you need to not only research the group that’s generating the most business for your company, but also the group that’s most impacted by the problem you’re trying to solve.

If return users drive the majority of your revenue, don’t research new users. Similarly, don’t ask someone to look at your mobile design if they don’t fit the demographics of the segment you’re trying to reach.

Here at Brooks Bell, we believe it’s important for our clients to be closely involved in the process of selecting user segments for research. This not only manages the scope of the project and ensures maximum impact, but it also helps to avoid the frustrating line of questioning I mentioned above.

Tip #4: Embrace Survey-Based Research

If you’re well-versed in usability testing, you know that elaborate usability tests are a waste of resources and you really can get the best results from testing no more than 5 users.  But to an executive, that number 5 can seem awfully small. And no matter how many times you reference or point them to this blog post, they still might just not buy it.

This is where survey-based research comes in. We’ve had tremendous success in conducting survey-based research for our clients, and find it is often better received by executives.

Executives respond well to survey research for a couple of reasons: You can survey a larger population of people. It’s fast—most of the time we get responses back within a day or two. And finally, depending on the types of questions you ask, it’s largely quantifiable.

While surveys are different from usability tests, oftentimes, you can use survey results to back up your usability test results.

Finally, it’s important that you also become the master of your research domains and empower yourself to dig in on your own.  For this, pivot tables are a great tool. Pivot tables unlock the magic of Excel by allowing us to take all of our survey results and slice and dice them any way we want… filtering answers by segments, averaging, counting, and creating data visualizations all without ever having to talk to an analyst.

How many of you thought you’d leave this post adding Excel to your list of preferred programs? 😉

Tip #5: Don’t Hoard Your Ideas – Bring Others Into the Creative Process

It’s every designer’s tale of despair: you spend tons of time on a project—putting in extra hours to make sure every pixel has been pushed into the perfect position, every line kerned and leaded—only to have your work completely shat on upon unveiling it.

Trust me on this one: hoarding your ideas and excluding other from your design process really only sets you up for disappointment, depression and frustration.  

So stop with the big reveal and instead invite others into the design process. Voice your ideas in a collaborative way. Position yourself as a guide within a creative process in which the objective is to build something collaboratively. Without a doubt, you’ll find you’ll get things approved faster and more frequently.

 


Interested in learning how Brooks Bell can help empower your creative and UX teams with data? Learn more about our services or contact us today.

The post What Happens When Data Meets Creative (and How to Make it a Reality at Your Company) appeared first on Brooks Bell.

Video Series: Conquer Your Biggest Testing Challenges

Here at Brooks Bell, we work with clients that are at varying stages of maturity when it comes to experimentation. Despite the differences in these partnerships, you might be surprised to learn that regardless of whether we’re working with a new or established testing program, they all face common enemies: pressure to deliver results; inefficient […]

The post Video Series: Conquer Your Biggest Testing Challenges appeared first on Brooks Bell.

Here at Brooks Bell, we work with clients that are at varying stages of maturity when it comes to experimentation. Despite the differences in these partnerships, you might be surprised to learn that regardless of whether we’re working with a new or established testing program, they all face common enemies: pressure to deliver results; inefficient processes; a lack of understanding and support for testing; and difficulty iterating on and applying learnings from test results.

In this four-part video series, you’ll hear from Suzi Tripp, our Sr. Director of Innovative Solutions, Jonathan Hildebrand, Sr. Director of Design & UX, and Claire Schmitt, VP of Strategic Consulting and Solutions at Brooks Bell. They’ll discuss tips and tricks for addressing these challenges. You’ll also get insight into best practices for organizing your testing program, developing smarter tests, showcasing your results and obtaining insights about your customers.

Check out the first video below, or watch the full series by filling out the form at the bottom of this post.

Part 1: Storing and Learning from Past Tests

Fill out the form below to view the other three videos, covering:

  • Collaborative Ideation / Strategizing Better Tests
  • Communicating Testing Insights Up The Ladder
  • Retaining and Growing Testing Budget

The post Video Series: Conquer Your Biggest Testing Challenges appeared first on Brooks Bell.

Free Resource: How to Staff Your Testing Program to Meet Your Velocity Goals

Say you’re getting some work done to your home—you wouldn’t want plumbers painting your walls, right? What about electricians doing your landscaping? Or HVAC technicians repairing your roof? The same thing goes for your experimentation program. When your budget is tight, it can be tempting to have members of your team take on multiple roles. […]

The post Free Resource: How to Staff Your Testing Program to Meet Your Velocity Goals appeared first on Brooks Bell.

Say you’re getting some work done to your home—you wouldn’t want plumbers painting your walls, right? What about electricians doing your landscaping? Or HVAC technicians repairing your roof?

The same thing goes for your experimentation program.

When your budget is tight, it can be tempting to have members of your team take on multiple roles. For example, your analysts might be okay with managing the entire process of launching a test, developers could do full QA on their own tests, and project managers could oversee both test strategy and the tactical work required to execute tests.

But here’s why this is a bad idea: if you want something done right, you need to not only make sure you have the right people on the job, but also the right amount of people doing the work.

As you finalize your plans for 2019, we’ve put together a free guide to staffing your testing team according to the number of tests you want to launch per month. Download it today.


 

The post Free Resource: How to Staff Your Testing Program to Meet Your Velocity Goals appeared first on Brooks Bell.

The Expert’s Guide to A/B Testing During the Holiday Season

In 2016, online spending topped in-store shopping for the first time ever. That trend continued in 2017, with Adobe Digital Insights reporting that 2017 holiday sales surpassed $91.7 billion, marking 11% YoY growth. Peak season offers peak opportunities for experimentation programs. Increased traffic and conversion rates open the door for higher velocity, shorter durations, and lower minimum […]

The post The Expert’s Guide to A/B Testing During the Holiday Season appeared first on Brooks Bell.

In 2016, online spending topped in-store shopping for the first time ever. That trend continued in 2017, with Adobe Digital Insights reporting that 2017 holiday sales surpassed $91.7 billion, marking 11% YoY growth.

Peak season offers peak opportunities for experimentation programs. Increased traffic and conversion rates open the door for higher velocity, shorter durations, and lower minimum detectable lifts without compromising statistical significance.

If you haven’t already created your experimentation strategy, the time is now. But here are some essential factors to consider while creating your holiday testing game plan.

Maximize Your Holiday Window
Thanksgiving Day kicks off the peak holiday season, which continues through December 23. If you know your holiday window and website traffic patterns and expectations, you’ve got what it takes to take full advantage of this opportunity.

It can get complicated, but here’s a simple way to start:

  1. Define your holiday window. Consult past data to determine when traffic and conversion increases start and stop.
  2. Layer in the changes your organization is forecasting over last year. For example, one of our clients is expecting a five percent increase in traffic over last year’s holiday season. That intel is reflected in our traffic assumptions.
  3. Start your roadmap with the most valuable pages so that early wins can positively impact the rest of the holiday season. Create a punch list of pages with this in mind.
  4. Use traffic assumptions, desired statistical significance, and minimum detectable lift to determine the sample size and duration of tests.
  5. Continue this process to fill the window of time. Use these dates to mobilize your team, communicating key dates of test strategy kickoff, when tests will move into development, when they will launch and end and when results will be shared.

Communication is Critical
Since the holiday season represents a large portion of annual revenue, stress and emotions run high. As a result, it’s important to create your communication plan in advance. Determine who your stakeholders are, the optimal frequency of updates and what information needs to be shared. This isn’t the time for surprises or big reveals, so plan to devote a chunk of time to telling the story of your program and communicating its value.

The Weather Outside May Not Be the Only Freeze You’re Experiencing

Some organizations implement a freeze on development code updates and changes during the holiday season to avoid the risk of broken digital experiences or performance disruptions.  Get acclimated with your company’s approach so you can have a plan for implementing winning test programs.

The ideal scenario is to push winners immediately into production. Based on years of experience with enterprise clients, Brooks Bell strongly advocates this approach so you can maximize the impact of that winning test.

If production updates aren’t on the table because of a code freeze, don’t immediately jump to the decision to push the winner to 100 percent through your testing tool. Though it sounds like the best way to manage through code freezes, it could cause delays and create an undesirable experience. Before you make the decision how to handle, get your organization’s development experts involved to help you evaluate the risks and rewards.

Holiday Shoppers are Different
Think about your own shopping behaviors during the holidays compared to the rest of the year.

When I’m shopping during the holidays, I find myself on a mission to knock out my shopping list. As the countdown clock ticks away in my brain (and often literally on websites), I have a very real and intense sense of urgency. For me, customer confidence indicators, obvious savings and a clear and easy path to checkout are the ticket.

During the rest of the year, shopping is more leisurely for me and allows time for more browsing and consideration. I may even visit a website a few times before making a purchase. I zoom in on product details. I read customer reviews. I have more time, and the only restrictions are my own.

I’m the same person but have a very different mindset. The same goes for your customers. Keep this in mind as you develop your holiday testing roadmap.

Here are four tips to help ensure your holiday experimentation wins continue to add business value:

  • Keep it simple. As illustrated in my example above, successful holiday strategies are frequently based on a streamlined path to purchase, removing any friction and creating a sense of urgency and scarcity.
  • Test your hypotheses again after the holiday season. Do these experiences still produce a conversion lift when the holiday rush isn’t in full effect? If not, it’s okay! It’s an important learning you can use to build your Holiday/Non-Holiday playbook to make each holiday season better than the last.
  • Know your “Out of Stock” strategy. Regardless of what changes you make to your Product Detail page, nothing zaps excitement out of a customer experience faster than something being Out of Stock. Understand how your site handles Out of Stock messages, such as using red copy or suggesting alternate options. If it’s less than optimal, do some early testing to determine the most effective messaging. If your site includes a lot of Out of Stock product, it’s even more important to make sure it‘s been optimized.
  • Document your findings. Carve out time to tell the story of your testing through the chaos of increased velocity. The data and insights will be helpful after the rush and can greatly influence your future program success. Be sure to look at new, returning and loyal segments, and evaluate the differences in their holiday and non-holiday shopping behaviors.

For more intel on how to make the most of the merry months ahead, download our white paper, “5 Testing Tips for the Holidays.”

Need help developing a game plan for holiday testing? Contact us today!

The post The Expert’s Guide to A/B Testing During the Holiday Season appeared first on Brooks Bell.

Changing Your A/B Testing Software? Read These Tips First.

With the number of testing and personalization tools available, it can be difficult to choose one to invest in. But once you’ve already selected a software, making the decision to transition to a new tool altogether can feel overwhelming. But this happens quite often. For many clients, cost is often the deciding factor in making […]

The post Changing Your A/B Testing Software? Read These Tips First. appeared first on Brooks Bell.

With the number of testing and personalization tools available, it can be difficult to choose one to invest in. But once you’ve already selected a software, making the decision to transition to a new tool altogether can feel overwhelming.

But this happens quite often. For many clients, cost is often the deciding factor in making the decision to switch testing tools–there are a few testing tools that offer similar capabilities at a lower price point. On the flip side, if you’ve increased your program budget and capabilities, it may be time for an upgrade.

And although all testing tools offer similar functions, each has unique features that are important to consider. Personalization, for example, has become a point of focus for many testing programs – perhaps you’re interested in transitioning to a tool such as Evergage or Dynamic Yield that puts personalization at the forefront. Or your testing program has enough velocity to run multiple experiments simultaneously, and you feel you’d make good use of Optimizely’s built-in mutually exclusive experiments feature. Maybe your company uses other Adobe products, like Adobe Experience Manager, so you feel Adobe Target is a good fit.

Regardless of which tool you select, once you select a new software–the next major obstacle is implementing it. Here are our tips for going about the process:

First, examine your testing roadmap.

Take inventory of the tests that will be running close to the date when you plan to stop using your previous tool. Make sure they will have reached significance and be ready to be turned off before you lose access. 

If your budget allows for it, we recommend giving your team a period of time where both tools are available. This will ensure your testing cadence isn’t affected while your team gets up to speed on using the new tool and allows you to transition more seamlessly – you’ll be able to let current tests run their course in the old tool while launching new ones in the new tool.

Then, test your testing software.

While you might be excited to dive in and start launching tests left and right, it’s important to take the time to ensure your new tool is implemented correctly.

Run a QA test that visually changes the page to check that the code is being delivered and the flicker looks reasonable. If there are a lot of flickers, you may need to move the testing tool tag higher up in the head of your HTML.

We also recommend running a live test without visual changes, just for the purpose of checking metrics. This enables your analyst to see that metrics are being tracked correctly within the testing tool, or if you’re using an outside analytics tool, that those metrics are being passed accurately to it. 

Once you’ve confirmed that visual changes are showing up as expected and metrics are tracking accurately, you’re ready to start using your new tool!

Switching testing software comes with its challenges. However, in the right circumstance, switching can offer substantial benefits to your testing program. Taking the time to pinpoint your reasons for switching, plan your testing roadmap carefully around the transition, and having patience as the new tool is implemented will ensure your tool transition goes smoothly.


Brooks Bell has over 15 years of experience working with enterprise brands to establish and scale their experimentation programs. We take a holistic approach to our technical diagnostics and analytics services, providing technology and data recommendations based on your business, your goals, your team, and your unique challenges.

What can Brooks Bell do for you?
✓   Clean, organize and centralize your customer data.
✓   Help you select the right a/b testing and personalization tools.
✓   Ensure your tools and systems integrate with one another.
✓   Train your developers and analysts.

Contact us to learn more.

The post Changing Your A/B Testing Software? Read These Tips First. appeared first on Brooks Bell.