Video Series: Conquer Your Biggest Testing Challenges

Here at Brooks Bell, we work with clients that are at varying stages of maturity when it comes to experimentation. Despite the differences in these partnerships, you might be surprised to learn that regardless of whether we’re working with a new or established testing program, they all face common enemies: pressure to deliver results; inefficient […]

The post Video Series: Conquer Your Biggest Testing Challenges appeared first on Brooks Bell.

Here at Brooks Bell, we work with clients that are at varying stages of maturity when it comes to experimentation. Despite the differences in these partnerships, you might be surprised to learn that regardless of whether we’re working with a new or established testing program, they all face common enemies: pressure to deliver results; inefficient processes; a lack of understanding and support for testing; and difficulty iterating on and applying learnings from test results.

In this four-part video series, you’ll hear from Suzi Tripp, our Sr. Director of Innovative Solutions, Jonathan Hildebrand, Sr. Director of Design & UX, and Claire Schmitt, VP of Strategic Consulting and Solutions at Brooks Bell. They’ll discuss tips and tricks for addressing these challenges. You’ll also get insight into best practices for organizing your testing program, developing smarter tests, showcasing your results and obtaining insights about your customers.

Check out the first video below, or watch the full series by filling out the form at the bottom of this post.

Part 1: Storing and Learning from Past Tests

Fill out the form below to view the other three videos, covering:

  • Collaborative Ideation / Strategizing Better Tests
  • Communicating Testing Insights Up The Ladder
  • Retaining and Growing Testing Budget

The post Video Series: Conquer Your Biggest Testing Challenges appeared first on Brooks Bell.

Free Resource: How to Staff Your Testing Program to Meet Your Velocity Goals

Say you’re getting some work done to your home—you wouldn’t want plumbers painting your walls, right? What about electricians doing your landscaping? Or HVAC technicians repairing your roof? The same thing goes for your experimentation program. When your budget is tight, it can be tempting to have members of your team take on multiple roles. […]

The post Free Resource: How to Staff Your Testing Program to Meet Your Velocity Goals appeared first on Brooks Bell.

Say you’re getting some work done to your home—you wouldn’t want plumbers painting your walls, right? What about electricians doing your landscaping? Or HVAC technicians repairing your roof?

The same thing goes for your experimentation program.

When your budget is tight, it can be tempting to have members of your team take on multiple roles. For example, your analysts might be okay with managing the entire process of launching a test, developers could do full QA on their own tests, and project managers could oversee both test strategy and the tactical work required to execute tests.

But here’s why this is a bad idea: if you want something done right, you need to not only make sure you have the right people on the job, but also the right amount of people doing the work.

As you finalize your plans for 2019, we’ve put together a free guide to staffing your testing team according to the number of tests you want to launch per month. Download it today.


 

The post Free Resource: How to Staff Your Testing Program to Meet Your Velocity Goals appeared first on Brooks Bell.

The Expert’s Guide to A/B Testing During the Holiday Season

In 2016, online spending topped in-store shopping for the first time ever. That trend continued in 2017, with Adobe Digital Insights reporting that 2017 holiday sales surpassed $91.7 billion, marking 11% YoY growth. Peak season offers peak opportunities for experimentation programs. Increased traffic and conversion rates open the door for higher velocity, shorter durations, and lower minimum […]

The post The Expert’s Guide to A/B Testing During the Holiday Season appeared first on Brooks Bell.

In 2016, online spending topped in-store shopping for the first time ever. That trend continued in 2017, with Adobe Digital Insights reporting that 2017 holiday sales surpassed $91.7 billion, marking 11% YoY growth.

Peak season offers peak opportunities for experimentation programs. Increased traffic and conversion rates open the door for higher velocity, shorter durations, and lower minimum detectable lifts without compromising statistical significance.

If you haven’t already created your experimentation strategy, the time is now. But here are some essential factors to consider while creating your holiday testing game plan.

Maximize Your Holiday Window
Thanksgiving Day kicks off the peak holiday season, which continues through December 23. If you know your holiday window and website traffic patterns and expectations, you’ve got what it takes to take full advantage of this opportunity.

It can get complicated, but here’s a simple way to start:

  1. Define your holiday window. Consult past data to determine when traffic and conversion increases start and stop.
  2. Layer in the changes your organization is forecasting over last year. For example, one of our clients is expecting a five percent increase in traffic over last year’s holiday season. That intel is reflected in our traffic assumptions.
  3. Start your roadmap with the most valuable pages so that early wins can positively impact the rest of the holiday season. Create a punch list of pages with this in mind.
  4. Use traffic assumptions, desired statistical significance, and minimum detectable lift to determine the sample size and duration of tests.
  5. Continue this process to fill the window of time. Use these dates to mobilize your team, communicating key dates of test strategy kickoff, when tests will move into development, when they will launch and end and when results will be shared.

Communication is Critical
Since the holiday season represents a large portion of annual revenue, stress and emotions run high. As a result, it’s important to create your communication plan in advance. Determine who your stakeholders are, the optimal frequency of updates and what information needs to be shared. This isn’t the time for surprises or big reveals, so plan to devote a chunk of time to telling the story of your program and communicating its value.

The Weather Outside May Not Be the Only Freeze You’re Experiencing

Some organizations implement a freeze on development code updates and changes during the holiday season to avoid the risk of broken digital experiences or performance disruptions.  Get acclimated with your company’s approach so you can have a plan for implementing winning test programs.

The ideal scenario is to push winners immediately into production. Based on years of experience with enterprise clients, Brooks Bell strongly advocates this approach so you can maximize the impact of that winning test.

If production updates aren’t on the table because of a code freeze, don’t immediately jump to the decision to push the winner to 100 percent through your testing tool. Though it sounds like the best way to manage through code freezes, it could cause delays and create an undesirable experience. Before you make the decision how to handle, get your organization’s development experts involved to help you evaluate the risks and rewards.

Holiday Shoppers are Different
Think about your own shopping behaviors during the holidays compared to the rest of the year.

When I’m shopping during the holidays, I find myself on a mission to knock out my shopping list. As the countdown clock ticks away in my brain (and often literally on websites), I have a very real and intense sense of urgency. For me, customer confidence indicators, obvious savings and a clear and easy path to checkout are the ticket.

During the rest of the year, shopping is more leisurely for me and allows time for more browsing and consideration. I may even visit a website a few times before making a purchase. I zoom in on product details. I read customer reviews. I have more time, and the only restrictions are my own.

I’m the same person but have a very different mindset. The same goes for your customers. Keep this in mind as you develop your holiday testing roadmap.

Here are four tips to help ensure your holiday experimentation wins continue to add business value:

  • Keep it simple. As illustrated in my example above, successful holiday strategies are frequently based on a streamlined path to purchase, removing any friction and creating a sense of urgency and scarcity.
  • Test your hypotheses again after the holiday season. Do these experiences still produce a conversion lift when the holiday rush isn’t in full effect? If not, it’s okay! It’s an important learning you can use to build your Holiday/Non-Holiday playbook to make each holiday season better than the last.
  • Know your “Out of Stock” strategy. Regardless of what changes you make to your Product Detail page, nothing zaps excitement out of a customer experience faster than something being Out of Stock. Understand how your site handles Out of Stock messages, such as using red copy or suggesting alternate options. If it’s less than optimal, do some early testing to determine the most effective messaging. If your site includes a lot of Out of Stock product, it’s even more important to make sure it‘s been optimized.
  • Document your findings. Carve out time to tell the story of your testing through the chaos of increased velocity. The data and insights will be helpful after the rush and can greatly influence your future program success. Be sure to look at new, returning and loyal segments, and evaluate the differences in their holiday and non-holiday shopping behaviors.

For more intel on how to make the most of the merry months ahead, download our white paper, “5 Testing Tips for the Holidays.”

Need help developing a game plan for holiday testing? Contact us today!

The post The Expert’s Guide to A/B Testing During the Holiday Season appeared first on Brooks Bell.

Changing Your A/B Testing Software? Read These Tips First.

With the number of testing and personalization tools available, it can be difficult to choose one to invest in. But once you’ve already selected a software, making the decision to transition to a new tool altogether can feel overwhelming. But this happens quite often. For many clients, cost is often the deciding factor in making […]

The post Changing Your A/B Testing Software? Read These Tips First. appeared first on Brooks Bell.

With the number of testing and personalization tools available, it can be difficult to choose one to invest in. But once you’ve already selected a software, making the decision to transition to a new tool altogether can feel overwhelming.

But this happens quite often. For many clients, cost is often the deciding factor in making the decision to switch testing tools–there are a few testing tools that offer similar capabilities at a lower price point. On the flip side, if you’ve increased your program budget and capabilities, it may be time for an upgrade.

And although all testing tools offer similar functions, each has unique features that are important to consider. Personalization, for example, has become a point of focus for many testing programs – perhaps you’re interested in transitioning to a tool such as Evergage or Dynamic Yield that puts personalization at the forefront. Or your testing program has enough velocity to run multiple experiments simultaneously, and you feel you’d make good use of Optimizely’s built-in mutually exclusive experiments feature. Maybe your company uses other Adobe products, like Adobe Experience Manager, so you feel Adobe Target is a good fit.

Regardless of which tool you select, once you select a new software–the next major obstacle is implementing it. Here are our tips for going about the process:

First, examine your testing roadmap.

Take inventory of the tests that will be running close to the date when you plan to stop using your previous tool. Make sure they will have reached significance and be ready to be turned off before you lose access. 

If your budget allows for it, we recommend giving your team a period of time where both tools are available. This will ensure your testing cadence isn’t affected while your team gets up to speed on using the new tool and allows you to transition more seamlessly – you’ll be able to let current tests run their course in the old tool while launching new ones in the new tool.

Then, test your testing software.

While you might be excited to dive in and start launching tests left and right, it’s important to take the time to ensure your new tool is implemented correctly.

Run a QA test that visually changes the page to check that the code is being delivered and the flicker looks reasonable. If there are a lot of flickers, you may need to move the testing tool tag higher up in the head of your HTML.

We also recommend running a live test without visual changes, just for the purpose of checking metrics. This enables your analyst to see that metrics are being tracked correctly within the testing tool, or if you’re using an outside analytics tool, that those metrics are being passed accurately to it. 

Once you’ve confirmed that visual changes are showing up as expected and metrics are tracking accurately, you’re ready to start using your new tool!

Switching testing software comes with its challenges. However, in the right circumstance, switching can offer substantial benefits to your testing program. Taking the time to pinpoint your reasons for switching, plan your testing roadmap carefully around the transition, and having patience as the new tool is implemented will ensure your tool transition goes smoothly.


Brooks Bell has over 15 years of experience working with enterprise brands to establish and scale their experimentation programs. We take a holistic approach to our technical diagnostics and analytics services, providing technology and data recommendations based on your business, your goals, your team, and your unique challenges.

What can Brooks Bell do for you?
✓   Clean, organize and centralize your customer data.
✓   Help you select the right a/b testing and personalization tools.
✓   Ensure your tools and systems integrate with one another.
✓   Train your developers and analysts.

Contact us to learn more.

The post Changing Your A/B Testing Software? Read These Tips First. appeared first on Brooks Bell.

Don’t Make It Weird: 5 Tips for Balancing Privacy & Personalization

Imagine a simple scenario: Your coworkers are participating in a fun run for charity and want you to join. You’re up for it, but you know you need a decent pair of running shoes. The logical solution is to go online, search for information about running shoes and identify a few possible options. You could […]

The post Don’t Make It Weird: 5 Tips for Balancing Privacy & Personalization appeared first on Brooks Bell.

Imagine a simple scenario: Your coworkers are participating in a fun run for charity and want you to join. You’re up for it, but you know you need a decent pair of running shoes.

The logical solution is to go online, search for information about running shoes and identify a few possible options. You could order the shoes from an online retailer, but because proper fit is important for running shoes, you decide to visit a specialty retailer at the mall. A salesperson there is friendly and knowledgeable. The store has a pair of shoes you like, in your size. They’re a bit more expensive, but the fitting service added value and there’s no additional shipping cost, so you purchase the shoes on the spot. The next weekend, you run the race and the shoes feel great.

This illustrates a relatively traditional model of consumer decision-making. It begins with a spark that motivates a search for a product. It leads to a research phase, and a consideration set is developed. It then progresses to some type of product experience that narrows the consideration set. Ultimately, a purchase decision occurs and an evaluation of the final product is made.

But today’s online customers may notice a glaring omission from the process: It occurs a week after the run, when you visit a news website. There, in the right column of the page, is an ad for a pair of running shoes. The ad is tailored to your expressed preferences, but not personalized enough to know that a purchase has already occurred.

Seeing these ads, which follow us around the web, can be annoying, unnerving, and even potentially embarrassing. Because the targeting is so crude, it’s obvious that we’ve exchanged some degree of privacy for a marginal—in this case questionable—convenience. And, if this exchange has happened so frictionlessly with one online retailer, how often is it happening elsewhere?

If you can relate to this, you’re not alone. Research has found that consumers generally dislike targeted and personalized advertising. So if personalization makes customers uncomfortable, does this means brands should stop using tailored messages, offers and experiences?

The answer is, decisively, no.

Here’s why: the same body of literature that outlines a negative attitude towards personalization, also highlights the undeniable benefits of personalization. When an ad or message—such as an email subject line—is tailored, even superficially, there is almost always an increase in engagement with the subsequent content.

This contradiction is known as the “privacy paradox.” Consumers are willing to make a long-term trade of personal privacy in exchange for a short-term benefit or convenience, like more relevant advertising or a more specific shopping experience.

But while attitudes toward privacy may contradict behavior, they certainly shouldn’t be ignored. When an ad, message, or experience feels intrusive or creepy, it can diminish the effect personalization could have on your customer and their overall perception of your brand.

Luckily, there are many ways to deliver personalized experiences while also making your customers feel more at ease about their privacy. Here are our tips.

1. Be transparent

Numerous studies have found that the more transparently personalized content is presented, the more effective—and importantly, the more broadly effective—it is.

While making explicit references to data collection and sharing policies can increase privacy concerns, it can also diminish the effect the concern has on consumer behavior.

In an extreme example, Facebook somewhat-recently rolling out a new way for users to see their ad preferences, after the company’s advertisement platform and practices faced scrutiny following the 2016 Presidential Election.

But it doesn’t have to be that complicated. For instance, simply including ad security icons, for example, has been shown to increase the effectiveness of tailored ads even when the icon is unrecognized.

In addition, referencing privacy policies can diminish concerns over data sharing and personalization, even if consumers never read the policy. One study found that consumers interpret a privacy policy as a blank slate populated with all the usual safeguards. This means, of course, the burden is on your company to draft and enforce a responsible privacy policy whenever consumer data is being collected.

2. Be public about your data security efforts

Unsurprisingly, reassuring your customers of data security and describing the efforts you’re taking to protect their data can make them feel more at ease. But the effectiveness of this approach really depends on how much your customers trust your brand and your site. Building this relationship is difficult and can be easily destroyed.

But, if your brand has built a relationship of trust over time and is authentically dedicated to preserving this relationship, referencing the care you have taken to secure private information can not only be a boon to overall perceptions but increase the effectiveness of personalization.

3. Be personal to the right people, at the right time.

In marketing, timing is everything; and the same goes for personalization, it turns out. In e-commerce, personalization is most effective when your customer has established a consideration set and a final decision is about to be made. Additionally, as your customer engages more with a product category or brand, they begin to expect and look forward to a more targeted, relevant experience.  

4. Let newer customers opt-in to personalization

When it comes to moving customers toward a purchase, personalization is more effective in the “pull” direction than in the “push” direction. This means that you should implement personalization with more loyal customers and especially those who have requested more tailored experiences.

For new customers, we suggest waiting to provide personalization until a visitor has shown a specific interest in your company or product: they’ve viewed a few pages on a website, downloaded your app or signed up for your email newsletter. Once this happens, offer a dialog asking “Would you like a more personal shopping experience?”

While it’s true that many visitors may choose to continue on their own, others may not. This also gives you an early opportunity to show your brands’ interest in providing a relevant, convenient shopping experience, which may come into play later once they become loyal customers.

5. Let your customers run the show

Perhaps the most unsettling recommendation for balancing privacy and personalization is to give up some control over the degree of personalization consumers experience. Doing so evokes many of the tips we’ve already covered: it improves transparency, allows consumers to opt into personalization, and helps to build trust. Additionally, offering this service has been found to dramatically improve the effectiveness of personalization, even when some customers actually change settings beyond the default.

Personalization is a powerful tool. The effect personalized messages and experiences have on customers, however, is variable and possibly unpredictable. It’s important that companies balance concerns for privacy and general feelings of intrusion when delivering personalized experiences. Testing these approaches we’ve outlined above will help make your personalization efforts feel less creepy and ultimately, increase the effectiveness of the customer experience.


Transform your customer experience through personalization.

Brooks Bells’ Personalization Jumpstart Program uses a comprehensive, five-step process to help top brands incorporate personalization across their customer experience. Learn more today >> 

The post Don’t Make It Weird: 5 Tips for Balancing Privacy & Personalization appeared first on Brooks Bell.