New Features in Illuminate: Impact Analysis, Enhanced Filters, Updated Dashboard & More

Since we launched Illuminate back in May, our team has been working around the clock to develop even more features to help optimization teams better organize experiments, report performance and maximize impact. Today, we’re excited to share a few of these with you. What’s new in Illuminate? Show impact and determine priority Use our new Impact […]

The post New Features in Illuminate: Impact Analysis, Enhanced Filters, Updated Dashboard & More appeared first on Brooks Bell.

Since we launched Illuminate back in May, our team has been working around the clock to develop even more features to help optimization teams better organize experiments, report performance and maximize impact. Today, we’re excited to share a few of these with you.

What’s new in Illuminate?

Show impact and determine priority

Use our new Impact Analysis to show the overall impact of your tests by page type and identify where you should be focusing your testing efforts.

Sort and filter by what matters most

Filter your tests by 15 attributes including target audience, page type, start and end date, KPIs, revenue impact and more. Not seeing what you need? Add your own using our new custom tagging feature.

Keep sight of the bigger picture

Our new dashboard view enables you to view your program’s overall performance or view performance by a specific team or line of business.

+ a new tiled layout

If you love a good masonry layout (á la Pinterest), then you’re going to love our updated experiment view. Easily switch between a basic list of your experiments or a super slick-looking tiled layout.

Many of these features were developed in response to feedback from our beta users, bringing more of Brooks Bell’s advanced experimentation methodologies directly into the software.

“With Illuminate, you’re not just getting another test repository,” said Suzi Tripp, Senior Director of Innovative Solutions at Brooks Bell. “You’re getting 15 years of experimentation expertise and proven frameworks to help you do more, and do it better.”

Interested in learning more about illuminate? Learn more on our website or schedule a demo using the form below.

The post New Features in Illuminate: Impact Analysis, Enhanced Filters, Updated Dashboard & More appeared first on Brooks Bell.

Running conversion optimization experiments the right way with Chad Sanderson

Learn how to run conversion optimization experiments the right way. In this video, I sit down with Chad Sanderson, Program Manager on the Microsoft Experimentation Platform team, to discuss statistical testing, calculating sample size, and selecting the right tools to help you run statistically significant conversion optimization tests. Subscribe to our YouTube Channel  

The post Running conversion optimization experiments the right way with Chad Sanderson appeared first on CXL.

Learn how to run conversion optimization experiments the right way. In this video, I sit down with Chad Sanderson, Program Manager on the Microsoft Experimentation Platform team, to discuss statistical testing, calculating sample size, and selecting the right tools to help you run statistically significant conversion optimization tests.

[This post contains video, click to play]

Subscribe to our YouTube Channel

 

The post Running conversion optimization experiments the right way with Chad Sanderson appeared first on CXL.

How to Optimize Websites to land B2B Deals with Bill Leake

Digital Marketers wanting to land B2B deals are often optimizing for the wrong metrics, focused solely on conversion rate and getting low-quality leads. You’re doing it wrong! That’s why I interviewed my friend and seasoned B2B expert, Bill Leake, to discuss the most common mistakes in b2b marketing and how to optimize your website to […]

The post How to Optimize Websites to land B2B Deals with Bill Leake appeared first on CXL.

Digital Marketers wanting to land B2B deals are often optimizing for the wrong metrics, focused solely on conversion rate and getting low-quality leads. You’re doing it wrong! That’s why I interviewed my friend and seasoned B2B expert, Bill Leake, to discuss the most common mistakes in b2b marketing and how to optimize your website to land b2b deals the right way.

[This post contains video, click to play]

Subscribe to our YouTube Channel

 

The post How to Optimize Websites to land B2B Deals with Bill Leake appeared first on CXL.

Video Series: Conquer Your Biggest Testing Challenges

Here at Brooks Bell, we work with clients that are at varying stages of maturity when it comes to experimentation. Despite the differences in these partnerships, you might be surprised to learn that regardless of whether we’re working with a new or established testing program, they all face common enemies: pressure to deliver results; inefficient […]

The post Video Series: Conquer Your Biggest Testing Challenges appeared first on Brooks Bell.

Here at Brooks Bell, we work with clients that are at varying stages of maturity when it comes to experimentation. Despite the differences in these partnerships, you might be surprised to learn that regardless of whether we’re working with a new or established testing program, they all face common enemies: pressure to deliver results; inefficient processes; a lack of understanding and support for testing; and difficulty iterating on and applying learnings from test results.

In this four-part video series, you’ll hear from Suzi Tripp, our Sr. Director of Innovative Solutions, Jonathan Hildebrand, Sr. Director of Design & UX, and Claire Schmitt, VP of Strategic Consulting and Solutions at Brooks Bell. They’ll discuss tips and tricks for addressing these challenges. You’ll also get insight into best practices for organizing your testing program, developing smarter tests, showcasing your results and obtaining insights about your customers.

Check out the first video below, or watch the full series by filling out the form at the bottom of this post.

Part 1: Storing and Learning from Past Tests

Fill out the form below to view the other three videos, covering:

  • Collaborative Ideation / Strategizing Better Tests
  • Communicating Testing Insights Up The Ladder
  • Retaining and Growing Testing Budget

The post Video Series: Conquer Your Biggest Testing Challenges appeared first on Brooks Bell.

What are your website visitors doing?

Chances are that you’re tracking your website visitors en masse. You’re probably tracking acquisition sites, tallying up conversions and working to optimize your pages for the best success. But with all of that quantitative research, do you know about each individual user’s journey, and where they are struggling on your site? If not, you should […]

The post What are your website visitors doing? appeared first on Brooks Bell.

Chances are that you’re tracking your website visitors en masse. You’re probably tracking acquisition sites, tallying up conversions and working to optimize your pages for the best success. But with all of that quantitative research, do you know about each individual user’s journey, and where they are struggling on your site? If not, you should check out one of our partners: SessionCam.

Jonathan Hildebrand, Brooks Bell’s Sr. Director of UX & Design, spoke at SessionCam’s user conference last week in Chicago. If you’re unfamiliar with SessionCam, the company began with a mission of building the best session replay solution on the market.  Over time the solution has grown into a fully-fledged behavioral analytics solution including heatmaps, conversion funnels, form analytics and more.

We’ve been blown away by the machine learning algorithms which identify signs of customer struggle and frustration on a website.  We sat down with Jonathan to ask him for a couple takeaways from the event.

As a UX expert, what do you appreciate most about SessionCam?

Where SessionCam really shines is in the qualitative data it provides, which can uncover major hurdles on your site in ways that quantitative data could never reveal. SessionCam’s recordings allow customers to watch a complete play-by-play of a visitor’s experience on the site, whether it’s through a mobile device or desktop.

What about specific to testing?

From a testing perspective, SessionCam can be great for post-test analysis since it allows you to watch videos from the live test experiences. The Customer Struggle Score is also a great way to understand where problems are occurring.

Any interesting case studies?

Definitely. One that comes to mind is a retailer that has a buy online, pick up in store (BOPUS) program. They were using SessionCam to uncover the source of order mistakes. When there was an error at pickup, they would go back and watch that customer’s online session to see if a problem occurred during the online order process and determine if there were any improvements they could make.

And you only need to check out their website to see the kind of value that SessionCam has added to many of the world’s leading brands.

If you’re interested in finding out more about SessionCam, give us a shout.

The post What are your website visitors doing? appeared first on Brooks Bell.

Who’s Hiring in October?

 Here are our picks: A/B Testing & Personalization Analyst – Barnes and Noble is looking for a candidate in New York to “execute critical tests and personalization initiatives alongside business units such as merchandising, UX, creative, editorial and technology to ensure the most optimum experience for customers to drive greater conversions.” Analyst, eCommerce Intelligence – Join […]

The post Who’s Hiring in October? appeared first on Brooks Bell.


Here are our picks:

A/B Testing & Personalization Analyst – Barnes and Noble is looking for a candidate in New York to “execute critical tests and personalization initiatives alongside business units such as merchandising, UX, creative, editorial and technology to ensure the most optimum experience for customers to drive greater conversions.”

Analyst, eCommerce Intelligence – Join the Perrigo’s Allegan Mi Facility in Michigan to help drive account specific eCommerce Intelligence strategy development, implementation, and maintenance along with providing analytic support to eCommerce Marketing enabling them to develop best in class go to market strategies.

Senior Data Scientist, SEM – “Using data science techniques that include predictive modeling and machine learning,” TripAdvisor is looking for someone in Needham, Massachusetts to focus on “optimizing SEM campaigns in such areas as auction bid optimization, user-based targeting, remarketing and landing page optimization.”

Marketing Manager (Strategy & Analytics) – GrubHub is looking for a Manager in New York, to “support the Marketing Strategy and Analytics team by analyzing A/B tests, managing budget, building key-metric dashboards, overseeing ETL data pipelines, and analyzing trends.”

Marketing Personalization Manager – In Bolingbrook, Illinois Ulta Beauty is looking for Manager of Personalization Strategy to “develop strategies and tactics for leveraging the customer data collected through the loyalty program, Ultamate Rewards, to deliver personalized, relevant communications across all channels that foster long-term loyalty and drive enterprise sales.”

JavaScript Developer – Brooks Bell is looking for a front-end JavaScript Developer with experience in modern technologies like REACT, AngularJS and Backbone.  This role will work closely with the execution and consulting teams in Raleigh, North Carolina coding test variations on complex sites along with interacting directly with clients.

Personalization Manager – Rooms To Go is seeking a Personalization Manager in Atlanta, Georgia.  “This position develops and executes e-commerce personalization strategies based on real-time data in partnership with analytics, email, merchandising, content, other marketing teams and business units.”

Manager Digital Optimization – Mayo Clinic is looking for a candidate is Rochester, New York to be “responsible for developing, implementing and leading the Digital conversion rate optimization program with AB and MVT testing discipline, focused on data-driven continuous improvement of Mayo Clinic’s™ digital customer experience.

Data Analyst, Personalized Customer Experience – Join Bose Corporation in Framingham, Massachusetts as a member of the Personalized Customer Experience (PCE) program Insights for Action (IFA) team and collaborate with business and technical stakeholders to produce reports and dashboards to drive decision making.

Experience Center, Front End Developer, Senior Associate – PwC is looking for a Senior Associate to join their team in New York to help clients change customer behaviors through the experience journey from strategy to implementation bringing interactive tools in the form of mobile apps, websites, or other digital platforms.

 

Have a job posting within the testing space?  Send it to us and we’ll post it in next month’s blog!

 

The post Who’s Hiring in October? appeared first on Brooks Bell.

Free Resource: How to Staff Your Testing Program to Meet Your Velocity Goals

Say you’re getting some work done to your home—you wouldn’t want plumbers painting your walls, right? What about electricians doing your landscaping? Or HVAC technicians repairing your roof? The same thing goes for your experimentation program. When your budget is tight, it can be tempting to have members of your team take on multiple roles. […]

The post Free Resource: How to Staff Your Testing Program to Meet Your Velocity Goals appeared first on Brooks Bell.

Say you’re getting some work done to your home—you wouldn’t want plumbers painting your walls, right? What about electricians doing your landscaping? Or HVAC technicians repairing your roof?

The same thing goes for your experimentation program.

When your budget is tight, it can be tempting to have members of your team take on multiple roles. For example, your analysts might be okay with managing the entire process of launching a test, developers could do full QA on their own tests, and project managers could oversee both test strategy and the tactical work required to execute tests.

But here’s why this is a bad idea: if you want something done right, you need to not only make sure you have the right people on the job, but also the right amount of people doing the work.

As you finalize your plans for 2019, we’ve put together a free guide to staffing your testing team according to the number of tests you want to launch per month. Download it today.


 

The post Free Resource: How to Staff Your Testing Program to Meet Your Velocity Goals appeared first on Brooks Bell.

The Expert’s Guide to A/B Testing During the Holiday Season

In 2016, online spending topped in-store shopping for the first time ever. That trend continued in 2017, with Adobe Digital Insights reporting that 2017 holiday sales surpassed $91.7 billion, marking 11% YoY growth. Peak season offers peak opportunities for experimentation programs. Increased traffic and conversion rates open the door for higher velocity, shorter durations, and lower minimum […]

The post The Expert’s Guide to A/B Testing During the Holiday Season appeared first on Brooks Bell.

In 2016, online spending topped in-store shopping for the first time ever. That trend continued in 2017, with Adobe Digital Insights reporting that 2017 holiday sales surpassed $91.7 billion, marking 11% YoY growth.

Peak season offers peak opportunities for experimentation programs. Increased traffic and conversion rates open the door for higher velocity, shorter durations, and lower minimum detectable lifts without compromising statistical significance.

If you haven’t already created your experimentation strategy, the time is now. But here are some essential factors to consider while creating your holiday testing game plan.

Maximize Your Holiday Window
Thanksgiving Day kicks off the peak holiday season, which continues through December 23. If you know your holiday window and website traffic patterns and expectations, you’ve got what it takes to take full advantage of this opportunity.

It can get complicated, but here’s a simple way to start:

  1. Define your holiday window. Consult past data to determine when traffic and conversion increases start and stop.
  2. Layer in the changes your organization is forecasting over last year. For example, one of our clients is expecting a five percent increase in traffic over last year’s holiday season. That intel is reflected in our traffic assumptions.
  3. Start your roadmap with the most valuable pages so that early wins can positively impact the rest of the holiday season. Create a punch list of pages with this in mind.
  4. Use traffic assumptions, desired statistical significance, and minimum detectable lift to determine the sample size and duration of tests.
  5. Continue this process to fill the window of time. Use these dates to mobilize your team, communicating key dates of test strategy kickoff, when tests will move into development, when they will launch and end and when results will be shared.

Communication is Critical
Since the holiday season represents a large portion of annual revenue, stress and emotions run high. As a result, it’s important to create your communication plan in advance. Determine who your stakeholders are, the optimal frequency of updates and what information needs to be shared. This isn’t the time for surprises or big reveals, so plan to devote a chunk of time to telling the story of your program and communicating its value.

The Weather Outside May Not Be the Only Freeze You’re Experiencing

Some organizations implement a freeze on development code updates and changes during the holiday season to avoid the risk of broken digital experiences or performance disruptions.  Get acclimated with your company’s approach so you can have a plan for implementing winning test programs.

The ideal scenario is to push winners immediately into production. Based on years of experience with enterprise clients, Brooks Bell strongly advocates this approach so you can maximize the impact of that winning test.

If production updates aren’t on the table because of a code freeze, don’t immediately jump to the decision to push the winner to 100 percent through your testing tool. Though it sounds like the best way to manage through code freezes, it could cause delays and create an undesirable experience. Before you make the decision how to handle, get your organization’s development experts involved to help you evaluate the risks and rewards.

Holiday Shoppers are Different
Think about your own shopping behaviors during the holidays compared to the rest of the year.

When I’m shopping during the holidays, I find myself on a mission to knock out my shopping list. As the countdown clock ticks away in my brain (and often literally on websites), I have a very real and intense sense of urgency. For me, customer confidence indicators, obvious savings and a clear and easy path to checkout are the ticket.

During the rest of the year, shopping is more leisurely for me and allows time for more browsing and consideration. I may even visit a website a few times before making a purchase. I zoom in on product details. I read customer reviews. I have more time, and the only restrictions are my own.

I’m the same person but have a very different mindset. The same goes for your customers. Keep this in mind as you develop your holiday testing roadmap.

Here are four tips to help ensure your holiday experimentation wins continue to add business value:

  • Keep it simple. As illustrated in my example above, successful holiday strategies are frequently based on a streamlined path to purchase, removing any friction and creating a sense of urgency and scarcity.
  • Test your hypotheses again after the holiday season. Do these experiences still produce a conversion lift when the holiday rush isn’t in full effect? If not, it’s okay! It’s an important learning you can use to build your Holiday/Non-Holiday playbook to make each holiday season better than the last.
  • Know your “Out of Stock” strategy. Regardless of what changes you make to your Product Detail page, nothing zaps excitement out of a customer experience faster than something being Out of Stock. Understand how your site handles Out of Stock messages, such as using red copy or suggesting alternate options. If it’s less than optimal, do some early testing to determine the most effective messaging. If your site includes a lot of Out of Stock product, it’s even more important to make sure it‘s been optimized.
  • Document your findings. Carve out time to tell the story of your testing through the chaos of increased velocity. The data and insights will be helpful after the rush and can greatly influence your future program success. Be sure to look at new, returning and loyal segments, and evaluate the differences in their holiday and non-holiday shopping behaviors.

For more intel on how to make the most of the merry months ahead, download our white paper, “5 Testing Tips for the Holidays.”

Need help developing a game plan for holiday testing? Contact us today!

The post The Expert’s Guide to A/B Testing During the Holiday Season appeared first on Brooks Bell.

Changing Your A/B Testing Software? Read These Tips First.

With the number of testing and personalization tools available, it can be difficult to choose one to invest in. But once you’ve already selected a software, making the decision to transition to a new tool altogether can feel overwhelming. But this happens quite often. For many clients, cost is often the deciding factor in making […]

The post Changing Your A/B Testing Software? Read These Tips First. appeared first on Brooks Bell.

With the number of testing and personalization tools available, it can be difficult to choose one to invest in. But once you’ve already selected a software, making the decision to transition to a new tool altogether can feel overwhelming.

But this happens quite often. For many clients, cost is often the deciding factor in making the decision to switch testing tools–there are a few testing tools that offer similar capabilities at a lower price point. On the flip side, if you’ve increased your program budget and capabilities, it may be time for an upgrade.

And although all testing tools offer similar functions, each has unique features that are important to consider. Personalization, for example, has become a point of focus for many testing programs – perhaps you’re interested in transitioning to a tool such as Evergage or Dynamic Yield that puts personalization at the forefront. Or your testing program has enough velocity to run multiple experiments simultaneously, and you feel you’d make good use of Optimizely’s built-in mutually exclusive experiments feature. Maybe your company uses other Adobe products, like Adobe Experience Manager, so you feel Adobe Target is a good fit.

Regardless of which tool you select, once you select a new software–the next major obstacle is implementing it. Here are our tips for going about the process:

First, examine your testing roadmap.

Take inventory of the tests that will be running close to the date when you plan to stop using your previous tool. Make sure they will have reached significance and be ready to be turned off before you lose access. 

If your budget allows for it, we recommend giving your team a period of time where both tools are available. This will ensure your testing cadence isn’t affected while your team gets up to speed on using the new tool and allows you to transition more seamlessly – you’ll be able to let current tests run their course in the old tool while launching new ones in the new tool.

Then, test your testing software.

While you might be excited to dive in and start launching tests left and right, it’s important to take the time to ensure your new tool is implemented correctly.

Run a QA test that visually changes the page to check that the code is being delivered and the flicker looks reasonable. If there are a lot of flickers, you may need to move the testing tool tag higher up in the head of your HTML.

We also recommend running a live test without visual changes, just for the purpose of checking metrics. This enables your analyst to see that metrics are being tracked correctly within the testing tool, or if you’re using an outside analytics tool, that those metrics are being passed accurately to it. 

Once you’ve confirmed that visual changes are showing up as expected and metrics are tracking accurately, you’re ready to start using your new tool!

Switching testing software comes with its challenges. However, in the right circumstance, switching can offer substantial benefits to your testing program. Taking the time to pinpoint your reasons for switching, plan your testing roadmap carefully around the transition, and having patience as the new tool is implemented will ensure your tool transition goes smoothly.


Brooks Bell has over 15 years of experience working with enterprise brands to establish and scale their experimentation programs. We take a holistic approach to our technical diagnostics and analytics services, providing technology and data recommendations based on your business, your goals, your team, and your unique challenges.

What can Brooks Bell do for you?
✓   Clean, organize and centralize your customer data.
✓   Help you select the right a/b testing and personalization tools.
✓   Ensure your tools and systems integrate with one another.
✓   Train your developers and analysts.

Contact us to learn more.

The post Changing Your A/B Testing Software? Read These Tips First. appeared first on Brooks Bell.