Bayesian Probability and Nonsensical Bayesian Statistics in A/B Testing

Many adherents of Bayesian methods put forth claims of superiority of Bayesian statistics and inference over the established frequentist approach based mainly on the supposedly intuitive nature of the Bayesian approach. Rational thinking or even human …

Many adherents of Bayesian methods put forth claims of superiority of Bayesian statistics and inference over the established frequentist approach based mainly on the supposedly intuitive nature of the Bayesian approach. Rational thinking or even human reasoning in general is Bayesian by nature according to some of them. Others argue that proper decision-making is inherently […] Read More...

The Perils of Using Google Analytics User Counts in A/B Testing

Many analysts, marketers, product managers, UX and CRO professionals nowadays rely on user counts provided by Google Analytics, Adobe Analytics, or similar tools, in order to perform various statistical analyses. Such analyses may involve the statistic…

Many analysts, marketers, product managers, UX and CRO professionals nowadays rely on user counts provided by Google Analytics, Adobe Analytics, or similar tools, in order to perform various statistical analyses. Such analyses may involve the statistical hypothesis tests and estimations part of A/B testing, and may also include regressions and predictive models (LTV, churn, etc.). […] Read More...

The Effect of Using Cardinality Estimates Like HyperLogLog in Statistical Analyses

This article will examine the effects of using the HyperLogLog++ (HLL++) cardinality estimation algorithm in applications where its output serves as input for statistical calculations. A prominent example of such a scenario can be found in online contr…

This article will examine the effects of using the HyperLogLog++ (HLL++) cardinality estimation algorithm in applications where its output serves as input for statistical calculations. A prominent example of such a scenario can be found in online controlled experiments (online A/B tests) where key performance measures are often based on the number of unique users, […] Read More...

Error Spending in Sequential Testing Explained

Sequential analysis of experimental data from A/B tests has been quite prominent in recent years due to the myriad of Bayesian solutions offered by big industry players. However, this type of sequential analysis is not sequential testing proper as thes…

Sequential analysis of experimental data from A/B tests has been quite prominent in recent years due to the myriad of Bayesian solutions offered by big industry players. However, this type of sequential analysis is not sequential testing proper as these solutions have generally abandoned the idea of testing and therefore error control, substituting it for […] Read More...

Frequentist vs Bayesian Inference

In this article I’m revisiting* the topic of frequentist vs Bayesian inference with specific focus on online A/B testing as usual. The present discussion easily generalizes to any area where we need to measure uncertainty while using data to guid…

In this article I’m revisiting* the topic of frequentist vs Bayesian inference with specific focus on online A/B testing as usual. The present discussion easily generalizes to any area where we need to measure uncertainty while using data to guide decision-making and/or business risk management. In particular, I will discuss each of the following five […] Read More...

Underpowered A/B Tests – Confusions, Myths, and Reality

In recent years a lot more CRO & A/B testing practitioners have started paying more attention to the statistical power of their online experiments, at least based on my observations. While this a positive development for which I hope I had contribu…

In recent years a lot more CRO & A/B testing practitioners have started paying more attention to the statistical power of their online experiments, at least based on my observations. While this a positive development for which I hope I had contributed somewhat, it comes with the inevitable confusions and misunderstandings surrounding a complex concept […] Read More...

The Perils of Poor Data Visualization in CRO & A/B Testing

As any UX & CRO expert should now, the way we present information matters a lot both in terms of how well it is understood and in terms of the probability that it will lead to the desired action. A/B testing calculators and other tools of the trade…

As any UX & CRO expert should now, the way we present information matters a lot both in terms of how well it is understood and in terms of the probability that it will lead to the desired action. A/B testing calculators and other tools of the trade are no exception and here I will […] Read More...

The Cost of Not A/B Testing – a Case Study

Most of the time when discussing A/B testing, regardless of context, we discuss costs such as the expense of running an experimentation program, of shipping ‘winners’ to production. Only rarely do I see references to the less obvious, but u…

Most of the time when discussing A/B testing, regardless of context, we discuss costs such as the expense of running an experimentation program, of shipping ‘winners’ to production. Only rarely do I see references to the less obvious, but usually more important costs in terms of opportunity cost (incurred during testing) and the cost of […] Read More...

A Well Balanced Content Personalization Diet: 3 New Years Resolutions to Increase Goal Conversions

Happy New Year, travel marketers! The beginning of January always brings its own kind of magic with resolutions and the opportunity to both reflect on the past year and look towards the next.  It’s also a time that, if I can be honest, is a little overwhelming with the pressure of setting life-changing goals. And… Read More

The post A Well Balanced Content Personalization Diet: <br/>3 New Years Resolutions to Increase Goal Conversions appeared first on Bound.

Happy New Year, travel marketers! The beginning of January always brings its own kind of magic with resolutions and the opportunity to both reflect on the past year and look towards the next.  It’s also a time that, if I can be honest, is a little overwhelming with the pressure of setting life-changing goals. And it’s not only personal goals! Working within the digital marketing space I feel that every other content piece is focused on “new year, new marketing strategy” resolutions that couldn’t be easier to implement – or so the articles read…

At Bound, we’re big believers in starting where you’re at, especially when it comes to personalization and your marketing strategy.  That’s why one of our resolutions this year is to focus on something that we know has an impact: optimizing our goal conversions

When it comes to our monthly content reports, few things give our Customer Success Managers more joy than seeing an increase in click through rates on goal related content pieces.  But as fun as these increases are to see, we are even more thrilled by increases in the goal conversions themselves. As we’ve become increasingly aware of the important relationship between clicks and conversions – and the very different stories each can highlight when they don’t align  – we’re excited to share our new Goal Dashboard and highlight three resolutions on increasing your conversions in 2020:

Read More (into your A/B tests):

When in doubt about your content, run an A/B Test!  While click through rates can certainly highlight your audience’s preferences for the imagery, copy or CTA, how do you account for the content’s impact on the actual conversion?  Within the new Goal Dashboard, you can now compare conversion rates against your campaigns, segments and pieces of content, allowing for a deeper level of insight. We recently took a closer look at an eNewsletter related A/B test we have been running with a DMO.  Month over month, we found that one content piece had consistently less clicks than the other. However, in comparing the conversion rates between the two pieces, we saw that the content piece with a lower CTR had a considerably higher conversion rate. This comparison helped us see the value of a content piece we might have otherwise removed and will help inform future A/B tests.

Exercise (your understanding of your Mobile and Desktop visitors differences):

As we’ve written about before, there are many things to take into consideration when creating content for your Desktop and Mobile visitors.  Goal conversions are no different, especially given that our Mobile visitors are often less likely to convert. Within the new Goal Dashboard, we can now dive into the conversion rates for our different segments across campaigns, allowing us to compare, for example, fly-ins served to desktop visitors and banners served to mobile audiences.  Layering in this insight can help us develop content best suited for each of our unique visitors groups.

Spend Less (time guessing how your content is performing):

Over the past few years, we’ve increasingly become fans of thoughtful “abandonment” content and the way these direct CTAs can increase conversions for visitors who have initiated, but not completed, a conversion goal.  While we often see this content with high CTRs, it can be challenging to determine how exactly this content contributes to the overall goal. Thankfully, our new Goal Dashboard takes the guesswork out of content creation and helps us see exactly which Abandonment content is best contributing to the goal. 

Our hope for your 2020 is that your conversion related content is directly increasing your goal conversions (leaving you with more time to increase engagement for your ad visitors!)   Knowing that goal conversions are a vital piece to understanding your visitors intent to travel, we’re excited that our new Goal Dashboard will bring new awareness and insight this year.  Cheers to you and your increased conversions!

Want to learn more about the Goal Dashboard or personalizing to increase your conversions?  We’d love to chat with you and hear all about your 2020 marketing resolutions!

The post A Well Balanced Content Personalization Diet: <br/>3 New Years Resolutions to Increase Goal Conversions appeared first on Bound.

What Can You Learn From Running an A/B Test for 2 Years?

We just concluded an A/B test on Analytics-Toolkit.com that has been left to run for just over 2 years. And it failed, as in failing to demonstrate a statistically significant effect based on the significance threshold it was designed for. Has it been …

We just concluded an A/B test on Analytics-Toolkit.com that has been left to run for just over 2 years. And it failed, as in failing to demonstrate a statistically significant effect based on the significance threshold it was designed for. Has it been a waste of time, though, or can we actually learn something from […] Read More...