Inherent costs of A/B testing: limited risk results in limited gains

I’ve already done a detailed breakdown of costs & benefits in A/B testing as well as the risks and rewards and how A/B testing is essentially a risk management solution. In this short installment I’d like to focus on the trade-off betwe…

I’ve already done a detailed breakdown of costs & benefits in A/B testing as well as the risks and rewards and how A/B testing is essentially a risk management solution. In this short installment I’d like to focus on the trade-off between limiting the downside and restricting the upside which is present in all risk management […] Read More...

Results From Our Latest A/B Test: Here’s The New VWO Logo!

Over the past 8 years, we’ve made some key (and some minor) changes to the look and feel of our brand. Around this time last year, we revamped our website for the launch of VWO Conversion Optimization Platform™. As an organization that thrives on a culture of experimentation, we are always looking into data to […]

The post Results From Our Latest A/B Test: Here’s The New VWO Logo! appeared first on Blog.

Over the past 8 years, we’ve made some key (and some minor) changes to the look and feel of our brand. Around this time last year, we revamped our website for the launch of VWO Conversion Optimization Platform™.

As an organization that thrives on a culture of experimentation, we are always looking into data to discover insights for optimization. By turning our opinions into hypotheses, we test changes for almost everything which could have a significant impact on the business, and then derive the next logical step. Based on this simple framework, we recently made a minor change to the VWO logo. Before we delve further into the hypothesis behind this change, look at the logo in its full glory:

The Hypothesis: Making The Letters V, W, and O Prominent Will Improve Readability

In the beginning, our product was called Visual Website Optimizer. However, over the years, people (including us) fondly started abbreviating it to VWO. This is what the VWO logo looked like during this gradual change:

More recently, we dropped the accompanying text “Visual Website Optimizer” completely, and also started referring to our product as just “VWO.”

With this change, we realized that it would be hard for someone unfamiliar with our brand to read or understand our logo. We hypothesized that if the letters “V,” “W,” and “O” were made distinguishable, the brand name VWO would stand out more clearly.

The Test: Conducting an A/B/C Test to Choose a Winner

After the hypothesis was finalized, our design team created a new variation of the logo, per the new specification. Next, we decided to test the hypothesis by conducting extensive user testing through 5-second tests on UsabilityHub.

Five-second tests are a method of usability testing, where the participants are shown a visual for only 5 seconds, and then asked questions corresponding to it.

For our tests, we selected a sample of participants from across the globe, with varying demographics, location, and other attributes. They were showed the 3 variations of the logo—the existing one, the proposed one, and the one with VWO written as well-spaced plain text. Next, we asked the participants the question “What do you read?” to which they had to type in a response.

For the proposed logo, we got 90% of them answering “VWO”, as opposed to only 66% for the existing one. For the variation with VWO written as well-spaced text, the response was around 96%.

The Result: Reinforced Belief in the Potential of Testing

As an obvious next step, we decided to make this minor update to our logo which can now be seen to be live across all our digital properties. We’re proud of the fact that the basic tenets of experimentation continue to give direction to our efforts.

If it wasn’t for validating our initial, seemingly insignificant hypothesis, VWO wouldn’t have got a brand new identity. We strive to uphold this culture in our organization for the years to come.

What do you think of our new logo? Feel free to share your thoughts in the comments below.

The post Results From Our Latest A/B Test: Here’s The New VWO Logo! appeared first on Blog.

Representative samples and generalizability of A/B testing results

I see a nice trend in recent discussions on A/B testing: more and more people realize the need for proper statistical design and analysis which is a topic I hold dear as I’ve written dozens of articles and a few white-papers on. However, there ar…

I see a nice trend in recent discussions on A/B testing: more and more people realize the need for proper statistical design and analysis which is a topic I hold dear as I’ve written dozens of articles and a few white-papers on. However, there are cases in which statistical validity is discussed without consideration for […] Read More...

Designing successful A/B tests in Email Marketing

The process of A/B testing (a.k.a. online controlled experiments) is well-established in conversion rate optimization for all kinds of online properties and is widely used by e-commerce websites. On this blog I have already written in depth about the s…

The process of A/B testing (a.k.a. online controlled experiments) is well-established in conversion rate optimization for all kinds of online properties and is widely used by e-commerce websites. On this blog I have already written in depth about the statistics involved as well as the ROI calculations in terms of balancing risk and reward for […] Read More...

46 Conversion Rate Optimization Hacks

Conversion is the ultimate goal of a business’s online efforts. If your website has a glorious design and drives huge traffic but you’re still not getting enough leads, you need to get serious about conversion rate optimization and these 46 conversion …

Conversion is the ultimate goal of a business’s online efforts. If your website has a glorious design and drives huge traffic but you’re still not getting enough leads, you need to get serious about conversion rate optimization and these 46 conversion rate optimization hacks will help you get there. Conversion rate optimization is a systematic process...

The post 46 Conversion Rate Optimization Hacks appeared first on Conversion Sciences.

It’s The Little Things That Count With CRO

Big, bold, radical, redesign tests always move the needle the most. That is if you know your market, really really well.  But through the years, I’ve found that it’s the little things that count with CRO.  Even today I’m excited every time …

Big, bold, radical, redesign tests always move the needle the most. That is if you know your market, really really well.  But through the years, I’ve found that it’s the little things that count with CRO.  Even today I’m excited every time I find an interesting case study that highlights something that you wouldn’t have likely...

The post It’s The Little Things That Count With CRO appeared first on Conversion Sciences.

Make Testimonials More Persuasive

Don’t kid yourself into thinking that your viewers believe your testimonials are real and genuine. Any website that hosts its own testimonials has the opportunity to molest and curate its own testimonials and every viewer knows it. There are thre…

Don’t kid yourself into thinking that your viewers believe your testimonials are real and genuine. Any website that hosts its own testimonials has the opportunity to molest and curate its own testimonials and every viewer knows it. There are three rules to make testimonials more persuasive that you can start using today. If everybody knows...

The post Make Testimonials More Persuasive appeared first on Conversion Sciences.

Instantly Establish Credibility Online

When selling anything online people almost always prefer to buy from people and companies that they trust.  In fact, many times people are willing to pay more to buy from websites they trust. It’s one of those key components that can boost your online …

When selling anything online people almost always prefer to buy from people and companies that they trust.  In fact, many times people are willing to pay more to buy from websites they trust. It’s one of those key components that can boost your online conversion rate in multiples if done right so you need to...

The post Instantly Establish Credibility Online appeared first on Conversion Sciences.

Analysis of 115 A/B Tests: Average Lift is 4%, Most Lack Statistical Power

What can you learn from 115 publicly available A/B tests? Usually, not much, since in most cases you would be looking at case studies with very basic data about what was tested and the outcome of the A/B test. Confidence intervals, p-values and other m…

What can you learn from 115 publicly available A/B tests? Usually, not much, since in most cases you would be looking at case studies with very basic data about what was tested and the outcome of the A/B test. Confidence intervals, p-values and other measurements of uncertainty will often be missing, and when present they […] Read More...

Confidence Intervals & P-values for Percent Change / Relative Difference

In many controlled experiments, including online controlled experiments (a.k.a. A/B tests) the result of interest and hence the inference made is about the relative difference between the control and treatment group. In A/B testing as part of conversio…

In many controlled experiments, including online controlled experiments (a.k.a. A/B tests) the result of interest and hence the inference made is about the relative difference between the control and treatment group. In A/B testing as part of conversion rate optimization and in marketing experiments in general we use the term “percent lift” (“percentage lift”) while in […] Read More...