Part 2: Our Top Takeaways from Click Summit 2018

Last week, we shared the first of many takeaways from Click Summit 2018, our annual conference for professionals in digital experimentation and personalization. This week, we’re back with more insights from each impactful conversation, inspired by this year’s edition of Clickaways. 1. Manage the three P’s of scaling your testing program: people, process, prioritization. Many companies […]

The post Part 2: Our Top Takeaways from Click Summit 2018 appeared first on Brooks Bell.

Last week, we shared the first of many takeaways from Click Summit 2018, our annual conference for professionals in digital experimentation and personalization. This week, we’re back with more insights from each impactful conversation, inspired by this year’s edition of Clickaways.

1. Manage the three P’s of scaling your testing program: people, process, prioritization.

Many companies have found it more effective to establish a dedicated optimization team rather than having these duties dispersed across the organization. However, if that’s not possible for you, let your Center of Excellence take the lead on defining key processes, training and developing a maturity model to determine when each team is ready to start testing.

Develop a formal process for submitting, presenting, prioritizing and executing new testing ideas. Using various automation technologies can further simplify these steps.

Additionally, agree to one source of truth for your test results across multiple platforms. Companies that have various groups looking at different data sources struggle to establish the necessary credibility to scale their programs. This is one area where a knowledge platform that houses testing results, insights and ideas (like Brooks Bell’s Illuminate platform, or Optimizely’s Program Management) can help.

Finally, growing your experimentation program comes with the expectation of more tests, executed faster. When determining your velocity goals, be sure to consider quality over quantity. Always prioritize running a few, quality tests over many, low-impact tests.

2. Personalization and optimization teams should remain separate functions with connected but distinct goals.

Personalization is a worthwhile investment for any online industry, but it has to be adopted as a company-wide strategy in order to ensure you’re delivering a consistent customer experience.

To get the most out of your investment, establish a separate personalization team to run your program rather than looking to your existing experimentation team. Here are a few reasons for this: First, personalization is a longer-term strategy and “wins” occur at a much slower rate. Additionally, while there are similarities between A/B testing and personalization technologies, the questions you ask and the answers you get are very different.

Finally, running split tests is inherently easier and faster than implementing personalization. So long as your team is overseeing both functions, they’re likely to focus more on testing than personalization.

3. Focus on organizational outputs and customer insights, not just test outcomes.



Oftentimes, experimentation professionals find themselves nearest to the customer. Sure, you may not speak with them directly, but your work can have a direct effect on your customers’ experience and brand perception. That’s a lot of power, but also a lot of opportunity.

So here’s the challenge: Go beyond simple tests like button color or check out features and consider the bigger picture. Use testing to seek out insights that would be useful for other departments within your organization.

Here at Brooks Bell, we have our own framework for doing this (and we’d be happy to tell you about it). In lieu of our services, we’d encourage you to take a step back from test outcomes, spot trends and use these to develop testable customer theories.

Developing a customer theory requires you to conduct a deeper interpretation of your results–so don’t do it alone. Look to your working team to brainstorm customer theories and additional tests to validate or invalidate those. Bring in additional data sources like NPS, VOC or qualitative research to paint a more detailed picture of your customers.

Doing this can have huge implications for your customers, your experimentation program and your brand overall.

4. Build a program that strikes the perfect balance of innovation and ROI.

In order for creativity to flourish within your experimentation program, you have to establish clear goals. These are used as a framework within which your team can look for opportunities to innovate.

Develop a process for brainstorming test ideas that encourages participation and creative thinking, like using Post-It notes.



Finally, demonstrate a willingness to take calculated risks in order to make room for creativity in your optimization strategy. There is always something to be learned from negative or flat results.

Like the information in this post? Download this year’s Clickaways to access more tips, tricks and ideas from Click Summit 2018.

The post Part 2: Our Top Takeaways from Click Summit 2018 appeared first on Brooks Bell.

Testing Your App Listing in the Google Play Store

Recently, while attending a native app session at our annual conference, Click Summit, it was brought to my attention that not many people know about the ability to run A/B tests on their Google Play Store app listing.   Testing your store listing can be an untapped area for gaining key insights about your customers and […]

The post Testing Your App Listing in the Google Play Store appeared first on Brooks Bell.

Recently, while attending a native app session at our annual conference, Click Summit, it was brought to my attention that not many people know about the ability to run A/B tests on their Google Play Store app listing.  

Testing your store listing can be an untapped area for gaining key insights about your customers and increasing app installs. Additionally, the insights you gain by testing your Google Play Store listings could be transferable to your Apple App Store listing as well.

Within the Google Play Console, there’s a little known tool that will allow you to A/B test your app listing called “store listing experiments.” This can be found under the “store presence” menu item.  

There is no need for a technical resources or technical knowledge as the technology is built right into the Google Play Console. The Store Listing Experiments feature allows you to A/B Test six different attributes of the store listing: Hi-res icon, Feature Graphic, Screenshots, Promo Video, Short Description and Long Description. Tests can include all of these in combination or individually. You can run tests globally (graphics only) or localized (text and graphics). Note that you are limited to 3 variations in a test.

The analytics and reporting is all housed within the Google Play Console and unfortunately, cannot be exported. Three metrics area automatically tracked: Installs on active devices, installs by user, uninstalls by user. Results are measured at a 90% confidence interval.

For more details, check out Google’s step by step documentation.

When it comes to experimentation, Brooks Bell is happy to lend our expertise to help your optimization program expand its reach, capabilities, and impact. This can include testing store listings, to landing pages, to check out experiences and more. If you’re interested in learning more about Brooks Bell and how we can help optimize your web experiences, contact us today.

The post Testing Your App Listing in the Google Play Store appeared first on Brooks Bell.

Four Challenges to Building & Scaling Your Experimentation Program, Solved with Illuminate

Since our company was founded in 2003, we’ve worked with clients facing a multitude of challenges in establishing their experimentation programs: team turnover leading to loss of institutional testing knowledge; test results lost in a sea of monthly reports or lengthy spreadsheets; disagreements with creative, engineering or analytics teams; and senior-level executives that just don’t […]

The post Four Challenges to Building & Scaling Your Experimentation Program, Solved with Illuminate appeared first on Brooks Bell.

Since our company was founded in 2003, we’ve worked with clients facing a multitude of challenges in establishing their experimentation programs: team turnover leading to loss of institutional testing knowledge; test results lost in a sea of monthly reports or lengthy spreadsheets; disagreements with creative, engineering or analytics teams; and senior-level executives that just don’t understand the true value of testing–just to name a few.

Over time, these challenges became so prevalent that we started offering a group “therapy” session at Click Summit, our annual conference for experimentation professionals.  Here, attendees are free to voice their feelings and frustrations in a small group-style conversation. Over time, this has become one of our more popular sessions at Click Summit (and if you were wondering, yes, there are mimosas involved).

But talking it out only does so much, and after nearly fifteen years of working in this space, we knew we could to do more. So last week, we launched Illuminate, the world’s first customer insights software for enterprise experimentation teams. 

Illuminate was built to address some key challenges faced by clients in building and scaling their experimentation programs. We’ve outlined a few of them here:

Challenge 1: Lack of institutional testing knowledge 

You’ve heard of him. Y’know–the guy. The guy who spent years establishing your company’s experimentation program; the guy who was a walking encyclopedia of your testing history; the guy who also unceremoniously peaced out right before you joined the team–taking with him years of institutional knowledge about which tests had been run, and what was learned.

Illuminate solves for this by providing an organized, searchable history of all your tests–and along with them, the winners and any key learnings. It also integrates directly with Optimizely, making it easy to sync your test results and KPIs.

Challenge 2: Building a testing culture

How does the saying go? If you give a man a fish, you feed him for a day. But if you teach a man to A/B test, you’re not only teaching him how to fish, but how to catch more fish in the right place, and at the right time–all the time…or something like that.

We built Illuminate specifically for teams that want to use testing to uncover high-impact, meaningful insights about their customers, and share these insights with other departments that would likely benefit from their data.  

With Illuminate’s case study generator, you can tell the story of your test in a way that is both detailed and easy to understand (for even your most testing-illiterate colleagues).

Illuminate also invites key stakeholders to participate in your test brainstorming sessions using Illuminate’s guided brainstorming feature. These tools are framed using the Brooks Bell test ideation method, our proprietary process for coming up with new test opportunities. This is one of many processes that we’ve developed and refined over the years. 

The result? Happier customers and an optimization-savvy organization that is more aligned around your test results. Also, lots and lots of fish.

Challenge 3:  Learning about your customers

Reality check: So long as making changes to your website requires working with creative, brand, engineering and analytics (or the like), it’s likely you’re always going to face battles over who really owns the website. And so long as technology and UX best practices keep changing, you’re always going to be working under threat of the next redesign.

Illuminate solves for this in a few different ways. In addition to the case study and brainstorming features we’ve already mentioned, Illuminate also was built to enable deep customer learning.

In our consulting practice, we use the Brooks Bell Insight Framework to help our clients connect testing outcomes to high-impact insights about their customers. Illuminate codifies this method of thinking in software, empowering you and your team to develop key customer insights that are not only transferable across your organization, but also able to withstand the test of time (and redesign).

 

Challenge 4: Communicating ROI of experimentation

Slashed budgets. Under-resourced teams. Rapidly changing goal posts. These pressures can bother anyone, but combined with constantly having to justify your team’s existence to executives who don’t speak optimization…well, we’d understand if you’re at risk of going Britney-Spears-circa-2007-levels of crazy.

With Illuminate’s custom dashboard and one-click reporting tools, you can communicate and share your program’s impact on the metrics that are most important to your company, and senior management. 

 

Illuminate is the culmination of nearly 15 years of experience working with clients on experimentation strategy, ideation, execution and deep customer learning. Illuminate is currently launching in private beta, with a planned public roll out later this year.

Interested in seeing Illuminate in action? Request a demo today.

The post Four Challenges to Building & Scaling Your Experimentation Program, Solved with Illuminate appeared first on Brooks Bell.

How to Use Personalization to Enhance Your Existing Optimization Program

We know an A/B test can lead to powerful insights. However, the information gained from traditional A/B tests tends to be focused on what’s best for the majority of users – not every individual user. That’s where personalization comes in. Personalization enables you to leverage the specific wants and needs of each individual user on […]

The post How to Use Personalization to Enhance Your Existing Optimization Program appeared first on Brooks Bell.

We know an A/B test can lead to powerful insights. However, the information gained from traditional A/B tests tends to be focused on what’s best for the majority of users – not every individual user. That’s where personalization comes in.

Personalization enables you to leverage the specific wants and needs of each individual user on your site. This can lead to even more substantial results–higher conversion rates, deeper engagement with your site, and increased revenue.

Many of the traditional testing tools–Adobe Target, Optimizely and Maxymiser–have personalization capabilities available. There has also been an emergence of companies like Dynamic Yield and Evergage, which offer personalization technology as their core focus.

As technology in this space improves, personalization has become a major focus for many Brooks Bell clients. However, the question we’re often asked is not whether to implement personalization alongside existing optimization efforts – rather, its how to do this.

Luckily, there are many ways to do just that. For the purposes of this blog post, we’ve outlined two relatively simple strategies for implementing personalization alongside your existing optimization program.

Strategy 1: Rule-based targeting

Rule based targeting is a personalization technique that’s available on most A/B testing platforms. Instead of targeting all users, you select a specific segment of users to target an experience to: new or returning users; mobile or desktop users; or users in a specific location.

Because these different types of users are interacting with your site differently, you’ll likely see higher returns by personalizing your content to each group.

You can also apply rule-based targeting after running a traditional A/B test, by breaking down your results by those specific user segments. In doing so, you may find that a “winning” homepage experience performed very well among new users, but was flat for returning users.

Though pushing the winning variation live to all users would increase revenue, you might see a bigger increase if you were to push it live to new users only. This gives way to additional opportunities to test different strategies for returning visitors.

Strategy 2: Predictive personalization

Many testing platforms now offer predictive personalization, which works in real time to learn which experiences are ideal for certain types of users.

A predictive personalization “test” runs indefinitely – and adjusts as users’ preferences change over time, showing the optimal experience to each user.

Predictive targeting technology is exciting for many reasons. It accounts for the fact that a winner from a year ago might not be the best option for your users now.  

The technology also makes it easier to figure out the best option for short term website changes, like a holiday promotion–for which A/B testing is not a viable option due to time constraints.

Additionally, having the ability to step back and leave the analysis to the computer – instead of spending the time analyzing data yourself – is a huge benefit to experimentation professionals and the companies they work for.  

There are, of course, potential pitfalls to this form of personalization.

When you run a traditional A/B test with a clear winner across all users, it’s easy to make the decision to build the winning code into your site. However, with predictive personalization, you may have many different versions of a page for different segments of users, and continue relying on the testing tool to deliver the code, never building it into your site.

This can be risky for a few reasons: it can increase load time; and if, over time, other updates are made to your site, those updates could break the experience.

Additionally, you’ll also want to make sure you trust that the machine learning algorithms are actually making the best decisions for your users. To that end, many platforms offer a control experience which segments users randomly. You can then compare metrics from the control against the personalized segments to ensure the algorithm is working optimally.

Personalization offers the opportunity to gain new insights about your users and deliver the most valuable content for each individual. Incorporating personalization into your testing program is certainly worth the investment, with the potential for huge rewards.

At Brooks Bell, our Personalization Jumpstart program enables enterprise optimization teams to incorporate and scale personalization strategies into their existing optimization programs. To learn more about our services, contact us today.

The post How to Use Personalization to Enhance Your Existing Optimization Program appeared first on Brooks Bell.