“Statistics are no substitute for judgement”
LOR
LOR
Every mobile app professional today uses mobile app analytics to track their app. Yet there are some key elements in their analytics workflows that are naturally flawed. The solution is out there, and you might have missed it.
The flaw, and a fairly bi…
Every mobile app professional today uses mobile app analytics to track their app. Yet there are some key elements in their analytics workflows that are naturally flawed. The solution is out there, and you might have missed it.
The flaw, and a fairly big one at that, is in the fact that app analytics pros sometimes focus solely on quantitative analytics to optimize their apps. Don't take this the wrong way – quantitative analytics is a very important part of app optimization. It can tell you if people are leaving your app too soon; if they're not completing the signup process, how often users launch your app, and things like that. However, it won't give you the answer as to why people are doing it, or why certain unwanted things are happening in your app. And that's the general flaw.
The answer lies in expanding your arsenal – adding qualitative analytics to your workflow. Together with quantitative analytics, these tools can help you form a complete picture of your app and its users, identify the main pain points and user experience friction, helping you optimize your app and deliver the ultimate product.
So today, you are going to learn how to totally revamp your analytics workflow using qualitative analytics, and why you should do it in the first place. You'll read about the fundamentals of qualitative analytics, and how it improves one's analysis accuracy, troubleshooting and overall workflows. And finally, you'll find two main ways to use qualitative analytics which can help you turn your app(s) into mobile powerhouse.
Qualitative analytics can be split into two main features: heatmaps and user session recordings. Let's dig a little deeper to see what they do.
This tool gathers all of the gestures a user does in every screen of the app, like tapping, double-tapping, or swiping. It then aggregates these interactions to create a visual touch heatmap. This allows app pros to quickly and easily see where the majority of users are actually interacting with the app, as well as which parts of an app are being left out.
Another important advantage of touch heatmaps is the ability to see where users are trying to interact, without the app responding. These are called unresponsive gestures, and they are extremely important because they're very annoying and could severely hurt the user experience.
Unresponsive gestures can be an indication of a bug or a flaw in the design of your user interface. Also, it could show you how your users think they should move through the app. As you might imagine, being bug-free and intuitive are two very important parts of a successful app, which is why tackling unresponsive gestures can make a huge difference in your app analytics workflow.
User session recordings are a fundamental feature of qualitative app analytics. They allow app pros to see just what their users are doing, as they are progressing through the app. That means every interaction, every sequence of events, on every screen in the app, gets recorded. This allows app pros an unbiased, unaltered view of the user experience.
With such a tool, you'll be able to better understand why users sometimes abandon an app too soon, why they decide to use it once and never again, or even why the app crashes on a particular platform or device.
Through video recordings, it becomes much easier to get to the very core of any problem your app might be experiencing. A single recording can shine light on a problem many users are struggling with. Obviously, the tool doesn't just mindlessly record everything – app pros can choose different screens, different demographics, mobile devices or their operating systems to record from. It is also important for this tool to work quietly in the background and not leave a strain on the app's performance.
Qualitative analytics is too big of a field to be covered in a single article. Those looking to learn more might as well take a free course via this link. For all others, it's time to discuss two main workflows where they can be used - ‘Data-fueled Optimization' and ‘Proactive Troubleshooting'.
Both qualitative analytics and quantitative analytics tools are 'attacking' the same problem from different angles. While both are tasked with helping app pros optimize their mobile products, they have different, even opposite approaches to the solution. That makes them an insanely powerful combo, when used together.
Employing inherently opposite systems to tackle the same problem at the same time helps app pros form a complete picture of their app and how it behaves 'in the wild'. While quantitative analytics can be used as an alarm system, notifying app pros to a condition or a problem, qualitative analytics can be used to analyze the problem more thoroughly.
For example, using quantitative analytics tools you are alerted to the fact that a third of your visitors abandon their shopping cart just before making a purchase. You identify it as a problem, but cannot answer the question as to why this is happening.
With tools like user session recordings, you can streamline your optimization workflow and learn exactly where the problem lies. You could try to fix a problem without insights from qualitative data, but you'll essentially be "blindly taking a stab".
By watching a few user session recordings, you realize that the required registration process prior to making a purchase is simply too long. Users come halfway through it and just quit. By shortening the registration process and making checkout faster, you can lower the abandonment rate. Alert, investigate, resolve. This flow can easily become your "lather, rinse, repeat."
Can you truly be proactive in your troubleshooting? Especially when using analytics? Well, if you rely solely on quantitative analytics, probably not. After all, you need a certain amount of users to actually be using the app for some time before you can get any numbers out, like app abandonment rates or crash rates. Only then will you be able to do anything, and at that point – you're only reacting to a problem already present. With qualitative analytics, that's not the case.
By watching real user session recordings and keeping an eye out on touch heatmaps, you can spot issues with your app's usability or user experience long before a bigger issue arises, therefore proactively troubleshooting any problems.
For example, by watching user session recordings you notice that people are trying to log into Twitter through your app and post a tweet. However, as soon as they try to log in, the app crashes. Some users decide to quit the app altogether. Spotting such an issue helps you fix your app before it witnesses a bigger fallout in new user retention.
Not being proactive about looking for bugs and crashes doesn't mean they won't happen – it means they might go longer unattended. By the time you spot them through quantitative analytics, they will have already hurt your user experience and probably pushed a few users your competitor's way.
They say new ideas are nothing more than old ideas with a fresh twist, and if that's true, than qualitative analytics are the ‘fresh twist' of mobile app analytics. Combining quantitative and qualitative analytics is a simple process that has incredible potency in terms of your workflows and app optimization. Plus, when you understand the reasons behind the numbers on your app, you are able to make crucial decisions with more confidence.
Have you ever wondered how divorce and marriage rates have trended over the last 150 years? Or what reasons husbands and wives give when getting a divorce? Fortunately these, and other questions, can be answered with data. The UK Office for National St…
Have you ever wondered how divorce and marriage rates have trended over the last 150 years? Or what reasons husbands and wives give when getting a divorce? Fortunately these, and other questions, can be answered with data. The UK Office for National Statistics make available two extremely interesting and rich datasets on marriages and divorces, providing data for the last 150 years.
Following the discovery of these datasets, I decided to uncover trends and patterns in the numbers, working with my colleague Lizzie Silvey. Two important questions were in our minds when exploring the data:
We discuss our findings in this article, but you can also drill down into the data using this interactive visualization that we created using Google Data Studio.
The ratio of petitioners has been stable since around 1974 (70% women and 30% men), the time at which both genders started having the same rights and divorce could be attained more easily.
In the charts below we see the trends for 'Adultery' and 'Unreasonable behaviour', the two most common reasons provided (out of five possible) - each line shows the number of divorces granted to the husband or wife for a specific reason.
In order to use Adultery grounds the petitioner must prove that the partner had sexual intercourse with someone else, which might not be simple. We can see in the chart that Adultery follows the exact same pattern for husbands and wives, but analyzing the statistics further we see that, on average, 40% of the adultery divorces are granted to husbands - since only 30% of total divorces are petitioned by husbands, it seems adultery is a particularly strong reason for men to file for a divorce.
The second chart, showing 'Unreasonable behaviour', is more enigmatic. While husbands were granted divorces in an increasing pace for behavioural reasons, and while the lines seem to be converging, there is a strange hump in the wives line. Why were wives granted a massive amount of divorces up to 1992 based on unreasonable behaviour? Could that be related to a “backlog” of cases of domestic violence (classified as a behavioural reason) that came to light after women could divorce based on those grounds more easily? Unfortunately we could not find data showing possible reasons for that.
When looking at the marriage and divorce trends since 1862, there were a few clear turning points.
The wars seemed to affect marriages quite significantly. Around the beginning of World War I & II we see spikes in marriages, maybe as a result of young men wanting to vow their love before going to fight. Then, during the wars, the marriages plunged as soldiers went away, and up again when they came back home.
As for divorces influenced by the wars, we can only look at World War II, as women had a limited ability to divorce after World War I. It seems the Matrimonial Causes Act 1937, which made other grounds legal (e.g. drunkenness and insanity), coupled with premature weddings (discussed above) and possibly a estrangement due to separation led to a spike in divorces starting in 1946 - who would have the heart to divorce in war times?
But what seems to be the strongest influence in divorces in the history of the UK is the Divorce Reform Act 1969 (link to PDF), which came into effect in 1971. This act states that divorce can be granted on the grounds that the marriage has irretrievably broken down, and it is not essential for either partner to prove an offense. While that explains the strong increase in divorce, we could not find a strong reason for the decline in marriages at the same time - we invite possible explanations in the comments section.
While we couldn't bring answers as to why trends are going in a certain direction and predict upcoming changes, we believe that the data can shed new light into the British society and family relations. Hopefully with new releases of data in the future we will also be able to dive deeper and answer more existential questions.
If you are interested in exploring the data further, check the interactive visualization, created with Google Data Studio, you will find more context and charts showing trends and pattern on marriage and divorce in the UK.
[Cross-posted from The Next Web]
Whether you are a marketer trying to persuade people, a technologist building a startup, or an executive making business decisions, data is your partner. You can use it to make better decisions and create insightful dat…
[Cross-posted from The Next Web]
Whether you are a marketer trying to persuade people, a technologist building a startup, or an executive making business decisions, data is your partner. You can use it to make better decisions and create insightful data stories inside and outside your company.
The first step is to accept your data relationship: you are partners forever. Once you understand that, there is an important consideration that will define how to tell your data stories: the context of where they live, which also defines the audience that will interact with them. In this post I will go through some important lessons I learned when visualizing and communicating data in and outside Google.
Data is no longer "next year's big thing", we have gone through that many times over and almost everyone accepts data as a valuable team member. But not everyone can understand and make use of it optimally, which means lots of decisions are still made based on intuition - if you don't believe me, check PwC's Global Data and Analytics Survey 2016, it shows some interesting numbers on how often managers use data during the decision-making process. Data education is a crooked road and we have a long journey ahead of us.
One of the reasons for that is similar to the well-known phenomenon called mathematical anxiety, where people are afraid of maths as a result of past difficulties and traumas. Every one of us have interacted with data analyses (at work, newspapers or academic research) that were created by unskilful communicators, people that might be amazing statisticians but lack the ability to convey the stories behind the numbers. That creates anxiety and could prevent professionals from even trying to understand data.
I believe the reason the data community is not growing like weeds is because professionals are not confident enough with numbers and charts. I have written about how to overcome the fear of analytics (and help others), here is a quick summary.
- Never mock people for not understanding a chart
- Take baby-steps towards numeracy
- Make analytics more fun
When you create a visualization you may affect other people both positively and negatively. If you create a complex and unintuitive visualization, you might be creating a phobia on other people, and they will forever hate numbers and stats. However, if you create a powerful and beautiful visualization, you might be persuading another mind to join the data visualization tribe.
Below are some ideas that might help you craft better data stories, both for businesses and in general.
There are many ways to communicate data, but choosing the right format will depend on where the data will be published or presented, the context. Is it a daily performance report or a quarterly result presentation? Or a behavioral analysis using web data? Or an interactive visualization showing global trends?
I'd like to break down data stories into two main branches: business reporting or analysis, and visualizing the world. These groups can show very different characteristics, so let's look into each separately.
I recently had the opportunity to interview Avinash Kaushik, Digital Marketing Evangelist at Google. In our conversation we discussed techniques to create great data stories, focusing on businesses. Avinash talked about his business framework See, Think, Do, Care and the role of data visualization during the decision making process.
We also discussed data visualization (see minute 11:08), and Avinash explains how not to make silly mistakes when using data in a business context. He makes the differentiation between three main types of visualizations:
Avinash differentiates between analysis "packed together" with a storyteller, which allow for more complex visualizations, and day-to-day reporting, which are supposed to stand on their own and help people make decisions by themselves.
Considering the data delivery circumstances is a great start when designing your visualizations as they will inform the presentation style and level of complexity that can be used. While every visualization should strive for simplicity, a daily report (and business visualizations in general) must be extremely clean and self-explanatory, as the data storyteller won't be there to help the decision maker.
Below is a quote by Avinash summarizing his views on how to succeed with data.
"On a business context, a data visualization has to do one job really well, and it has to answer the question ‘so what?' If your data doesn't answer the 'so what' question, and if there isn't a punchy insight that drives action, all you have is a customized data puke, it looks really nice but it serves no purpose. If you want to drive change, you have to get to the simplest possible way to present the data, and once you get to it ask the so what question. After you answer it, ask if it quantifies the opportunity, if it does you are going to win."
Luckily to our society, visualizations are increasingly used in a broader context, where the goal is not to understand the business or track performance, but to educate the public and change people's minds. There are some great examples of visualizations that make a difference, but probably the most famous is Hans Rosling motion charts, where he debunks several myths about world development.
I've written about data stories in the past, discussing why it is important and providing some ideas on how to use data visualization to tell stories. Basically, here are two really important things you need on a good data story:
Recently I worked on a data story with my colleague Lizzie Silvey, where we analyzed stats from the UK Office for National Statistics. We looked into Divorce and Marriage trends starting from 1862, and came up with an interactive visualization. Below is a screenshot with some of the insights on how changes in the law impacted marriage and divorce rates in the UK. Check the visualization to play with the data.
Whether you are working on a monthly report or a world-changing visualization, if you take the time to uncover and communicate the stories behind the data, you will be contributing to better decisions in your company and in society in general.
So here’s the deal: you’ve spent a ton of time with your data and you know it inside out. You’ve wrangled, sliced and diced it and are now the expert with this data for this problem. You’ve uncovered new, actionable insights that will lead to fantastic…
So here's the deal: you've spent a ton of time with your data and you know it inside out. You've wrangled, sliced and diced it and are now the expert with this data for this problem. You've uncovered new, actionable insights that will lead to fantastic opportunities or improve your bottom line. Great! Time to show your colleagues or your boss or your clients these findings.
You open your data tool of choice, quickly create some charts and make it all look pretty with a flashy color scheme or fancy logos. More often than not, we fly through this final stage and don't give the data visualization step the due care it needs. This is insane!
Think about it. Your charts and dashboards are most likely the only piece of information your boss or client will interact with. The only information! And yet, here we are, creating default charts and missing the opportunity to really convey our message.
Effective charts are a compelling way to show your data. The human brain is simply better at retaining and recalling information that has been presented visually.
In this article I will discuss several techniques that will help you create more effective charts to communicate the underlying data.There's no big secret here. However, by applying deliberate thought, a handful of best practices, and allocating sufficient time in projects for the data visualization step, you can make a big difference to the impact of your charts.
Before firing up your favorite data visualization software, it pays to spend some time thinking about your output and your goals. Start by answering a few simple questions:
- Who is the intended audience?
- What medium will you use to show your charts? (e.g. slides / dashboard / report etc.)
- What is the goal of this project?
For example, consider the audience who will view your chart. How long will they have to study it? How familiar are they with the data? Are they technically inclined? Do they want detailed charts, or quick summaries?
You want to optimize your message to resonate with your audience, so the more you know about them, the more likely you'll be able to achieve that.
Likewise, how you deliver your message will affect your decisions. Is it a chart in a slide deck? In an informal email? A formal report? An interactive dashboard?
Reports and dashboards are typically pored over for longer periods of time, so charts and findings can be more detailed, whereas presentations or client pitches are short and sweet, where the audience will only have a moment to understand and absorb the information.
Lastly, think about what your end goal is. What do you want your audience to do with the information you show them? For example, if you want your manager to make a cost-benefit decision for a new hire or expensive research tool, make sure your solution answers the question and facilitates making that decision.
Remember, the point of your visualizations is to communicate information, and you can ensure they do that more effectively by giving prominence to the key message within your chart.
You can do this by using attributes, for example color, to highlight specific elements of your charts and focus your audience's attention there. These are known as pre-attentive attributes, and they dramatically help speed up the absorption of information.
Consider this chart showing the open rates for four newsletters that you manage. There's an important story in there, but it's difficult to see with the default colors:
However, by carefully using colors, we can bring that story to the fore:
Consider the two charts above, showing email newsletter open rates. The second chart also has a heading that adds context to the story. The words complement the chart and reinforce the message.
Much like writing titles for your blog posts or newsletters, think about the title of your chart in the same way. It should tell the viewer what to expect in your chart and summarize the message.
Similarly, your data may have unexpected spikes or dips, so you might want to use annotations directly on the data points or as footnotes, to make sure the viewers have all the context they need.
Renowned data visualization pioneer Edward Tufte coined the term data-ink ratio to convey the ratio of ink needed to tell the core message in your display, divided by the total ink in the display. The idea is to maximize this ratio, in other words, reduce the amount of non-essential ink.
Let's see that in practice. Compare the following two charts showing Amazon's revenue between 2007 and 2016:
After decluttering, the annual revenue figures jump out at the viewer and the information is much quicker to absorb:
There are a lot of complex chart types out there: waterfall charts, radar charts, box and whisker plots, bubble graphs, steamgraphs, tree maps, pareto charts, etc. etc.
Sometimes these may be appropriate for specific cases (e.g. a Sankey chart to show web traffic flow) but it really comes back to the question of who your intended audience is and what medium you'll be showing your chart through.
Does this radar chart really communicate your message well? Would a simple bar chart, which is widely understood, be a better alternative?
Whenever I teach a dataviz class, I always say that a good chart should be like a good joke: it should be understood without you having to explain it.
Pie charts are popular and ubiquitous, but somewhat maligned by the data visualization community. Why is that?
Consider this default pie chart in Data Studio, showing website Sessions broken out by Medium:
This chart (and pie charts in general), have two main drawbacks: 1) it's hard as human beings to decipher the relative sizes of the slices (and the order and position of them affects this), and 2) the long tail is unreadable. Plus, the legend is ugly to look at.
A much better chart for data with many categories and a long-tail would be a standard bar chart. Nothing fancy here, but it's super quick and easy to read off the values, especially for the smaller categories (e.g. compare trying to understand email sessions in the pie chart vs. the bar chart).
So if you're going to use them, restrict pie charts to small numbers of categories (I'd advise three or less), and always ask yourself if a simple bar chart or table would suffice and be quicker to read.
Dual axes charts should be used with caution as they often cause confusion. It's tempting to use them when trying to chart data series with large size differences, as shown in the following image. Which series goes with which axis? Lines that overlap will also confer meaning that doesn't actually exist, because the series are on different scales.
Some strategies you can use to mitigate confusion include matching the series and axes with different colors, labeling the axes clearly and even using different chart types for the different series (line with a bar).
However, I'd still advocate only using them sparingly. It's often better to show the two series in separate charts next to each other.
For bar charts, you should always start the y-axis at 0 since the height of the bar represents the count in that category. We look at the height of the bars and compare them. If one bar is twice the height of the other, then we're going to conclude that the value of that category is twice the value of the other category, even if the axis shows otherwise.
Consider this simple example. Both bar charts have been plotted from the same data but they tell very different stories:
Vox Media created an excellent video about truncating the y-axis. With line charts we don't need to be so strict with truncated y-axes as the visual lines are used to compare trends, not actual values, as in the case of bars. Indeed you sometimes need to narrow the range with line charts to show the story.
Approximately 10% of the male population and 1% of the female population identify as color-blind, and the most common type is Red-Green color-blind. So it pays to keep this in mind when designing your charts.
Once you've created your charts, or your dashboard, pause and ask yourself these few questions:
There is no single right answer with data visualizations, as it will depend on many of the factors discussed above. People will come out with different charts from the same dataset, all of which could be equally effective. However, by following some best practices and thinking critically about your charts, you can improve them dramatically.
I'll leave you with some parting words from a master in this field:
"Above all else show the data" Edward Tufte