By: Yevdokiya Yermolayeva
Using data to drive decision-making has become an expectation, but getting the right data and using it in the right way to make the right decisions can be challenging.
All too often, our focus is on the statistical analysis and the visual presentation, and other factors that affect the quality and interpretation of our data might not get as much attention. During my time as a researcher, instructor of research methods classes, and product manager at a nonprofit I have relied on several techniques to maximize the value provided by data analysis.
These techniques can be easily implemented in any nonprofit, regardless of its mission, size, or how long it has been around.
Align your measurements to your goals
When collecting data, it can be tempting to focus only on usage metrics. After all, measuring what people are doing as they engage with your products and services seems clean and objective (think metrics such as how your users find you, the amount of time they spend on your website, and how many clicks they make). Unfortunately, just focusing on what users are doing is not the best measurement of whether or not you are achieving your mission. Users may very well be using what your nonprofit has to offer but may not be getting value out of it.
What you need to measure is the impact that your products or services have on your users, and the best way to do that is to align your metrics to your mission and your organizational goals, so that you are measuring the value your nonprofit brings. Make sure to set up comparisons to check for growth in value. For example, track your metrics over time to check that value grows as users go from weeks to months to years of using your services, or compare low-frequency and high-frequency users to check that the latter get more value.
Keep context in mind when looking at your data
A common pitfall when looking at data is to assume that your users are in a vacuum, interacting only with your nonprofit and immune to any external factors. Unfortunately, that is never the case. Whether you are a business-to-business or a business-to-consumer nonprofit, any number of factors, such as time of year, current events, competitor actions, and even the weather can cause changes in your data. Seeing a sudden dropoff in use? Could be due to an upcoming holiday. Seeing a big influx of new users? An organization that is providing similar services may have made some changes that have driven those users away. Be mindful of the full context in which your users operate and think back to that context whenever you see a pattern in your data. Could an external factor explain what you are seeing?
Don’t assume that numbers will tell you everything
If you are focused only on quantitative data, you might be missing out on a whole wealth of information about your users. It is true that working with qualitative data collected from interviews, observations, or open-ended surveys can be daunting because it can take longer to collect, typically does not come with a natural structure, and can feel very subjective to analyze. However, taking on this challenge can bring you great rewards, revealing insights that cannot be seen from numbers alone.
By interviewing your users or conducting observations, you can capture their attitude towards your products or services, identify points of frustration or confusion, and capture their feedback and ideas about changes that you could make to meet their needs more effectively. Getting this qualitative information can help you tell a story with your data, helping you answer the “Why?” question related to your numbers. For example, low usage of your services can be due to confusion about what you are offering or the fact that the services do not quite meet potential users’ needs. You will not know the true cause until you talk to some of your users.
Be open to being wrong
There is a wide range of statistical analysis techniques and data visualization options that can make your results quite malleable. While helpful for initial exploration of your data, it can be easy to get on a path of massaging your data until you get the results you expect by filtering out certain users, narrowing down your data only to a certain time period, or applying complex statistical transformations.
Be open to the fact that your data might surprise you, and that you may not find what you were hoping to find. If you find yourself in this situation, focus your energy on figuring out why you are seeing something different, not on extensive fiddling with your data. This is a good time to look at the data’s context and to gather some qualitative information.
Make data collection and analysis a part of your regular routine
Collecting and analyzing data on a regular basis brings several benefits to your organization. First, it is more efficient. If you analyze data very irregularly, it will take you longer to get going every time, to remember what metrics you use, to select the right types of analyses, and to interpret the results. If you analyze data routinely, all of these steps become a habit. Second, it allows you to track changes over time and to spot problem areas more quickly.
Working with data on a regular basis gives you a strong sense of what is normal and what is unusual by establishing a good baseline for comparison. Finally, insights from regular data collection typically carry more weight in an organization. Regularly using data establishes a common expectation that your decision-making process should be backed by research and not solely reliant on subjective opinions.
Selecting the right metrics, looking beyond the numbers at the context and some qualitative information, keeping an open mind, and setting up a regular data practice can yield significant benefits for your nonprofit’s ability to meet its mission and goals. What strategies have you found helpful in working with data? Share in the comments below!
About the author
Yevdokiya Yermolayeva is a product manager at Verity International with experience using data to build and refine products in the education technology and HR spaces. You can find her on LinkedIn and Twitter.