Every week, analysts around the globe are working hard to create data warehouses.
And far too often, most of their effort is wasted.
Why? Because it's easy to create the right solution for the wrong problem.
If you build a data warehouse that's too complex, business users will avoid working with it in favor of systems that are less powerful but more familiar. On the other hand, if you don't pull in enough data—or the right data—business users won't find value in it and it'll sit in the corner, gathering dust.
Luckily, there's a simple way to avoid this fate. Before you start building your data warehouse, make sure you understand the business needs you're trying to address.
In this article, I'll walk you through actionable steps you can take to lay the foundation of your data warehouse's success by fleshing out your business needs.
Before we dive into the nitty-gritty of analyzing your company's business needs, there are 4 principles you need to know that are critical to a successful analysis.
If you've never analyzed a company's business needs before, you may be tempted to skip it—trying to wrap your head around the whole business can feel pretty daunting.
You'd definitely feel that way if you've read articles about analyzing business needs of a traditional data warehouse. Changing the structure of a traditional data warehouse is time-consuming and painful. So nailing your needs analysis is absolutely essential.
But with a modern data warehouse, it's much easier to modify your data warehouse's structure if you're a little off the mark. As a result, you don't need to spend endless hours writing a detailed analysis. Spend just enough time with your key users and managers to understand the business problems they need to solve.
The other reason not to go overboard in analyzing your business needs is that it doesn't make sense to try to analyze all of your business needs at once:
So if you want to help your company thrive, divide the creation of your data warehouse into iterations of analysis and development.
For example, when Josh Temple, currently an analytics engineer at Spotify and cofounder of Spectacles, was hired to build the analytics operations for Milk Bar, he began by doing the following:
I realized it would be great to build a stack end-to-end in a thin slice of the organization, basically like a proof of concept... That gives something people can look at and say, "Oh this is cool, we didn't used to be able to do that." And then expand it out into other data sources or applications.
- Josh Temple
If you just ask your users for the analytics they need, they'll often give you metrics that would be interesting to know but that won't help them solve their most pressing business problems.
This is entirely understandable. Many staff don't have enough experience to know how to translate their business problems into something that can be measured. And for some business problems, coming up with actionable metrics isn't easy.
So, you'll need to dig a little deeper. The easiest way to do that is to ask users and managers, if the data warehouse could give you an answer, what decision would you make? For example:
Here's some advice you're probably going to hate: your job is a political job.
If you enjoy working with data, odds are dealing with organizational politics can feel gross and wasteful—and sometimes it is. But if you're going to build a successful data warehouse, you're going to have to get comfortable with the political aspects of your work.
Michael Kaminsky, founder of Kaminsky Data Strategies and the first analyst at Harry's, is extremely skilled at the most technical aspects of building a data warehouse. And yet he says:
And that's why as you do your analysis, it's critical to also work on developing relationships, setting expectations, and building alliances.The toughest problems are organizational and political problems. Like getting the organization to agree on what the right metrics are. If you're an analyst who doesn't have practice getting buy-in at the executive level, that can be really challenging.
- Michael Kaminsky
Now that you understand the principles that should guide your work, let's explore the four sets of questions your business needs analysis must answer.
Before you get too deep into the analysis, it's useful to have a quick-and-dirty understanding of what information your users currently have access to and are regularly using.
You don't need a ton of details. But by getting a cursory overview of their information landscape, you can find out:
Based on what you know about your company, you may think you have a pretty good idea of what analytics your users need.
Don't assume you're right.
You don't want to spend weeks on an analysis—or setting up a warehouse—only to discover that it's not useful or that it's a much lower priority than several needs you could quickly knock out.
Therefore, it's crucial that you talk to users to find out what information they’re looking for—and check your assumptions about what they need.
If you just ask users what information they need, odds are you'll miss some critical issues. That's because there are analyses users know could be extremely useful, but obtaining them seems like a pipe dream.
Their assumption might be inaccurate. For example:
So ask them what's on their wish list.
Even if some of their wish list isn't doable, understanding what's on that list may give you clues about business needs you can address that would be a big win for the organization.
Speaking of not doable, when talking with users about their wish lists, make sure to set expectations from the jump. Be clear that you're not making any commitments, you're just trying to understand the full scope of what they’d like to be able to do.
Correctly answering this question can make or break your data warehouse project.
For example, if Marketing has some staff who are super comfortable slicing and dicing numbers, their needs may be pretty self service and addressable with geekier tools. In contrast, an Operations department that isn’t comfortable with data may require another approach.
Miss this and you may end up setting up Looker for an Operations department whose manager may parrot the importance of being data-driven...but whose staff will be paralyzed by—and extremely resentful about—too much access to Looker's powerful but complex capabilities.
If you'd taken the time to understand their current capacity, you could have built a solution that would get you a couple of quick wins now and that would allow you to set Operations on a path to becoming data ninjas in a few years.
Analysts often blame users for being "too stupid" or "too demanding" where the real problem was that the analyst didn't understand how comfortable a department was with thinking and working with data. Don't make that mistake. Take the time to briefly assess each department's capacity to be data-driven.
There's no one right way to gather the info you need to analyze your company's business needs. If your company lives in Zoomlandia but is Slackphobic, you're going to use a different mix of techniques than you would at a company spanning the globe where most communication is asynchronous and everything runs through Slack.
As you're figuring out the best way to gather intel, here are a few techniques you might consider:
However you gather business needs, be sure to demonstrate empathy and respect. You aren't just gathering info, you're also beginning to build new partnerships and alliances. So how you do it is as important as what you do.
In fact, Temple argues that the biggest problem analysts face isn't a technical one, it's building trust.
It's so much about trust, especially if you are coming into a company that doesn't have a mental model for what analytics should look like or why it would be valuable to them. It's really important to be winning people over.
- Josh Temple
In this article, you've learned what you need to do to analyze your company's business needs for your data warehouse.
With only a moderate amount of work, you can ensure your data warehouse gives your users the tools they need to solve the business problems that are key to helping your company thrive.