So you’ve heard of a data warehouse, but now what? Do you need one? How can a data warehouse work for you and your business? We’ll look into the basics of a data warehouse, uses of a data warehouse and how it can be a great asset for you and your company.
But first, we have to look into data, and how it can be used to drive your business forward.
How Data Can Drive Your Business Forward
Organizations today have access to more data than ever before. This data is flowing in from many different areas – retail point-of-sale (PoS), CRM information, data from social networks, or even manufacturing data.
Combined with the ability of modern computers to process this massive amount of data, valuable lessons about past events, current performance and future opportunities can be gleaned. Companies are combining this data from the various disparate sources to form a fuller picture upon which decisions can be made.
The Harvard Business Review published a paper entitled “The Evolution of Decision Making: How Leading Organizations Are Adopting a Data-Driven Culture." The paper notes that “in a rapidly changing global business environment, the pressure on organizations to make accurate and timely decisions has never been greater. The ability to identify challenges, spot opportunities, and adapt with agility is not just a competitive advantage but also a requirement for survival.” Data, the paper goes on to note, offers key business intelligence (BI) information, that can be leveraged to make faster, better decisions.
This access to data to drive better decision making affects the entire organization. It could be the CEO deciding on which geographical market to pursue next, the product team deciding on new features, or marketing looking at the results of the latest campaign. In fact, the more accessible the data is, the better the synergies and opportunities that become available.
Getting all of this information from different sources, and making it accessible to users, is challenging. Not lease because of:
- Data is sorted in different ways by different systems
- Data might be updated at different times for each data source
- Data might not make sense to the end user, in its current form
All of these challenges—and many more—can be solved through the use of a data warehouse. So what exactly is a data warehouse?
Data Warehouse
A data warehouse is any system that collates data from a wide range of sources within an organization. Data warehouses are used as centralized data repositories for analytical and reporting purposes.
Lately, data warehouses have increasingly moved towards cloud-based warehouses and away from traditional on-site warehouses. There are a number of advantages to using a cloud-based data warehouse, including:
- Scalability: unlike on-site warehouses, scalability can be achieved with the click of a button
- Cost: no hardware or upfront licensing costs
- Time to market: it’s quick and easy to get a data warehouse up and running in the cloud
- Performance: cloud data warehouses are optimized for analytics
- Maintenance: when on-site data warehouses run into problems, they requires significant resources (time, manpower, money) to keep them effective
- Functionality: adding new data sources, for example, to an on-premise data warehouse can be quite an undertaking, whereas cloud data warehouses are often set up to easily accept new sources
In talking about what a data warehouse is, it's helpful to understand what a data warehouse isn't.
1. A data warehouse is not a database.
The “data warehouse vs database” question is often asked. Databases are commonly used for transactional processing (called “OLTP," or “online transaction processing”). Database software needs to provide easy access to information and fast querying so that transactions can be carried out efficiently. They are often referred to as operational systems, meaning they are used to process day-to-day transactions in an organization.
A data warehouse on the other hand is used for online analytical processing (OLAP), which uses complex queries to analyze, rather than process, transactions. These data warehouse concepts are important in understanding the value of a data warehouse. In short, a data warehouse is built to store large quantities of data and enable fast, complex queries across all this data, while a database was is primarily used to store current transactions and enable fast access to specific transactions for ongoing business processes.
2. A data warehouse is not a data mart.
A data mart is a subset of a data warehouse oriented to a specific business line. Data marts contain repositories of summarized data collected for analysis on a specific section or unit within an organization, for example, the sales department.
A data warehouse on the other hand is a large centralized repository of data that contains information from many sources within an organization. The collated data is used to guide business decisions through analysis, reporting, and data mining tools. Whereas in the past, organizations would need to decide whether to build specialized data marts and how these would fit into the data warehouse, today with cloud-based data warehouse services being so cost-effective, scalable, and extremely accessible, organizations of all sizes can leverage cloud infrastructure and build a centralized data warehouse. For more detailed information, and a data warehouse tutorial, check this article.
3. A data warehouse is not a data lake.
A data lake is a highly scalable storage system that holds structured and unstructured data in its original form and format. A data lake does not require planning or prior knowledge of the data analysis needed—it assumes that analysis will happen later, on-demand. A data warehouse, however, contains structured, processed, mature data, and is more likely to be used by a business professional than a data scientist.
How a data warehouse can help you
So you now have all of your data in a data warehouse. Now what? And what are the advantages of a data warehouse?
This is where the really interesting part comes in. Using BI tools, for example, you can now query your data and take out key learnings – many of which would not have been obvious without this data warehouse/BI combination. Using a BI tool on top of your data warehouse lets you visualize the data, and see patterns, trends and correlations.
In addition to the benefits of using a BI tool to drive data-driven decisions, you will have all of your data stored, across the organization, in one place and in a structured manner. The need of a data warehouse is critical for anyone that wants a data-oriented business approach.
Data warehouse example
One of the best ways to see a data warehouse in action, and appreciate the benefits of a good data warehouse, is to look at a data warehouse example and the uses of a data warehouse.
At Foursquare, the company leverages a data warehouse to ensure that critical, up-to-date and aggregated information is available to anyone that needs it throughout the organization. Jon Hoffman, a Foursquare software engineer, notes that “anyone in the company can set up any queries they like — from how users are reacting to a feature, to growth by demographic or geography, to the impact sales efforts had in different areas". Foursquare leverages this to drive data-oriented decisions across the organization.
Data warehouse tools
There are many data warehouse tools available that can make the data warehousing process a lot smoother and easier. Some of these deal with moving data to the data warehouse (the most commonly used is the ETL process), while others deal with various other parts of the process including testing the data in the data warehouse to ensure it is correct.
While these tools help you to achieve different things, using a solution like Panoply can take care of both with one simple platform. Built for analytics professionals, by analytics professionals, Panoply puts analysis-ready data at your fingertips so you can focus on finding insights, not maintaining infrastructure.
