Panoply Blog: Data Management, Warehousing & Data Analysis

How to Build a Salesforce Data Warehouse: Budget-Friendly Solution

Written by Anders Schneiderman | May 5, 2020 12:00:00 PM

Salesforce is a powerful tool for managing your sales processes. But if you're serious about using your business data to drive your organization, at some point you're going to run up against Salesforce's limitations.

Maybe you've heard that the best way to level up is to deploy a data warehouse. You've also heard that data warehouses can cost a lot of time and money—and you don't have either to spare.

Just a few years ago, that would be the end of the story. But the world of data warehouses has changed dramatically.

In this article, we'll give you a brief overview of how a data warehouse can help you move beyond Salesforce's limits, why traditional approaches to data warehouses just don't cut it anymore, and how to take advantage of a modern data warehouse on a budget and timeline you can afford.

Why Salesforce Isn't Enough for Your Analytic Needs

Salesforce is the industry standard for managing your core sales relationship with prospects and customers. But if you've used it for any length of time, you've almost certainly bumped up against its limits as a tool for analyzing your business data.

Salesforce's Reporting is Inflexible

Basic Salesforce reports are quite useful when you're first getting started. As you develop more sophisticated needs, you'll find that it's increasingly difficult if not impossible to get the information you need to grow your business.

For example, because of the way Salesforce data is structured, it's hard to run a number of critical queries that you'll most certainly need. Is it possible to analyze which accounts have purchased which products? Yes it is. But do you or your staff have the time and energy to bang your head against that wall?

Salesforce Offers a Limited View of Your Business

Not only are you limited in what you can do to get insights from your business data that's in Salesforce, but your Salesforce data only offers a partial look into your prospects and customers.

Salesforce won't have all the relevant information from your customer support system's data. It won't have info from your email marketing and web analytics systems. And it won't have vital business data from all of the other systems your business uses.

While it's possible to create data integrations between Salesforce and other systems, getting it to work exactly the way you need it to and maintaining it can be time-consuming and difficult. And Salesforce isn't designed to be your Single Source of Truth (SSOT).

Extensively Customizing Salesforce Can Be Very Expensive and Time-Consuming

Salesforce experts will tell you that because Salesforce is so customizable, there are ways to get around many of these problems.

That's true—if you can afford to spend a lot of time and money. But if you're a startup looking to create a data stack or a lean business that can't add headcount just to make Salesforce's data model work for you instead of against you, you'll need a little help.

People who are experts in Salesforce make a very good living. There's a reason for that. Getting Salesforce set up so you can take full advantage of what it can do requires a high degree of skill. And once you go beyond what Salesforce does well, getting around its quirks and limitations can rapidly chew up a lot of hours.

You could train some of your staff to be Salesforce experts. But is it really worth spending so much of their time wrestling with Salesforce instead of delving into your data to find the business insights that can make the difference between failure and success?

Salesforce's Pricing Limits Who Has Access to Your Analyses

Because Salesforce can get pretty pricey, odds are not all of your staff have access to it. Staff who don't have access may only need reporting and analytics on a slice of your Salesforce data or on a much higher level view of the data. So you'll be faced with the unenviable choice of forking out a lot more money or limiting who can derive insights from your business data.

Traditional Salesforce Data Warehouses: Powerful but Painful

A Data warehouse is a specialized tool for storing and analyzing large amounts of data from databases and other sources of information. Combine your Salesforce data with all the other information your business gathers, and you have the potential to understand the opportunities and risks facing your business that will help you thrive.

But what many businesses have learned the hard way is that far too often, the road to building a robust traditional data warehouse is long and treacherous.

Traditional Data Warehouses Are Complex And Often Painful to Build

Ask people in your industry who have created a data warehouse about what the process was like, and odds are you'll get an earful. Sometimes businesses get lucky. But more often than not, traditional data warehouses are giant projects with a staggering number of complex details that you need to get right.

To get a feel for what it's like to build a traditional data warehouse, let's take a look at an article on Salesforce's website. In Advantages of Implementing an Enterprise Data Warehouse, Salesforce talks about how awesome data warehouses are and explains that for a data warehouse solution "that fits perfectly with your existing systems and processes, you’ll be better off building your own." Then they link to an article on a high level overview of what's involved.

Here's what you're signing up for:

Step 1: "Define Your Logical Data Model (LDM)," which means "you have taken the time to document your business’s description of the operations," including "a catalog of business entities and relationships that are meaningful and critical to the business." And don't forget to "take any industry relevant standards (HL7, Party Model, etc) into consideration"!

Step 2: "Define Your Enterprise Data Strategy (including Master Data Management)." This lovely step includes answering questions like, "is there a Customer Master or Master Data Management system and if so what LDM entities are involved?"

In Step 3, we get to the fun stuff: "Document the Data Lifecycle of Each Entity in the LDM." Because "each entity within the LDM will have its own lifecycle," the article cheerfully informs you, "it is critical to capture, document, and analyze each specific entity." And you better get it right -- later you're going to use all this info you painstakingly collected and analyzed to decide "how to build a tiering strategy, and even how to build a governance model."

Now we're onto Step 4: " Translate Entities and Cardinality into Objects and Relationships" -- "an art and not a science and I definitely recommend working closely with someone very knowledgeable on the Salesforce platform."

If you haven't broken out in hives by this point, congratulations! You've only got another eight steps to go!

And that's assuming that everything goes as planned. Which of course it never does.

Traditional Data Warehouses Are Costly to Maintain

Once you've built your data warehouse, the fun doesn't stop. Data warehouses take a lot of care and feeding. If the data warehouse is on-prem, your IT staff will have to back up the servers, keep the system server software up to date, and a host of other tasks. And when your data warehouse inevitably outgrows your server? You've got the pain and suffering of server migrations to look forward to.

You can eliminate some maintenance work by managing your Salesforce ETL and data warehouse in the cloud. But your IT staff will still have lots to do. For example, they'll have to regularly tune your data warehouse's performance to ensure queries stay speedy as the volume of your business data grows and your staff use the data warehouse more intensively.

Your IT staff will also have to keep a close eye on the ever-changing cost of crunching lots of data in the cloud. The formulas for calculating cloud costs can be quite complex. And as many businesses have learned the hard way, the cost of a cloud-based data warehouse can easily jump dramatically.

But probably the biggest challenge you'll face is constantly having to adapt the data warehouse as your users' and your business' needs change.

For example, suppose your business is now running enough events that you decide to start using an online events management system. Or you realize that your social media marketing tool doesn't really fit your needs and you need to switch to a new one. Guess what? You've just created a whole new project to analyze the data, design a solution, test it out, etc.

On top of that, over time your users will get more sophisticated in how they want to slice and dice the data. More often than not, your IT staff will have to change the structure of your data warehouse so it can handle users' new needs. The more experienced your users get, the more your IT staff will have to scramble to keep up.

Most Traditional Data Warehouse Projects Fail

Given these daunting obstacles, it's not surprising that the business world is littered with traditional data warehouse projects which had high hopes when they started but eventually crashed and burned. As InfoWorld noted last year:

Big data projects are, well, big in size and scope, often very ambitious, and all too often, complete failures. In 2016, Gartner estimated that 60 percent of big data projects failed. A year later, Gartner analyst Nick Heudecker‏ said his company was "too conservative" with its 60 percent estimate and put the failure rate at closer to 85 percent. [In 2019], he says nothing has changed.

Alternatives To a Traditional Salesforce-Driven Data Warehouse

Far too often, traditional data warehouses are a tar pit that can swallow up a staggering amount of your business' time and money. But if Salesforce's analytical tools aren't enough, what's your alternative?

There are three ways to solve your problem:

1) You can push all of your data into Salesforce and maintain loads of integrations. Even Salesforce doesn't think this is a good idea. That's because Salesforce isn't designed to accommodate all of the data sources you'll need to integrate if you're serious about using your data to drive your business.

2) You can use an off-the-shelf data warehouse that provides black box, cookie-cutter solutions. For example, there are a number of companies which can help you whip up pretty dashboards that look great but don't give you any way to access the data used to create them. If you're comfortable not having any way to check to ensure the dashboard is accurate, that could work for you. These data warehouses also assume that your analytical needs are pretty simple. That's fine if all you care about is a basic overview of your data. But no cookie-cutter solution is going to give you the edge you need over your competitors.

3) You could try to do all your analyses by integrating all of your sources of data through a spreadsheet. The technical term for the solution is spreadsheet hell. If you're reading this article, you already know you don't want to go there.

It used to be that these were your only choices: roll the dice with a traditional data warehouse, or live with a solution that doesn't take care of your business needs. Today, you've got another option: modern data warehouses like Panoply.

Panoply: Data Warehouse Power without the Pain

Panoply is one of the first modern data warehouses to harness the latest tech developments in order to give you the analytic power of a traditional data warehouse on a budget and timeline you can afford. It takes advantage of the fact that cloud-based computing power and storage are cheaper than ever. And it automates most of the work of a traditional data warehouse engineering team. As a result, with Panoply all of your business data is in one place where it's always up-to-date and fully controlled by you.

Here's what that means for you:

If You a Marketing Manager or a CTO:

  • Everyone Can Compare Data across Multiple Sources. Tired of your staff having to waste time chasing down answers in all of your platforms and apps? Panoply comes with built-in data integrations that let you connect everything you already use, including all major CRM's, databases, file systems, ad networks, web analytics tools, and more. So consolidating all the information you have about your customers is a snap. You can even add in your planning spreadsheets -- handy if you want a dashboard that compares your monthly targets with how you actually did.
  • Everyone Gets Access to the Data They Need. Panoply democratizes access across the organization. Anyone can create sales reports within Panoply in minutes, with no custom scripts necessary. And Panoply seamlessly connects to visualization and analytics tools with a few clicks. The end result: anyone can get the insights they need to play their part in helping your business thrive.
  • Getting Set up Is Simple and Quick. Panoply is designed to get you up and running quickly and easily. Our built-in data integrations make it super easy to bring in all the data you count on, and connecting Panoply to your favorite tools for analyzing data takes minutes.
  • You Don't Have To Worry about Maintenance. With Panoply, you don't need to hire a small army of engineers to handle the care and feeding of servers, optimize your queries, and a host of other tasks that have nothing to do with building your business. And Panoply seamlessly scales as your business and your data needs grow. Because Panoply automates all of this work, you and your team can focus on your greatest opportunities for success.
  • Pricing is Simple, with No Hidden Surprises. If you are building a data warehouse on Google Cloud or Microsoft Azure, it can be maddeningly difficult to figure out how much it's going to cost you today let alone over time as your needs evolve. With Panoply you can try out the system for free. And if you decide Panoply fits your businesses' needs, you have a range of plans to choose from that all come with simple, upfront pricing.

If You Are an Analyst:

  • Access to Your Data Is Easy and Fast. Behind the scenes, Panoply uses sophisticated machine learning algorithms to optimize how your data is stored so you can quickly pull up the information you need. And the more analyses you do, the better Panoply understands your needs. That's why one study found that after just 2 weeks of using Panoply, dashboard runtimes decreased by 85%.
  • Your Data Is Always Up-To-Date. No more exporting. No more endless series of duct tape solutions. When you first connect one of your data sources using our built-in data integrations, you choose how often you want your data to be refreshed, and you're good to go -- Panoply takes care of the rest. And if you realize that you need to refresh your web analytics data more frequently, you don't need to wait for a data engineer to take care of it; change one drop-down and you're set.
  • Adding New Sources of Data Is Easy. No more waiting on IT or consultants to prioritize a new source of data that you need for critical analysis. Click a few buttons, add your credentials, choose a few options, and you're ready to go.
  • You Can Use Your Favorite Reporting/BI Tools. Some data warehouse systems are great with one reporting/analysis tool, and you can probably get another one up and running if you're willing to spend enough time. In contrast, Panoply is designed from the ground up to connect to most analysis and data visualization tools with just a few clicks.
  • Creating More Complicated Analyses Is Easy. Need to restructure your data or add in some complex calculations? With most data warehouses you're looking at a long wait - not to mention the time it takes for MySQL replication. And if you're tweaking the data using a reporting tool, odds are whatever flexibility they offer comes with a steep price -- wrapping your head around yet another proprietary language. Panoply doesn't think you have that kind of time to waste. That's why Panoply let you use standard SQL to slice, dice, and tweak your data to your heart's content. And Panoply's built-in data integrations automate away one of the hardest parts of using SQL -- having to join seemingly endless numbers of tables -- by giving you a simplified, cleaned up model of your data.
  • You Can Spend More of Your Time On Analysis. Everyone knows that most analysts spend the vast majority of their time on data wrangling. Because Panoply automates so much of that work, you get to focus your work on unlocking new insights that will help your company thrive.

These and other advantages are why Panoply is ranked #1 in for Ease of Implementation and Ease of Use on sites such as G2 Crowd.

How To Build Your Salesforce Data Warehouse In 10 Minutes

What is it take to get a Salesforce Data warehouse up and running using Panoply? With Panoply it’s a three-step process:

  1. Sign up for Panoply
  2. Sync your Salesforce data into your Panoply data warehouse
  3. Explore it with SQL or connect your favorite BI tool like Power BI, Looker, or Chartio
  4. Want to try some SQL queries? We have a growing SQL library
 

 

As you can see, the process is really straightforward: choose Salesforce as a data source, log into Salesforce so Panoply can ingest the data, then connect Panoply to your favorite analytics/visualization tool, and you're ready to gain a new understanding of your business' opportunities to succeed.

Like what you see? Talk with a Panoply solutions specialist to learn more and to get a personalized demo. Or start your free trial today!