Recently, Panoply held a webinar with Nathan Patrick Taylor, the CIO of the Symphony Post Acute Network. The focus of the webinar was on what Nathan calls the scalability paradox—the dilemma of focusing too early or too much on scalability, so much so that you derail a product's growth and it never reaches the size where it needs to scale.
Nathan offered a ton of wisdom for others trying to build a data stack and for analysts trying to build a career in data, so watching the full webinar is definitely recommended. But if you’re looking for the highlights, look no further:
Early in his career, when Nathan was working as an app developer, his team reached a point where it became clear that their code wasn't going to scale. But Nathan also realized that focusing prematurely on scalability could undermine the need for scalability—they wouldn’t need scalability if their product "never sees the light of day or if people are never going to use it."
That’s a common problem for startups, as they often find themselves trying to balance the time it takes to write scalable code and the need to build quickly and get the product in users’ hands.
Nathan said the same dilemma is present in the work that analysts do. At the beginning, scalability is far less important than getting something out there and getting feedback because there's no way to know in advance which analyses and reports are going to gain enough traction to need scalability. For example,
[I'll have] somebody email me, “When are you going to build this report, when is it going to be done?” And then I publish it and no one uses it. It just sits there, collecting digital dust. Other times, I build something on a whim because I have my feelers out.
For example, we did this geospatial analysis of some of our patients. No one asked for it, we just thought it was a cool thing to try. People loved it, and it ended up turning into a full-blown project.
When someone gives a webinar on how they came up with the first-rate solution they're using today, the story often goes like this: First they built a solution that had serious problems, then they found an amazing tool or framework that overcame the problems, and so everyone lived happily ever after.
But the road to data success is often far more difficult than the cleaned up version we're given. In contrast, Nathan gave us the unvarnished story—and in the process reinforced the challenges of the scalability paradox.
Symphony started out by trying to build a traditional monolithic Microsoft data warehouse that captured all the complex data they had. After two attempts, they realized that building one data warehouse created a solution that might scale but was way too complex. So they switched to just building little data marts that were far easier to develop and "that ended up working out really well."
As the use of their data marts increased, eventually they ran into scalability issues. So, they switched to Snowflake. And as they did, they ran into a problem that Nathan thinks is one of the hidden issues of scalability for data teams: "if we build it and they come, is my estimate on how much data I'm going to need to move correct?" In their case, their estimate was way off. Instead of using 80-90 GB/month, they only ended up using 10 GB/month, which meant they were paying a lot for capacity they didn't need.
Eventually they realized that with Snowflake, "we were trying to kill a housefly with a nuclear bomb instead of a fly swatter," so they ended up going back to Microsoft's SQL Server. And then they found Panoply, which perfectly fit their need for speed of deployment and on-demand scalability.
With data infrastructure that fit their current needs, they rebuilt their data solution, using a strategy of "pick those winners first, those high-value targets that everyone's going to see." For Symphony, that meant starting with the one metric that mattered (OMTM): census, aka "heads in the beds." Next they tackled accounts receivable, invoicing, and aging, then worked their way down the list until they hit data that users need just once a month.
The value of this approach really paid off when the pandemic hit. For the first few months of the pandemic, their electronic medical records (EMR) vendor didn't have a way to store infection control information. Symphony couldn't wait that long, so they had users store the data in Excel spreadsheets. That would've been a huge pain to manage in a traditional data warehouse; with Panoply, they could easily import the data and analyze the results in Power BI.
Ultimately, Symphony ended up with what Nathan calls an "agile" data stack. By agile, he means a data stack that allows his team to quickly build prototypes and modify production systems. Their data stack included:
Now that all the basics are working well, Symphony is beginning to explore predictive analytics using platforms like DataRobot, an automated machine learning platform. They pull their data from Panoply, then use Alteryx to move the predictions from DataRobot to their Power BI service.
So far, they've been able to build models for readmissions, falls and pressure ulcers, and operations-related issues such as turnover in a remarkably short amount of time. In less than two weeks, they built a model and a means for outputting the results, a Power BI report that lets users know whether a patient is likely to end up becoming a readmission.
They are also beginning to gear up for forecasting, which "is its own beast." Here's an example of the kind of issue they're addressing:
[We] want to know if this person's likely to have a fall in the next couple of days, and why is that? Is it because they have the wrong assistive device? Is it because they're on a medication that makes them dizzy? Are they too weak? Maybe they're trying to do too much before they've actually regained strength.
Nathan stressed that it’s important to realize that their goal isn't perfect accuracy, it's to get a better forecast of what's reasonably likely to occur and to enable frontline staff to act.
Nathan argued that fear is one of the biggest obstacles to building a successful career:
One of the things that I recognized as I started to move from position to position was to drop your fear of saying the wrong thing, drop your fear of putting yourself out there.
In particular, he suggested:
It’s rare for a webinar to provide an unvarnished view of reality, but that’s exactly what you’ll get with The Scalability Paradox. We’ve hit the highlights here, but there are plenty more nuggets of wisdom for you to find, whether it’s about when to use incremental loading vs. kill-and-fill or how to make a posture of humility work for you in the workplace.
You can check out the full webinar here, but we also recommend following Nathan on Twitter and subscribing to his YouTube channel so you don’t miss his takes on tooling, career tips, or tricks for taking your Power BI dashboards to the next level.