New and Improved Data Engine Architecture

At Panoply, we aim to be transparent with our customers and partners by frequently updating you on new features within the product that help you get your data faster. 

On top of these user-focused updates, our team is also continuously developing improvements to the behind-the-scenes parts of the platform. Although these improvements may not be visible to users, they affect critical components that form the “heart and brain” of our product. 

Here’s a sneak peek into one such improvement.

Our Data Engine team recently made changes to the Panoply Data Engine architecture that support improved features in the platform’s scale and stability. These changes will yield a number of positive long-term effects to the backend of our platform, some of which are interconnected:

1. Improved stability: One example is separating the loading workload from the collector workload, enabling finer control over memory usage and enhancing the overall system stability.
2. Enhanced adaptability: These architecture changes empower us to develop future product features more efficiently, enabling faster implementation and response to customer needs.
3. Increased scalability: This new architecture takes us several steps closer to achieving fully parallel processing, from data sources to the loader. This scalability will be vital for handling significant amounts of data, and will enable you to scale even further, effectively solve big data challenges, and ask bigger questions of your data.
4. Optimal resource allocation: These modifications allow us to better fine-tune resource allocation, resulting in more efficient operations. We’ll be able to allocate resources like memory and CPU more effectively, increasing or reducing them as needed. 

By refining our framework and architecture on an ongoing basis through design changes, we are working towards building a more stable, adaptable, scalable, and cost-efficient product for you. Optimizing our platform behind the scenes means you can see the value from your business data faster than ever before.

Not a Panoply customer yet? Book a demo to learn more.