Technical insights and creative ideas about data management, data infrastructure, and data analysis.

Subscribe to our emails and get Panoply updates on the fly.

Data Management

“I choose you!” - criteria for selecting a data warehouse platform

I loved this line from an article I recently stumbled upon:  “Choosing between the different types of data warehouse platforms can be simplified once you know which deployment option best meets....

[+]
Written by Yaniv Leven |March 16, 2017

Data Management

SQL or NoSQL, That Is The Question!

We all know that in the database technology world, it comes down to two main database types – SQL (relational) and NoSQL (non-relational). The differences between them are rooted in the way they....

[+]
Written by Alon Brody |March 09, 2017

AWS Redshift

Introducing Panoply Multi Zone Redshift Service

Today’s considered data management best practices, advocate building resilient architectures that span multiple data centers, regions or even continents.

[+]
Written by Roi Avinoam |March 02, 2017

Data Management

Continuously Encrypt Amazon Redshift Loads with S3, KMS, and Lambda

One of the main goals of this blog is to help developers and data architects, just like us, with their Amazon Redshift operations. Starting from a full comparison with Google BigQuery, explaining....

[+]
Written by Alon Brody |February 22, 2017

From raw data to analysis in under 10 minutes.

Sign up now for a demo or a free trail of the Panoply.io platform.

Learn more about platform features

Data Analysis and Visualization

How to Visualize Your Amazon Redshift Data Using Tableau

Data analyst? You must be familiar with both Tableau and AWS Redshift. And you must know, or at least you should believe me that integrating the Redshift columnar-based database speed with the....

[+]
Written by Alon Brody |February 08, 2017

Data Management

Step-by-Step: How to Load Your Google Analytics Data into Amazon Redshift

The basic free Google Analytics (GA) functionality is great for small and medium websites and mobile applications, but as your online business expands, you will start looking to get more of this....

[+]
Written by Alon Brody |January 24, 2017

Data Management

The Cloud Is Disrupting Data Warehousing and This 2017 Survey Proves It

Amazon re:Invent is a great place to gather feedback from industry professionals, and especially gauge cloud-industry trends. This past year, we used the opportunity to collect and analyze....

[+]
Written by Peter Gorne |January 18, 2017

Data Management

11 Great ETL Tools, and the Case for Saying “No” to ETL

Extract, Transform, and Load (ETL) is a data warehousing process that uses batch processing to help business users analyze and report on data relevant to their business focus. The ETL process....

[+]
Written by Alon Brody |January 09, 2017

Panoply.io Labs

Panoply PGproxy: Smart Routing of Your Data Warehouse Queries

A smart, proactive PostgreSQL connection pooler, we are proud to announce our PGproxy offers the advantages of query routing and rewriting, making it optimal for data engineers who need to query....

[+]
Written by Roi Avinoam |January 02, 2017

Data Industry and Trends

Apache Spark: Promises and Challenges

If you’re looking for a solution for processing huge chuncks of data, then there are lots of options these days. Depending on your use case and the type of operations you want to perform on data,....

[+]
Written by Alon Brody |December 20, 2016

Data Management

Data Augmentation: Bringing New Life to Your Data

If you recognize your data as an asset, than augmenting it simply means growing your business assets. With data augmentation, you can run manipulations on existing data, use multiple sources from....

[+]
Written by Yaniv Leven |December 13, 2016

Data Industry and Trends

Data-Oriented Takeaways from AWS re:Invent 2016: Query S3, Batch, Glue and More

Attendance at this year’s AWS re:Invent conference almost doubled, from 18,000 people last year to 32,000 people last week. The large, international cloud computing event attracted professionals....

[+]
Written by Yaniv Leven |December 05, 2016

Data Management

How to Move Your MySQL to Amazon Redshift

 

Data analytics is a requirement for virtually every business today. However building an analytical data warehouse requires vast computing power to maintain performance and quick retrieval of....

[+]
Written by Roi Avinoam |November 21, 2016

Data Industry and Trends

AWS re:Invent 2016 - The Top 5 Amazon Redshift Breakout Sessions

We’re sure you’re already beginning to build your re:Invent schedule for 2016. As the largest cloud conference in the world approaches, it’s time to make sure that you’re maximizing your time and....

[+]
Written by Peter Gorne |November 16, 2016

Data Management

ETL vs ELT: The Difference is in the How

For the last couple of decades ETL (extract, transform, load) has been the traditional approach for data warehousing and analytics. The ELT (extract, load, transform) approach changes the old....

[+]
Written by Roi Avinoam |November 06, 2016

Data Analysis and Visualization

Load and Transform: How to Prepare Your Data for Business Intelligence

Preparing data for Business Intelligence (BI) can be a very tedious and time consuming process. You want the data to turn into the best reports for analysis. But, the raw data needs lots of....

[+]
Written by Alon Brody |November 01, 2016

Data Management

Castles in the Cloud: Tips for Cloud Infrastructure

The cloud, unlike the Force, is not a mystical energy that surrounds us and binds us.

[+]
Written by Ken Saggy |August 15, 2016

AWS Redshift

A Full Comparison of Redshift and BigQuery

As we speak the future of cloud computing is being decided. Amazon and Google, as well as Microsoft and a few others, offer multiple cloud solutions for practically everything. Despite the....

[+]
Written by Roi Avinoam |July 04, 2016

Data Tools: How to Ace Product Evaluation

We often come across new products aiming to help us with our lives or improve our productivity. It’s in our best interests to find the best products that will minimize tedious work to none, but....

[+]
Written by Alon Brody |June 07, 2016

Data Industry and Trends

Never judge a book by its cover: data infrastructure and data visualization, mutually exclusive or not?

This is the third and final part in the 101 series covering big data concepts, terminology and technology

[+]
Written by Moran Gilad Halevi |May 23, 2016

Data Management

Data Dithering: How White Noise Can Improve Data Importing

Data and music have much in common given that both tell stories with points and notes, yet each on its own lacks the coherence and context that distinguishes music from noise. Paradoxically, in....

[+]
Written by Alon Weissfeld |May 09, 2016

Data Management

The Syntax of Semi-Structured Data

The byproduct of the ever increasing integration of the internet in our daily lives is massive growth of data transitioning between applications and servers every minute. In order to make these....

[+]
Written by Oshri Bienhaker |April 25, 2016

AWS Redshift

DIY Redshift Write Performance Benchmarks

Amazon Redshift is a very powerful data warehouse, optimized for analyzing massive amounts of data in a blink of a second, when configured right.

[+]
Written by Roi Avinoam |April 12, 2016

Panoply.io Labs Data Management

Introducing Panoply.io’s Upsert Mechanism

One of the most critical issue in data warehouse management is avoiding duplications. When you are periodically loading data into your data warehouse you will want to skip the rows that have been....

[+]
Written by Alon Brody |March 28, 2016

Data Analysis and Visualization

Blitz Analyzing with Microsoft SandDance

Microsoft Research just released SandDance, a free web-based data visualization tool.

[+]
Written by Alon Brody |March 25, 2016

Data Management

Data Warehouse Automation. A Question of When Not If

The premise that to build a perfect data warehouse you must have perfect business understanding is simply irrelevant. We must look to technology to empower us to keep pace with our rapidly....

[+]
Written by Yaniv Leven |March 14, 2016

Data Management

How Data Structures Impact the Data Warehouse

This is the second part in a 101 series covering Big Data concepts, terminology and technology.

[+]
Written by Moran Gilad Halevi |February 29, 2016

Data Management

The fallacy of One Data Technology to Rule Them All

As a species we’ve invented some pretty nifty things. It’s practically consensus that the wheel, the printing press and the internet top the list but we don’t need to look that far back to find....

[+]
Written by Roi Avinoam |February 16, 2016

Data Management

Go Raw: Why Raw Data Reigns Supreme

ETL processes (extract, transform and load) might be fundamentally flawed.

[+]
Written by Alon Brody |February 01, 2016

Data Analysis and Visualization

Build KPIs not One Time Reports

Businesses need to deliver insights fast if not immediately. If this were not true, we would not be writing this post, you wouldn’t be reading it and over all, big data would be less of a diamond....

[+]
Written by Alon Brody |January 19, 2016

Data Management

Data Warehouse What?

This is the first part in a 101 series covering Big Data concepts, terminology and technology. Starting with the data warehouse.

[+]
Written by Moran Gilad Halevi |January 04, 2016

Data Management

Why we Built Panoply.io and what Ray Kurzweil’s Law of Accelerated Returns has to do with it

“An analysis of the history of technology shows that technological change is exponential…So we won’t experience 100 years of progress in the 21st century — it will be more like 20,000 years of....

[+]
Written by Yaniv Leven |December 23, 2015