Latest Stories In: Panoply Blog - Data Management, Data Infrastructure, And Data Analysis. | Data Engineering

The demand for skilled Data Engineers (or Big Data Engineers) is projected to rapidly grow.  No wonder that’s the case: no matter what your company does, to succeed in ...
Data Engineering

How To Become A Data Engineer: A Guide

On Feb 28th, Amazon Web Services (AWS), went down for approximately 4 hours. The outage occurred in Amazon’s US-East-1 facility in Virginia, their biggest region, and ...
Data Engineering

Lessons Learned From The Recent Amazon S3 Outage

As a data engineer, I enjoy my work, but when it comes  to integrating and managing data  there are quite a few tasks that are downright tedious. From data remodeling to ...
Data Engineering

What Does A Data Engineer Do That Keeps Him Awake At Night

We all know that in the database and data warehouse technology world, it comes down to two main database types – SQL (relational) and NoSQL (non-relational). The ...
Data Management

SQL Vs NoSQL: The Differences Explained

Extract, Transform, and Load (ETL) is a data warehousing process that uses batch processing to help business users analyze and report on data relevant to their business ...
ETL

11 Great ETL Tools, And The Case For Saying “No” To ETL

A smart, proactive PostgreSQL connection pooler, we are proud to announce our PGproxy offers the advantages of query routing and rewriting, making it optimal for data ...
Data Engineering

PGproxy: Smart Database Load Balancing And Query Routing

If you’re looking for a solution for processing huge chuncks of data, then there are lots of options these days. Depending on your use case and the type of operations ...
Data Engineering

Apache Spark Architecture, Use Cases And Issues

Attendance at this year’s AWS re:Invent conference almost doubled, from 18,000 people last year to 32,000 people last week. The large, international cloud computing ...
Data Engineering

Data-Oriented Takeaways From AWS Re:Invent 2016: Query S3, Batch, Glue And More

The cloud, unlike the Force, is not a mystical energy that surrounds us and binds us.
Data Engineering

Castles In The Cloud: Tips For Cloud Infrastructure

The byproduct of the ever increasing integration of the internet in our daily lives is massive growth of data transitioning between applications and servers every ...
Data Engineering

Handling And Constructing JSON Semi-Structured Data Models

This is the second part in a 101 series covering Big Data concepts, terminology and technology.
Data Warehousing

How Data Structures Impact The Data Warehouse

As a species we’ve invented some pretty nifty things. It’s practically consensus that the wheel, the printing press and the internet top the list but we don’t need to ...
Data Engineering

Hadoop Alternatives: What Other Big Data Solutions Are Out There

ETL processes (extract, transform and load) might be fundamentally flawed.
ETL

How To Improve The ETL Process Flow With Row Data Processing

Work smarter, better, and faster with weekly tips and how-tos.