Cloud Data Warehousing Trends 2017

It is an exciting time for cloud data warehousing with significant disruptions expected throughout 2017. As Gartner, in March 2016, showed, warehousing enterprises continue to race against one another to deliver both cloud solutions and hybrid offerings.

While Amazon Redshift is the undisputed market leader, other main contenders, including Google, IBM, Microsoft, Oracle, SAP and Teradata, are engaged in various efforts to differentiate themselves. Complexity and pricing remain the primary area of contest with the large vendors capturing the market share from the smaller ones.

Amazon is the king of the cloud

The total IaaS sector grew by 49 per cent year-on-year to $10.3bn in Q4 2016 according to analyst house Canalys. This exponential growth was driven by businesses hesitant to sink resources into capital intensive, on-premise server estates. It is noteworthy that AWS accounted for over one third of all cloud infrastructure sales globally. This was more than its three next biggest rivals combined, while Microsoft’s year over year growth rate continues to lead the segment. In gross terms, AWS accounted for $3.48bn in sales, giving it a 33.8 per cent market share, while Microsoft, Google and IBM accounted for $3.17bn between them.

Cloud game favors the big players

We are seeing the market consolidating around several large players that account for now 2/3 of the market share and growing. Microsoft, Google and IBM are beginning to gain ground on AWS at the expense of smaller cloud providers. Over the past year, Microsoft, Google and IBM have grown by a combined 5% now equaling 23% of the total market share, while AWS has remained stagnant at 40%. However, in order to stay at the top of the pac, AWS has been making serious investments in infrastructure, keeping execution strong and expanding cloud service. These efforts take serious financial backing and manpower, which reinforcesthe big players’ advantage over the smaller companies . There is no question that the market will continue to consolidate as a result.

Cloud price war is on

The four large cloud providers are leveraging economies of scale in an effort to solidify  their competitive advantage. The dynamic of the economy of scale are defined by the ability to cut prices or increase profits as cloud expands capacity with a new data center, new infrastructure, or more mature automation. According to Synergy Group’s analysis of cloud revenue, given that AWS is such a strong, growing market leader, none of the other big clouds can afford not to compete on price.

Competition beyond price

Microsoft, is grappling with the strategic challenge being posed by AWS. Apart from significantly slashing prices, An element of Microsoft’s approach at achieving differentiation is encouraging customers to work with channel partners. This is driven by Microsoft’s recognition that many of their cloud services, including warehousing, are being underutilized by customers having difficulty managing the systems. As described by Tim Wallace, CEO at Content and Code, "It's a smart move on Microsoft's part as it will drive more customers to fully use Azure and get far more value from the service."

Managing complexity - the next challenge

Panoply conducted a survey in November 2016, which reinforced Gartner’s analysis. We found that almost 60% of respondents found their data warehouse difficult to manage. When drilling down, we discovered that 20% of those who claimed to be struggling with management issues were using a cloud-based solution, including 41% of all Redshift users. Considering that most respondents were technical professionals, such as IT specialists and developers, it seems that ease-of-use is still a weak spot for data warehouse providers, and that streamlining operations is still a challenge they need to meet.

Still, our survey indicated that AWS will continue to make gains in market leadership. This trend is especially important for data engineers, including those working with medium and small enterprises, as AWS can provide solutions to their most basic challenges. This is especially true when they are supported by end-to-end data management platforms.

Moreover, organizations are becoming increasingly aware of the value of data warehousing beyond simple storage. They’re identifying a need for better ways to extract information from their data and to analyze it. While Amazon Redshift architecture may hold the key to simplifying that task, by making the BI and other benefits of data warehousing accessible and effective; this can also place a strain on data engineers if the challenges of management complexity are not adequately dealt with.


All this is to say that AWS remains the main actor to watch as it solidifies its position as the dominant player. However, given the real and ongoing difficulties in managing data warehousing and coping with problems of complexity, there remains a significant need for end-to-end data management solutions working in concert with the main vendors. Panoply is one such solution, with our unique, self-optimizing data warehouse architecture utilizing machine learning and natural language processing (NLP) to model and streamline the data journey from source to analysis. Sign up now for our blog and see how we can help your business.
Get a free consultation with a data architect to see how to build a data warehouse in minutes.
Request Demo
Read more in:
Share this post:

Work smarter, better, and faster with monthly tips and how-tos.