Amazon re:Invent is a great place to gather feedback from industry professionals, and especially gauge cloud-industry trends. This past year, we used the opportunity to collect and analyze statistics disclosed by industry professionals surveyed at our booth. The results reveal the industry’s most pressing challenges and needs, the future of data warehousing, and in particular illuminates how Redshift is gaining serious traction among the foremost data warehouse leaders.
Responses from more than 800 re:Invent attendees support what industry experts such as Gartner have been saying – that cloud-based data warehousing solutions, such as Amazon Redshift, are transforming the market, leading to a noticeable shift in industry leadership and the way vendors will have to approach clear needs and challenges that still exist in data warehousing.
Check out the full Panoply re:Invent 2016 Survey Report to learn more about our methodology, and for more detailed findings.
Here are the four most significant takeaways:
1. Amazon Redshift is tops
As the survey was conducted at an Amazon event, it wasn’t a revelation to learn that the majority of the respondents, 60%, use Amazon Redshift. Interestingly, however, the next largest group, at 21% of all respondents, run an on-premise solution, as opposed to a cloud-based one, while only 3% responded as being users of BigQuery!
This was somewhat of a surprise considering the recent media buzz around the solution. but not wholly considering Google is not recognized as a major market player by Gartner.
2. Complexity is the main challenge
We found that most respondents, almost 60%, claim their data warehouse difficult to manage. On the bright side, however, 51% of them were satisfied by the solution’s performance.
In addition, 60% of the respondents indicated that their major complaint was complexity. Of the respondents who claim they are challenged by the complexity of their data warehouse solution, 20% work in enterprises using a cloud-based solution; 41% were Redshift users.
That most respondents were technical professionals, such as IT specialists and developers, means that ease-of-use is still a weak spot for data warehouse providers and streamlining operations still a challenge they need to meet.
3. The majority are not using ETL tools
When asked which ETL tools you use, respondents named the three ETL leaders: Segment, Stitch and Talend. However, 61% of respondents are not using any ETL tool, which sheds some light onto the complexity challenge. Not using tools to automate data loading and transformation is a likely cause of dissatisfaction, failing to adopt a data warehouse solution.
In comparison to traditional DWH solutions, Amazon Redshift is still relatively new, and though we believe it will be a lead solution in coming years, there are still management gaps, such as multi-tenancy and simple streamlined loading of data to the cloud.
4. Cloud-based BI adoption is slow
Finally, in seeking to learn which BI tools users preferred, only 35% reported using Tableau (with 67% of those using Redshift). The next major group, 25% of all respondents, report using no BI tool at all.
Following these results and the fact that the majority are not using any ETL tools, we can confidently conclude that AWS Redshift opened the data warehouse opportunity, but organizations are still at the beginning of building an end-to-end BI solution.
Final thoughts
In recent years, the approach to data storage, organization, and analysis has been undergoing a radical change, led by a desire from executives in various industries who envision their organizations as data-driven businesses, and therefore, seek solutions from the industry.
In March, in conjunction with its release of 2016 Magic Quadrant for Data Warehouse and Data Management Solutions for Analytics, leading industry analyst Gartner cautioned all market leaders, including IBM, Microsoft, Oracle, SAP, and Teradata, to recognize the competition facing them as data warehousing has moved into the cloud, in particular from Amazon Redshift, who we see has gained great traction.
Our survey reflects the future of data warehousing; how organizations are becoming increasingly aware of the value of data warehousing beyond simple storage. They’re calling for better ways to extract information from their data and analyze it.
To learn more, download our full Cloud Data Warehouse 2017 Trends Report.
