In the 2020 Gartner Magic Quadrant for Data Integration report, Gartner reported, through 2025, over 80 percent of organizations will use more than one cloud service provider (CSP) for their data and analytics use cases, making it critical for them to prioritize an independent and CSP-neutral integration technology to avoid vendor lock-ins.1
In order to support data migration and consolidation, data integration tools must make use of cloud environments but the strategy of hybrid cloud, multi-cloud, or single cloud is up to the individual vendor. One thing is clear – the adoption of cloud data platforms will only grow. The multi-cloud strategy has many advantages but enterprises should be thoughtful about their approach in order to ensure they can quickly address the needs of their business and remain competitive as they ready their data for analytics in the cloud.
Technology vendors must support data movement across cloud environments
Data portability is an important requirement for a multi-cloud strategy as data sharing and access across clouds will enable data democratization and faster time to insights. As a component of your multi-cloud architecture, you should have a central or master cloud data warehouse (CDW) that your other cloud data platforms feed into. This will help centralize data ingestion and make it easier to track where the data came from.
Using Matillion ETL or other cloud-native solutions will allow you to load data from other clouds into your central CDW. There you can transform it into analytics-ready data for advanced analytics use cases, machine learning, or other business-critical processes.
Things to keep in mind for enterprise multi-cloud strategy
While there has been more adoption for multi-cloud, it is certainly not the norm for most businesses. In a recent IDG/Matillion survey of 200 enterprises data professionals, 57 percent said they will leverage a hybrid cloud strategy (on-premises and cloud) for data management, while only 22 percent are planning a multi-cloud strategy, and 21 percent will use a single cloud provider to manage all their cloud-based data.
Early adoption comes with its own set of challenges so enterprises should keep these things in mind while planning.
Expect friction between technology providers: A mistake enterprises make when going multi-cloud is forgetting to account for the complex challenges that come with more data and differing technologies. Integration between data sources and solutions will be tough since not all providers offer seamless connections with other tools. That is why it is so important that organizations use technology that is cloud-agnostic, saves on costs and seamlessly provides better transition between clouds.
Get your team up to speed with cloud tools: A multi-cloud strategy requires specialized skills to glean insights out of data, quickly. A team of data professionals that previously hand-coded ETL processes or worked with on-premises systems will need time to learn about the modern cloud data architecture and how to best implement a multi-cloud strategy. This team should include people with the ability to organize data for analysis and reporting that leads to actionable insights for the business. Essentially, data engineers with cloud training and experience and are well-versed in all things data.
Keep data security top of mind: Moving data from one platform to another – or from a geographic region to another – poses data security risks if there are not proper governance and security controls in place. Companies need a way to make the most of multi-cloud offerings within an optimal structure that also allows for the secure global movement of data.
Solutions should be born in the cloud for faster time to value
As mentioned above, to have a successful multi-cloud strategy, companies need to use cloud-agnostic solutions that will work with more than one cloud data platform. Different cloud providers do certain things better than others. For example, you might find that the data warehouse platform that works best for your business is Amazon Redshift but Google Cloud Platform has a better machine learning tool for processing that data and inferring meaning.
However, many legacy tools work in hybrid environments and were not built specifically for cloud computing. Solutions that are purpose-built for cloud platforms can take advantage of the power and performance of a cloud environment, making them much faster to perform ingestion, transformation, and orchestration jobs. A solution like Matillion ETL is built specifically for and also enhances the major cloud platform (Snowflake, Amazon Redshift, Azure Synapse, Databricks Delta Lake, and Google Big Query), to ensure you have the right strategies aligned to the right platform.
The state of data integration for cloud platforms
Learn more about the data integration challenges of an accelerated shift to hybrid and multi-cloud deployments and how to avoid cloud service provider (CSP) lock-in through considered ETL vendor selection in the 2020 Gartner Magic Quadrant for Data Integration Tools report.
1Gartner, “Magic Quadrant for Data Integration Tools,” by Ehtisham Zaidi, Eric Thoo, Nick Heudecker, Sharat Menon, Robert Thanaraj, August 18, 2020.