When it comes to data warehousing, sometimes two (or more) clouds are better than one – and many organizations agree. Multi-cloud strategies are growing in numbers. IBM reported that by 2021 98 percent of companies plan to use multiple hybrid clouds – 98 percent!
Many strategies, many clouds
Business problems are not One Size Fits All. Neither are cloud architectures. Therefore, organizations are looking for specific solutions that deliver on their bespoke needs, which can often require multiple different clouds. Additionally, enterprises are looking to multi-cloud for additional resilience and security when it comes to ensuring business continuity and protecting their business and customer data.
Although the benefits are evident, multi-cloud does not come without challenges. Organizations will need to overcome data silos, portability issues, and security concerns to truly optimize on the freedom that multi-cloud offers.
Cloud-agnostic vendors and technologies will be needed to create a unifying layer that allows for the secure passage of data across warehouses, platforms, and regions.
What does multi-cloud mean?
A multi-cloud strategy can come in many varieties. Multi-cloud could mean a mix of public and private cloud infrastructures. It could also mean using different cloud data warehouse (CDW) providers, such as Amazon Redshift and Snowflake. It might mean hosting operational data stores in AWS but transferring and performing analytics on that data in Azure. Or the underlying cloud platforms could differ, such as two different Snowflake instances, one on Google Cloud Platform and a second on Microsoft Azure. These different CDWs could be hosted in different regions. Sometimes multi-cloud is all of these at once.
Why would I use a multi-cloud architecture?
The reasons that a company uses multi-cloud are as diverse as the multi-cloud combinations. Within these, however, there are some common use cases that lead enterprises to design and implement multi-cloud infrastructures.
Cloud providers offer a number of regional data centers that you can leverage to meet regional compliance and sovereignty requirements when it comes to your business data. There are also benefits in choosing a cloud provider based on regional strength and ability to minimize latency. If your organization operates in different regions, it might make sense to deploy in the regions that make sense for your business
Data and disaster recovery
Organizations are taking advantage of multiple cloud platforms, cloud data warehouses, and data lakes to back up their data for peace of mind. Having a separate system with a copy of your data is a great safeguard against cloud outages, disasters, or any other unexpected downtime.
Diversification and avoiding vendor lock-in
Another reason some businesses are using multiple cloud offerings and providers is to diversify. Organizations may want to avoid vendor lock-in. For example, with platform diversification, organizations have a greater degree of flexibility incase pricing, storage or compute offerings change.
Varying teams and data needs
Some companies will choose to invest in different platforms because different teams have an affinity for various underlying technologies. This allows users to take advantage of a service only available on a particular platform. For example, using Sagemaker in AWS but Snowflake on Azure, or Google ML with Snowflake on GCP. By enabling each division with the technology they are comfortable with, experienced in, and that supports their needs, companies can gain efficiencies.
As new cloud data warehouses are spinning up on different platforms, companies have more choices. As preferred platforms introduce additional warehouse preferences, companies may spin up new environments and then use these alongside existing CDWs. Or they may use multi-cloud for a period of time to foster a smooth transition from one cloud environment to another.
What are the challenges with a multi-cloud strategy?
Like any technology strategy, multi-cloud comes with significant benefits, but also has its risks and challenges.
Inherently, a multi-cloud design creates data silos. It allows for data to be stored in different warehouses across different platforms in different locations. While these data silos are unintentional, they can become massive blockers to creating a single source of the truth. As individuals attempt to apply their own business rules, inconsistencies arise in their application of solutions meaning that outputs can differ. This prevents organizations from gaining the knowledge necessary to make data-driven decisions that deliver a competitive advantage.
Data silos are hard to break down because organizations can’t move data that is in different formats and resides in different technologies. Current portability solutions are expensive to obtain and maintain. On the one hand, organizations avoid vendor lock-in by using multiple technologies. The lack of portability that results can be a risk to a multi-cloud strategy.
Finally, data silos and lack of portability endure because moving data from one platform to another – or from one region to another – can also pose a data security risk without proper governance and security controls. Companies need a way to make the most of multi-cloud offerings within an optimal structure that also allows for the secure global movement of data.
How can I solve these challenges?
There are ways to help safeguard against these risks. Different multi-cloud strategies present different options and opportunities for data accessibility, portability, and security.
One solution is ‘cross-cloud’ data sharing. This method uses a unified data management layer: the same type of cloud data warehouse, which can operate on various cloud data platforms. For example, Snowflake customers can launch the Snowflake CDW on Amazon Web Services, Google Cloud Platform, and Microsoft Azure.
In another multi-cloud option, enterprises can use the same underlying cloud platform and different cloud data warehouses. For example, an AWS customer can have both an Amazon Redshift CDW and a Snowflake CDW running on the AWS platform. Or a Google Cloud Platform customer can use both Google BigQuery and Snowflake.
Many clouds, one data transformation solution
No matter what multi-cloud infrastructure you choose, one thing is consistent: To use your data in multiple CDWs, across multiple cloud platforms, across different regions, you are going to need a comprehensive data transformation solution. Matillion is purpose-built for the cloud, and is cloud-agnostic. However, we recommend that as part of your multi-cloud architecture, you include a central, and master, CDW that your other CDWs can feed into. Using a cloud-native data transformation solution like Matillion ETL, you can load data from the many different cloud environments into a central warehouse where you can transform it and use it for advanced analytics.
How can I get started?
To get started with Matillion ETL and your multi-cloud strategy, request a demo today.