Cloud Data Migration: 5 Lessons Learned

Once you’ve decided to modernize your data warehouse, cloud data migration isn’t far behind. In some ways, cloud data migration has some benefits over a traditional data migration. Cloud-based models inherently reduce risk. Pay-per-usage pricing and improved storage and compute capabilities mean that you can spin up, trial, and run an economical proof of concept, largely unmanageable with on-premise resources. It’s a fail-fast method, and can yield a quick ROI.

Cloud data migration can be tricky

However, though some things are easier and less risky, a cloud data migration process can still pose challenges. Organization, proper expectations, and simply adapting to a new way of doing things can make the difference between a successful migration and a brave, but unsuccessful, attempt.

Based on many successful migrations with Matillion customers, here are five important lessons we’ve all learned. Keep them in mind, and there’s an excellent chance you will have a successful migration.

Lesson 1: Prepare for new technical best practices

Not setting expectations can put your initiative at risk. While standing up a new cloud data warehouse is fast and easy, the underlying technology is different from legacy warehouses. Such differences can be overlooked because cloud data warehouses use standard SQL dialects and similar connection options. A cloud data warehouse may warrant new design choices, to take advantage of columnar storage and bulk data loading, for example. The subtle differences will impact how table structures are designed, loaded, and queried, requiring a paradigm shift and a small learning curve.

Lesson 2: Evaluate all of your cloud options

For your cloud data migration, leverage the cloud consumption model. Take advantage of the wide variety of free or low-cost trials to evaluate the available cloud data warehouses. Do the same for all of the complementary solutions you will need to be successful, such as ETL and BI tools. For example, traditional ETL tools aren’t built for the cloud. (To read more about the differences, download our ebook From ETL to ELT.) While they may technically work with a variety of legacy warehouses, they’re unlikely to take advantage of the native improvements and best practices that your chosen cloud data warehouse offers. In fact, it’s more likely that they treat your cloud warehouse like your same old warehouse, which can result in some of the same old performance bottlenecks, appropriately causing you to question why you’re migrating in the first place.

Instead, evaluate new, cloud-native solutions and techniques, so that when it’s time to make the move, you’ve selected the very best that’s on offer for the cloud and data warehouse that suits you. Test things in a variety of use cases, ranging from simple tasks to your most challenging or poorest performing workloads. One of the key benefits to the cloud is the ability to stand up new resources on demand. Embrace it!

Lesson 3: Bring in stakeholders early on

When evaluating cloud data solutions and tools, talk to a wide variety of stakeholders if possible, including, but not limited to:

• Business analysts
• Data scientists
• Project managers
• Heavy consumers
• Power users

Remember, change is constant. But it’s also scary for some. It can be alarming, from an analyst’s perspective, to hear that the data they are familiar with will soon become stale. Not only that, they’ll have to develop rust for a new warehouse all over again. Allowing them to see some of the benefits of cloud data migration early on, and to provide their input on existing pain points, could help consumers become champions for the new warehouse, rather than alarmists.

Lesson 4: Create a plan for cloud data migration

Amazing news: if you’re moving to the cloud, you aren’t working under the typical licensing and hardware constraints that would force you to lift and shift your existing warehouse, and all associated processes, all in one go and on a tight schedule. So don’t rush–make a thoroughly considered plan.

You can migrate workloads gradually. Start with some of your simple workloads as a way to familiarize yourself with new features and to establish benchmarks. Or, start with the migration of raw data from your existing warehouse, and any other data silos that you may want to centralize. Take the opportunity to explore new design patterns, to clean up technical debt, or both.

Don’t just have a plan, have an exit plan. In times past, there was no exit plan because the contracts were signed and the ink long dry before the hardware even arrived at your data center. If things aren’t going as expected, you’ve allowed yourself some room for lessons learned. It should be okay, even encouraged, to change course whenever you need. Go back to your evaluation, pinpoint the missing criteria, and formulate a new plan. The cloud affords flexibility. Take advantage of it.

Lesson 5: Engage with a cloud data migration partner

Even with a rock-solid plan, sometimes there just isn’t time to figure everything out, despite best efforts. Some teams don’t have any bandwidth to spare, so sparing a resource or two for even a few hours a week to work on a modernization initiative isn’t an option. There are a wide variety of partners that perform nothing but data warehouse migrations. This is all they do, and they are very good at it. They stay up to date on the latest tooling and they’re uniquely aware of a variety of pertinent topics that may impact your use cases. And once your migration is complete, they can provide documentation, training, and a proper hand-off to your existing team.

For a list of consulting partners who can help with cloud data migration projects, check out the Matillion partner page.

Matillion can help with cloud data migration

When you’re ready for cloud data migration, Matillion is purpose-built data transformation for the cloud that can help you extract and load data from existing systems, in order to conduct a wide variety of transformations on that data, all using the power of the cloud.

Further, Matillion natively employs best practices ascribed by each cloud data warehouse vendor, taking full advantage of platform specific functionality, allowing your team to focus on what they do best.

For more information on cloud data migration and modernization, download our ebook, Modernize Your Data Warehouse.

The post Cloud Data Migration: 5 Lessons Learned appeared first on Matillion.