Cloud data migration is no one’s favorite job. It can be daunting, complicated, and even outright painful if you’re not prepared. It’s often hard to even know where to start. But the right tool (like Matillion Data Loader) and the right planning before you load your first batch of data can help ensure a successful cloud data migration.
You might be shifting from an on-premises data warehouse to one in the cloud. Or you may be jumping directly into a cloud data warehouse as part of a cloud-first approach. Either way, these tips for planning and implementation of a successful cloud data migration will help the process go more smoothly. And you’ll bypass many first-time challenges.
1. Inventory your data sources
Creating a comprehensive inventory of all your data sources before you begin the migration process. It reduces your risk of discovering halfway through the process that you have no access to a data source – or that you forgot to include it in your migration plan. We’ve developed a free template to help you track what your data sources are, how to authenticate with them, and the requirements to access their data.
2. Confirm your connections to data sources
Your data loader needs to be able to authenticate with each data source you plan to consolidate or unify. That means making sure it has the right credentials and requirements necessary to access those systems and extract the data. Ask these questions to confirm you have the information you need:
- Can the data loader connect to the data source at all?
- What authentication does the data source require for access?
- If a third-party data source requires authentication such as OAuth: What needs to be done on that third-party system to enable that authentication?
- Who can provide me with this information?
You may need to communicate with one or more administrators for each data source. For example, to migrate data from an on-premises database, you may need to ask someone on the networking team to allow a connection between the source and the data loader tool. Then you’ll need to get a login and password from the database administrator.
For a cloud or third-party data source, you may need to find the person who has administrator privileges with that vendor. Try to determine in advance if one person is responsible for access to multiple data sources so you can consolidate your requests.
3. Focus on business impact
A successful data migration project isn’t just one in which you move data. It’s one in which you move the data that helps deliver success for a business project. The key stakeholders who rely on the data from each source you’re migrating – that is, the people who depend on the outcome of your project – are the people who define “success.”
What data is most important to them, and what do they consider important in getting that data? Once the data is migrated, is it what they expected? And is it what they actually need (for example, to derive a critical KPI)? Use their answers to keep refining your migration.
4. Start with a high-visibility, high-value project
You already know that quick wins help secure buy-in for any IT project. For successful data migration, that means targeting data that already delivers high value. Then it means finding a way to make it even more valuable. One popular early use case is to migrate data from sources that drive widely known, well-defined KPIs, such as average order value or the click-through range on email marketing campaigns.
5. Avoid unknowns and “gotchas” at first
Achieving a quick win that builds your project’s reputation also means starting with data and processes your organization knows well and uses frequently, or third parties for which you have a dedicated connector with proven reliability. Unless your only data sources are complex migrations, such as custom API integrations, save those migrations until after you’ve delivered a successful first project.
6. Take small bites so you don’t choke
Break up your data migration process into manageable chunks, with clearly defined criteria for success for each one. For example, if your organization uses one of the common third-party email marketing services, your first step might simply be migrating the email data for one specific campaign from the third party’s systems into your data warehouse. The next steps might be to extract that campaign data – bounce rates, open rates, clickthroughs, etc. – one data point at a time.
7. Set reasonable expectations
As with any complex project, data migration can seem to drag on endlessly without achieving any benchmarks for success. Avoiding that perception is key to maintaining support among your key stakeholders. Keep the bar at an appropriate level by establishing frequent milestones you know you can reach and delivering incremental but unarguable results.
Matillion Data Loader can help ensure a successful cloud data migration
With a code-free, wizard-based pipeline builder to common data sources like Salesforce, Google Analytics, and more, Matillion Data Loader can help make your first data migration (and every one after) quick and easy. It’s also free. Sign up today to try Matillion Data Loader and kickstart your data migration project.