- Blog
- 01.26.2022
Matillion Data Loader and Matillion ETL: Which One is Right for You?
Once you have a cloud data warehouse (CDW), you’re ready to start using the near-infinite scalability and power of that CDW to transform data. But then what? Each organization’s journey to the cloud is different depending on your data maturity. Some companies will need complex, enterprise-grade data transformation, and need it now. Others first face the huge task of migrating data from multiple sources into the cloud.
Whether you are transforming data or just bringing it into the cloud, you want to do it as quickly and painlessly as possible.
Hand-coding is no longer the answer
Historically, integrating dozens of data sources that contain business critical data into one location has meant hand-coding extract, load, and transformation (ELT) processes. Today, a process 100 percent dependent on hand-coding just isn’t practical. Unless you have an army of engineers to hand-code ETL jobs, it simply puts too much burden on IT or data engineers, especially in large enterprises that need a robust and scalable solution.
Two Matillion solutions for data in the cloud
Matillion has two solutions that are purpose-built for the cloud solutions and involve either no hand-coding or are code-optional. Both products are designed to take advantage of your CDW, relying on it to do the heavy lifting. Matillion Data Loader is a SaaS pipeline solution that helps businesses load data into a CDW. Matillion ETL is a full-featured cloud ETL solution that not only loads data into the cloud but also transforms it as per your business logic.
Which solution is right for you and your business depends on a number of considerations. How much budget do you have? How much time do you have? Who is doing the transforming? We’ve outlined the key features of the two products and the key differences to help you determine which one will work best for you.
Matillion Data Loader and Matillion ETL: Side by Side
Common use cases
| Matillion ETL | Matillion Data Loader | |
|---|---|---|
| Data transformation | Combine individual transformation components to build out complex data transformations, in a guided GUI, for visualizations, business intelligence, reporting, and advanced analytics. | No native transformation capabilities. Matillion Data Loader helps bring data into your CDW. When you’re ready for transformation, you can migrate to Matillion ETL. |
| Data migration and ingestion | Use pre-built connectors to ingest data into the CDW from on-prem and cloud databases, ERPs, CRM and marketing automation platforms, social networks, and more! Use a universal API connector to connect to any API-enabled system. -CDC -Incremental loads -Iterators | Use a simple wizard-driven UI to bring data into your CDW from popular data sources, including applications, files, and databases. |
| Reduce the development life cycle | Decrease development time and improve the resiliency of ETL jobs by removing the need to hand-code. Leverage the object-oriented user interface to quickly create complex ETL jobs. | Enable analysts and citizen data professionals with ability to self-serve and integrate data sources within a CDW, helping to close data silos and reduce dependency on developers and engineers. |
| Automate data workflows | Variety of orchestration components allow you to control and manage your Matillion jobs from start to finish. Use scheduling, alerting, and version control (either internally or using GIT) to manage all of your data workflows in one place - Matillion ETL. | Configure your data pipelines to run at set intervals and get alerts and notifications so you always know that your data is where you need it, when you need it. |
| Improve data analytics performance | Purpose-built for your cloud data warehouse, Matillion ETL leverages the power of the cloud to perform simple to complex data transformations. | Built using the same great technology that underpins our award-winning, cloud-deployed ETL solution, but delivered as a SaaS solution, Matillion Data Loader runs performant pipelines for fast data loading. |
Features
| Matillion ETL | Matillion Data Loader | |
|---|---|---|
| Target warehouses | Amazon Redshift Snowflake (AWS, Azure, GCP) BigQuery | Amazon Redshift Snowflake (AWS, Azure, GCP) BigQuery |
| Sources | 80+ Applications Databases Files API No-SQL Matillion ETL integrations | 10+ Applications Databases Files Matillion Data Loader integrations |
| Extract and Load Data | Build orchestration jobs to extract data from your source systems and load it to your target CDW. | Use the simple wizard to build pipelines to extract data from your source systems and load it to your target CDW. |
| Transform Data | Build transformation jobs, selecting from 30+ components that can be used together to perform complex transformations. Option to use scripting components such as Bash or Python for specific data transformation requirements. | Easy migration path to Matillion ETL for data transformation. |
| Delivery Model | Cloud (machine image) | Fully-managed SaaS |
| Scheduler | Set your orchestration and transformation jobs to run at specified times or at regular frequencies. | Schedule at time based intervals |
| Notifications & Alerting | Leverage your cloud provider messaging services to get alerts and notification when and where you need them (i.e. email, Slack) | Email notifications and alerts at user specified intervals. |
| Common users | Database Administrator CTO Engineer / ETL Developer Head of BI Data Analyst | Business Analyst Marketing Analyst Data Analyst Management Information Analyst Data scientist |
| Concurrency | Teams can work concurrently. Number of users depends on the Matillion ETL instance size | The organization owner can invite team members to work concurrently |
| Role-based permissions | Yes, to a granular level | Yes |
| Automation | Sophisticated flow logic to handle data and Matillion API to trigger jobs. | No |
| Data Stage | Stage data in your own cloud environment | Leave the complexity to us. Choose what region your data is staged in. |
| Data handling | Data integration Data migration Data replication Data transformation Data preparation for analysis Automated ETL Enterprise data workloads | Data integration Data migration Data replication |
| Free Trial | 14 or 30 days depending on the cloud platform provider | Free |
| Pricing | Hourly pay-as-you-go or annual consumption-based contracts | Free |
| Enterprise-grade functionality | Enterprise instances of Matillion ETL come with additional functionality specifically designed for enterprise data workloads | None |
| Ability to add new connectors | Yes, using the universal API Query component. Additional options are coming soon. | No |
| Developer tools | REST API, Matillion API, Scripting components (Bash, Python, SQL) | No |
| Data Replication | CDC on AWS supported Matillion ETL products. Options for full table incremental loading via SELECT/replication key, or timestamp. | Incrementally load data where supported by the data source to load only new records at each run. |
| Data Selection | Yes | Yes |
| Support | Email support from our data transformation experts | Community forum support |
| Deployment | Cloud platform marketplaces | Matillion Data Loader registration |
| Reuse jobs | Shared jobs facility allows developers to develop generic jobs which can be reused across different projects and servers | No |
Selecting the right solution
Still not sure what to choose between Matillion Data Loader and Matillion ETL? Use this chart to help you decide.
Ready to get started?
To learn more about Matillion ETL, request a demo
To get started with Matillion Data Loader, simply register today at dataloader.matillion.com.
The post Matillion Data Loader and Matillion ETL: Which One is Right for You? appeared first on Matillion.
Featured Resources
Data Mesh vs. Data Fabric: Which Approach Is Right for Your Organization? Part 3
In our recent exploration, we've thoroughly analyzed two key ...
eBooks10 Best Practices for Maintaining Data Pipelines
Mastering Data Pipeline Maintenance: A Comprehensive GuideBeyond ...
NewsMatillion Adds AI Power to Pipelines with Amazon Bedrock
Data Productivity Cloud adds Amazon Bedrock to no-code generative ...
Share: