Skip to main content

Matillion Data Loader v1.27: More Support for Schema Drift

New Release: Support for Schema Drift in Matillion Data Loader v1.27

 

In the latest release of Matillion Data Loader, even more of our connectors have schema drift support!

 

Since our announcement of schema drift support, our goal has been to incorporate this support into all of our connectors. Nearly all of our connectors now support schema drift, enabling Matillion Data Loader to provide much more flexibility and more control than ever over data pipelines. 

 

Pipelines that self-heal

 

Having pipelines that adapt to schema changes is crucial, especially today when data requirements change so rapidly. With Matillion Data Loader’s self-healing pipelines, if data type changes occur at your data source, then your pipelines will not be affected. Rather, they will automatically adapt to reflect changes, ultimately save valuable time, and provide peace of mind that they won’t fail.

 

How schema drift support works

 

Your schema is the set of rules that govern your tables and the relationships between them. So when the data type is changed–for example, a date column changes to a text column–the schema also needs to update so that your pipeline tool can pick up and decipher the entire range of data. If it doesn’t update on its own, additional code needs to be written in order to encapsulate the change.

 

When Matillion Data Loader extracts and loads the data within the tables, it detects any data changes: Differences in data types, missing columns, anything else that’s new.  Then Matillion Data Loader simply adapts the existing schema to these changes and propagates them to the destination without any delay or failure. 

 

Editable pipelines for further flexibility and time savings

All of our connectors are also editable, which means that they can be amended to accommodate additional columns. This gives users the flexibility to choose which columns they want to include or exclude when loading data into their cloud data platform, and their ability to quickly make changes without having to rebuild pipelines from scratch.

 

See all of our connectors here

 

Try Matillion Data Loader for yourself

 

See for yourself how Matillion Data Loader can help you extract and load your data into Snowflake, Amazon Redshift, or Google Big Query–for free. 

 

Sign up today 

 

The post Matillion Data Loader v1.27: More Support for Schema Drift appeared first on Matillion.