What’s New in Matillion Release 1.41: Data Lineage, Query Result to Scalar component, Instagram Query Component, and More

  • Susana Cardoso, Product Analyst
  • October 14, 2019

Matillion ETL Release 1.41 is here

Want the very best Matillion ETL experience? Each new version of Matillion is better than the last. Make sure you are on the latest version to take advantage of the new features, new components, and improvements. Today, we’re excited to announce the new components, features, and improvements introduced in Matillion ETL v1.41.

Take a quick video tour of what’s new in Matillion ETL v1.41 with our Product Director, David Langton:

Ready to upgrade to Matillion v1.41? For more information on how to upgrade, check out our blog on Best Practices for Updating your Matillion ETL Instance.

Here’s a quick overview of what’s new:

New Components

Instagram Query component

Use the Instagram Query component to load your Instagram data into your cloud data warehouse.

Query Result to Scalar component 

Use the new Query Result to Scalar component in Matillion ETL to convert your query result into a scalar value.

Enterprise Features

Concurrent Connections

In release 1.40 we introduced Concurrent Connections for Matillion ETL for Amazon Redshift. This functionality is now available to Snowflake customers who can use “multiple connections” to increase the number of concurrent connections to Snowflake. This allows your jobs to run in parallel and some users may experience significant decreases in ETL job completion times.

Data Lineage

Track your column data backwards as it goes through its transformation process using the Data Lineage feature.

Other Improvements

Sharepoint OAuth

The Sharepoint Query component now supports OAuth as an authentication method.

S3 Load and Unload component 

The S3 Load component now supports ZSTD compressed objects. The S3 Unload component now supports CSV. On Amazon Redshift, the S3 Unload component also allows exporting to a different region than the one Redshift is running on.

Required Parameters for Shared Jobs

Shared Jobs now lets you make some of the parameters optional. This gives the shared job author more control over how the shared job will be used/called by other users.

LDAP Test

A test button has been added to the LDAP GUI to ensure the settings are correct. Testing the configuration before committing will result in fewer connection problems with the server.

Recursive Validation

You can ensure all settings are valid before running by validating a job via the API. To do so, all sub-jobs are recursively validated when a specific job is validating.

BigQuery Table Encryption

When creating a table, you can now set an encryption by specifying the KMS key. This should activate new parameters that allow you to set a Location, Key Ring, and a Key to encrypt your table.

BigQuery Clustered Tables

When creating a table, you can now cluster on a user-defined column. This lets you select up to four custom columns where data will be clustered.

BigQuery Cloud Storage Load

Cloud Storage Load Component now supports AVRO file format.

For more information on updates in Matillion ETL v1.41

Full Release notes are available on the Support Site