What’s new in Matillion ETL version 1.40

August 20, 2019

Dayna Shoemaker

Want the very best Matillion ETL experience? Each new version of is better than the last. Make sure you are on the latest version to take advantage of the features, new components, and improvements introduced in Matillion ETL v1.40.

Ready to upgrade? For more information on how to upgrade, check out our blog on How to Update Matillion – Best Practices.

New Data Connectors

Mandrill Query component

Use the Mandrill Query component to load your Mandrill data into your cloud data warehouse.

Snapchat Query component

Use the Snapchat Query component to load your Snapchat data into your cloud data warehouse.

Recurly Query component

Use the Recurly Query component to load your Recurly data into your cloud data warehouse.

Snowflake Streams Support

Create Stream Component

Snowflake “Streams” records changes made to tables (inserts, updates, deletes) so that you can take actions on that changed data. The Create Stream component introduces support for this functionality within Matillion ETL. The Create Steam component is an orchestration feature that registers change tracking on a table and gives that stream a name.

Stream Input Component

The Stream Input component, is a transformation component in Matillion ETL for Snowflake, to read changes from a stream. This component will detect changes regardless of whether the stream was created by our Create Stream component or within Snowflake.

Support for Structured Data

Construct Struct Component

This component will create structs by simply selecting the columns to go into the Struct. Based on this, Matillion will create the new Struct and populate it with the specified data.

Flatten Struct Component

The product now supports the ability to load and flatten Structs (nested fields) and Arrays (repeated fields).

Extract Nested Data

The Extract Nested Data component allows users to unpack their nested data structures (such as JSONs) guided by a visual tree structure of the data.

Aggregate component

The Aggregate component in Matillion ETL for BigQuery now has Array Agg functionality to better handle arrays.

Nested Data Support

A “Define Nested Metadata” check has been added to the following components:

Create Table
Create External Table
Sample Grid
Metadata Panel

Select this option to transpose nested metadata from a simplified column layout into an intricate tree structure. Then you can add new structures nested within the tree, along with strings, integers, arrays, and other data types.

What’s new in Matillion: Enterprise Features

Git Integration

Git functionality lets users convert a Matillion project into a Git project. Each Matillion “version” in a Git project points at a specific Git commit, enabling development teams to devise workflows with speed and efficiency. Users can perform actions such as Commit, Create Branch, Configure Remote Repository, Fetch, and Push.

Read more about how Matillion integrates with Git.

Concurrent Connections

Matillion ETL for Amazon Redshift now offers users “ConcurrentConnections” to increase the number of concurrent connections to Amazon Redshift. This allows your jobs to run in parallel. As a result, some users may experience significant decreases in ETL job completion times.

Additional Improvements

Catalog Queries

Catalog queries have been updated to improve validation performance.

S3 Load component

To maintain parity with Snowflake on AWS 4, we’ve added properties to the S3 Load Component:

Stage Database
Stage Schema
File Format Database
File Format Schema

Query Result to Grid component now offers support for Advanced Mode (SQL query)

Now users can write custom SQL as part of their Query Results to Grid component configuration using the Advanced Mode option. This gives developers an added degree of freedom over their data transformations.