Want the very best Matillion ETL experience? Each new version of is better than the last. Make sure you are on the latest version to take advantage of the features, new components, and improvements introduced in Matillion ETL v1.40.
Ready to upgrade? For more information on how to upgrade, check out our blog on How to Update Matillion – Best Practices.
New Data Connectors
Mandrill Query component
Use the Mandrill Query component to load your Mandrill data into your cloud data warehouse.
Snapchat Query component
Use the Snapchat Query component to load your Snapchat data into your cloud data warehouse.
Recurly Query component
Use the Recurly Query component to load your Recurly data into your cloud data warehouse.
Snowflake Streams Support
Create Stream Component
Snowflake “Streams” records changes made to tables (inserts, updates, deletes) so that you can take actions on that changed data. The Create Stream component introduces support for this functionality within Matillion ETL. The Create Steam component is an orchestration feature that registers change tracking on a table and gives that stream a name.
Stream Input Component
The Stream Input component, is a transformation component in Matillion ETL for Snowflake, to read changes from a stream. This component will detect changes regardless of whether the stream was created by our Create Stream component or within Snowflake.
Support for Structured Data
Construct Struct Component
This component will create structs by simply selecting the columns to go into the Struct. Based on this, Matillion will create the new Struct and populate it with the specified data.
Flatten Struct Component
The product now supports the ability to load and flatten Structs (nested fields) and Arrays (repeated fields).
Extract Nested Data
The Extract Nested Data component allows users to unpack their nested data structures (such as JSONs) guided by a visual tree structure of the data.
The Aggregate component in Matillion ETL for BigQuery now has Array Agg functionality to better handle arrays.
Nested Data Support
A “Define Nested Metadata” check has been added to the following components:
- Create Table
- Create External Table
- Sample Grid
- Metadata Panel
Select this option to transpose nested metadata from a simplified column layout into an intricate tree structure. Then you can add new structures nested within the tree, along with strings, integers, arrays, and other data types.
Read more about how Matillion ETL for BigQuery Supports Structured Data with Structs and Arrays.
What’s new in Matillion: Enterprise Features
Git functionality lets users convert a Matillion project into a Git project. Each Matillion “version” in a Git project points at a specific Git commit, enabling development teams to devise workflows with speed and efficiency. Users can perform actions such as Commit, Create Branch, Configure Remote Repository, Fetch, and Push.
Read more about how Matillion integrates with Git.
Matillion ETL for Amazon Redshift now offers users “ConcurrentConnections” to increase the number of concurrent connections to Amazon Redshift. This allows your jobs to run in parallel. As a result, some users may experience significant decreases in ETL job completion times.
Catalog queries have been updated to improve validation performance.
S3 Load component
To maintain parity with Snowflake on AWS 4, we’ve added properties to the S3 Load Component:
- Stage Database
- Stage Schema
- File Format Database
- File Format Schema
Query Result to Grid component now offers support for Advanced Mode (SQL query)
Now users can write custom SQL as part of their Query Results to Grid component configuration using the Advanced Mode option. This gives developers an added degree of freedom over their data transformations.
Windows Fileshare SMB2/3 Support
The Matillion ETL Data Transfer component and File Iterator component now support Windows Fileshare SMB2/3.
To read more about UI/UX improvements across all Matillion ETL products, check out our Matillion ETL v1.40 Improvements Blog.