Blog| Matillion ETL for Amazon Redshift

    What’s new in Matillion version 1.38?

    The latest release of Matillion, version 1.38, includes a whole host of new components, new features, and enhancements to make Matillion better than ever before! Upgrade your instance today. For More information on how to upgrade, check out our blog on Best Practices for Updating your Matillion Instance.

    All Matillion Products

    New Components

    Version 1.38 of Matillion introduces a new data staging component for all Matillion products:

    • Sharepoint Query component

     

    New Features

    We are happy to introduce a host of new features that will make your user experience of Matillion even better!

    Project and Environment UI updates

    Two Matillion features that have received user interface quality-of-life improvements are Environments and Projects. Editing or creating an Environment or Project is now done through our simple step-by-step setup wizard.

    Once you have filled out the wizard’s fields, you can click the Test button to ensure that all parameters are working as expected.

     

     

    Note: The functionality of both Environments and Projects remains the same as before.

    Variables text editing

    Another quality-of-life improvement is text editing for variables. This feature is available for both Manage Environment Variables and also Manage Job Variables. In this instance, we have a number of variables, and as usual, they can be edited, added, and removed through the user interface.

    You also have the option to flip this Editor into Text Mode. Text Mode will show a plain text version of these inputs, where the plain text can be edited, and the changes reflected in the relevant columns when we switch Text Mode off. You can also copy the plain text and paste it into another job variable’s Text Mode window.

     

    Matillion v1.38 Variables Text Mode

     

     

    Matillion ETL for Amazon Redshift

     

    New Components

    Version 1.38 of Matillion ETL for Amazon Redshift introduces a new data staging component:

     

    Updated Features

    Change Data Capture (CDC)

    We introduced Change Data Capture (CDC) for Matillion ETL for Amazon Redshift in the last release (v1.37). In this release, we have updated our CDC feature with an improved view panel, giving better access to Queue Status and Lambda Status. In addition, the new 6-step wizard makes setting up tasks much simpler.

    Newly supported databases:

    • Microsoft SQL Server
    • Postgresql
    • Amazon Aurora Postgres

     

    Improvements

    A number of features within Matillion have been improved for a better user experience and parity with Amazon Redshift product enhancements. These include:

    • S3 Unload Component, which now allows unloading to other AWS Regions, can insert header rows to unloaded files, perform BZip2 compression and can add source data types to the generated S3 Manifests.
    • Calculator Component has been updated to include addition JSON and Regex functionality.

    For more information about these, check out the full v1.38 Release Notes on our support site.

    Matillion ETL for Snowflake

     

    New Features

     

    Change Data Capture (CDC)

    We are really excited about introducing the foundations of Change Data Capture (CDC) into our Matillion ETL for Snowflake on AWS. CDC is a technology for efficiently capturing changes made to a source database and applying them to a target database, ensuring your data is in-sync. Matillion’s approach to data transformation looks to leverage the cloud data warehouse for scalability. For this reason, Matillion’s CDC solution leverages Amazon Database Migration Service for Snowflake.

    Currently supported databases:

      • MySQL
      • MariaDB
      • Amazon Aurora MySQL
      • Oracle
      • Microsoft SQL Server
      • Postgresql
    • Amazon Aurora Postgres

     

    New Components

    Azure Queue Storage Message component

    Users of Matillion ETL for Snowflake on Azure-hosted instances now have access to the Azure Queue Storage Message component.

    Using your instance credentials, you can select one of your Azure storage accounts and a pre-existing Queue within that account, (which users must set up). Then, you can write the message you want, choose from a variety of message types including Gzip, base64, and plain text, and then run this component to send your message to the Queue you have selected.

     

    Matillion v1.38 Azure queue storage message

     

    This message is then visible from within Microsoft Azure portal within the relevant Queue.

     

    Matillion v.38 Azure queue storage message

     

    Multi Table Input component

    A new Multi Table Input component is available for Snowflake users when running a Transformation Job.

    This component lets you query your target database for any table whose name matches a certain pattern. The component loads data from all of those matching tables at once provided that their column structure is similar to one another.

    Matillion ETL for BigQuery

     

    New Component

    Version 1.38 of Matillion introduces a new data staging component, in Matillion ETL for BigQuery:

     

    What’s new in Matillion version 1.38?

    For more details on what’s new in Matillion v1.38, watch our release video.

    Keep up to date with new releases and tutorial videos. Subscribe to our YouTube Channel.