Skip to main content

Matillion ETL 1.62 Release Blog: Preview for dbt Integration, Updated Ads Connectors, and More

 

The Matillion ETL data integration platform continues to get better in Version 1.62. Keep reading for highlights and details in this newest version.

Summary of release:

9 new and updated connectors/components
4 new features added
1 bug fix and maintenance improvement

 

New and updated data source connectors

We’ve added three new connectors in the 1.62 release and have updated six more to help you keep up with rapidly changing source APIs. In this release, the vast majority of our social media ads connectors have been updated to match the changes being released by major social media platforms soon. We encourage anyone using Matillion ETL for marketing analytics or customer 360 purposes to update as soon as possible to avoid outages in their social advertising data.

 

Google Ads

[Notice: Breaking Change on April 27, 2022]

 

Google is deprecating the Google AdWords API officially starting April 27, 2022. This is a breaking change and the Matillion Google AdWords Query component will stop working on April 27, 2022. Anyone using the Google AdWords component should update to Matillion ETL version 1.62 or later on or before April 27th, and configure the new Google Ads component to avoid any disruption to their pipelines.   Learn more about how to configure the new Google Ads component here.  The Google AdWords component will be officially disabled in a future Matillion ETL release.

 

Twitter Ads connector 

Twitter has released a second API dedicated to ads data, splitting this out from their historical tweets query API. Matillion currently has a single connector for both the Twitter API and Twitter Ads API. We are now in the process of aligning our connectors to Twitter APIs, and as a first step, we are introducing the new Twitter Ads connector.  This is not a breaking change and our Twitter Query component will continue to function as expected, delivering both tweets and twitter ads artifacts. We will be deprecating Twitter Ads endpoints from this component in the future, so all users should update to the new Twitter Ads component soon to avoid future disruption.

 

 

Facebook Ads connector updates 

[Notice: Breaking Change on August 9, 2022]

Facebook will be deprecating some of its API versions used by our Facebook Ads connector. To address this, we have updated this data connector in version 1.62 to match the new paradigm. Facebook will formally discontinue the existing v.12 of the Marketing API on August 9,  2022. Anyone using the Facebook Ads component should update to Matillion ETL version 1.62 or later on or before August 9th to avoid any disruption to their pipelines. 

 

 

LinkedIn Ads connector updates 

We have updated our LinkedIn connector to support LinkedIn Ads in a separate, dedicated connector. This means the endpoints for Linkedin Ads will be deprecated in the old LinkedIn Query component in favor of the new LinkedIn Ads Query component. We encourage you to use the future-proof LinkedIn Ads Query component for all new data pipelines concerning LinkedIn Ads. We recommend phasing out the use of the old LinkedIn Query component from all of your existing jobs in favor of the new LinkedIn Query and LinkedIn Ads Query.

 

The old LinkedIn Query component will be disabled in Matillion ETL 1.65. 

However, the component will continue to work with no impact in previous versions of Matillion ETL.

 

 

Microsoft Dynamics 365 Business Central connector updates 

[Notice: Breaking change in March 2022]

We have updated our Microsoft Dynamics 365 Business Central Query component to support the new OAuth2 protocol deployed by Microsoft in March 2022. We encourage all developers using the Microsoft Dynamics 365 Query component to update to Matillion ETL version 1.62 or later as soon as possible to avoid service disruptions, and configure the OAuth2 access token.

 

GMail Query

Matillion ETL is getting ready for managing tokens in a store. We have updated our GMail Query component. The old component version will be deprecated in METL 1.62 and will be permanently disabled in 1.65.

 

However, the component will continue to work in previous METL versions, without the iToken store enhancements.

 

 

Amazon RDS Query connector for Delta Lake on Databricks, Google BigQuery, and Microsoft Azure Synapse 

We have added the ability to query and extract data from the Amazon RDS database into Delta Lake on Databricks, Google BigQuery, and Microsoft Azure Synapse. Matillion customers on these platforms can now extract data from their RDS operational data stores into their data backend on any cloud of choice. This brings the ultimate flexibility to select your preferred cloud data platform without being locked into the cloud vendor used by your operational teams.

 

 

Salesforce Output connector improvements 

Data is only useful when it is accessible. That’s why Matillion ETL supports synchronizing data from your cloud data platform back into the operational systems used in your business every day. In Matillion ETL 1.62, we’ve made several improvements to our Salesforce Output connector, including the ability to specify whether a job should fail or continue after receiving an error back from the Salesforce POST API, and the ability to batch the output of large jobs synchronizing 10,000 records or more.

 

 

String manipulation improvements in the Calculator component for Google BigQuery 

Matillion ETL version 1.62 adds support for new string manipulation functions on Google BigQuery. Update your instance to start using these commands in the Calculator Component today. Please see Google’s release documentation  for more information about these new functions.

 

To learn more about these components, please visit our documentation page.

 

Preview Labs

 

Matillion is excited to introduce Preview Labs in Matillion ETL 1.62. The first feature released under this umbrella is our dbt support capability. You can enable Preview Labs in your admin menu and will be asked to click through to agree to the Terms & Conditions supporting the Preview Labs functionality. 

 

If your company has a policy of prohibiting the use of click through agreements please do not enable Preview Labs.

 

 

Preview for dbt integration  

We are launching our initial integration with dbt through our Preview Labs functionality available in version 1.62. You will now be able to trigger your existing dbt scripts in a Matillion pipeline without having to write any code or leave your Matillion instance. This feature is available in Preview, and we are actively seeking your feedback. Please email sarah.smith@matillion.com with any feedback you have here so we can ensure that this feature  fits your needs when we launch our GA dbt integration. 

 

Zero Copy Clone support for Snowflake

 

We are constantly updating Matillion ETL to incorporate no-code accessibility to the cutting edge features released on our partner platforms. In Matillion ETL version 1.62, we’ve added support for creating Zero Copy Clones of datasets directly in the Matillion ETL canvas. This feature is available to all accounts on Snowflake across Basic, Advanced, and Enterprise instances. Upgrade today to start creating and working from clones of your datasets so you can run low-cost tests and experiments on your datasets and support a number of other important DataOps workflows.

 

AVRO file support for Delta Lake on Databricks

 

Matillion ETL 1.62 adds the ability to ingest AVRO files from AWS S3 and Azure Blob Storage into Delta Lake on Databricks. This brings parity to our support for this popular file type across all of our platform integrations. Update to version 1.62 today to start integrating these files into your lakehouse and run all of your data science and corporate BI workloads on the same engine!

 

Bug fixes and maintenance

 

Instance migration enhancements

Version 1.62 includes some of our finishing touches on the upgrades we’ve made to our instance migration tool in Matillion ETL. Now it’s easier and more intuitive than ever with the improved usability and UX of the migration tool to stay up on the latest version with the ability to:

  • Remember users’ Git repo details when migrating to a new instance 
  • Migrate user credentials 

 

Product release notes

Full Release notes are available on the Support site: 

 

Ready to upgrade to Matillion v1.62?

 

For more information on how to upgrade, check out our blog on best practices for updating your Matillion ETL instance.