Universal Connectivity to the rescue!

No more data connector anxiety – create your own custom connector in minutes

Creating your own connector means never having to ask the question, “Does Matillion ETL have a connector for X?” again. By providing an easy-to-use interface for connecting to any REST API, you can quickly connect to any API source system and bring that data into your cloud data warehouse or data lake. Matillion’s wizard driven approach to universal connectivity means unlimited access to cost-effective, fast, future proof data connectivity.

Solving a critical customer issue

One of Matillion’s Core Values is that we are customer-obsessed, and I like to think this feature is a great example of that. We received a lot of feedback from our customers and our partners who all told us about the problems they had with extracting and loading data from their SaaS applications or in-house APIs. The common themes we saw were:

• “Enterprise platforms,” where it is difficult for third-party integrators like Matillion ETL and our competitors to get access, make it challenging to provide a first-class connector.
• API standards vary a lot, which poses obstacles to building a one-size-fits-all solution.
• The APIs that customers want to consume are esoteric or niche and often behind the customers’ firewalls.

When we asked for feedback on the existing solution, the API Query component, the message was clear. The user experience to create an API profile could be frustrating and, at worst, was bad enough that customers would write their own integrations from scratch rather than try to use it. That is not the experience we want for our users.

Still, despite these challenges faced by some users, the API Query component is the third most popular connector used by customers. It is a priority for us to always keep improving the user experience and simplify the data loading process.

Matillion ETL offers improved functionality for connecting to any Rest API – yes, any! – available to all Matillion ETL customers regardless of instance size, at no additional cost.

Connect to any REST API

With huge flexibility on authentication methods, paging types, and data structure, the brand new API Profile Wizards provide an easy-to-follow guided experience to build your own custom connectors. These wizards allow you to test that your profile works before ever building an ETL job.

API Extract Profile Wizard*

If you are familiar with our recent Extract Data Source connectors, such as Recurly, Snapchat, Mandril, Mindsphere, Zendesk Chat and Intercom, the API Extract functionality works similarly. We have exposed the framework we use to build these components to our customers for the first time with API Extract Profile Wizard. Easily build your own custom connectors by configuring your API Extract profiles using a guided wizard. Then you can use the profile with the API Extract Component to load your data into your cloud data warehouse.

API Extract Component*

If you have used our API Query Component, the API Extract Component is configured in that same way. Just set your API Extract Profile in the Properties tab to get started. What is different is the format of the data that is loaded. API Query Component loads data that has been transformed in some way to fit a tabular, structured format. The API Extract Component, on the other hand, loads the raw semi-structured data into your cloud data warehouse. Just like Query Profiles use the API Query Component, Extract Profiles will use the new API Extract Component to execute their connector profiles at runtime.

*Available for Amazon Redshift, Google BigQuery, and Snowflake

API Profile Wizard

Previously, you had to create an RSD file to use in the API Query Component. We have preserved this functionality for all our existing customers who have built integrations. But for new endpoints, the new API Profile Wizard can be used meaning that you never have to do any scripting.

Extract vs Query

We wanted to make sure all of our customers’ existing API profiles would continue to work and that the functionality provided by that technology was more accessible, so we created the API Query Profile Wizard to do that.

We also wanted to give users more flexibility around things like parameters, API versioning, paging options and, most crucially, access to the raw untransformed data provided by the endpoints, a feature requested by customers who use a data lake style architecture. For this purpose, we have the API Extract Profiles and API Extract Component.

To learn more about the difference between Query components and Extract components, and when to use each, take a look at this blog.

Get started building your own connectors today

We think that providing a highly configurable, no-code, extract and load connector framework is a huge improvement to alternatives currently available in the market.

“Matillion ETL’s Create Your Own Connector to build REST API components is the best solution I have tested. As the enterprises we work with continue to combat data silos, this feature will help consolidate data from diverse, bespoke sources and significantly reduce the time-to-value as we can connect to API source systems faster.” – Holt Calder, Data Engineer, InterWorks

We can’t wait to get it in the hands of our customers so you can start getting value as soon as possible. We would love to hear your feedback on how we can keep making this feature even better. Please join the discussion at the METL Community or learn more about how to use the new Create Your Own Connector functionality.

Visit our Universal Connectivity page to sign up for our upcoming Matillion Academy course.