Export and Import Shared Jobs in Matillion ETL

Matillion Exchange hosts Shared Jobs created by Matillion ETL users that can be accessed, downloaded, and utilized in your workflows. Leveraging Shared Jobs, which can be used across projects, reduces the development time providing you with a template that can be edited to meet your specific needs. With the Matillion Exchange, you can either import Shared Jobs into your Matillion ETL instance or export your own Shared Jobs to publish onto the Exchange. It’s one of the easiest ways to share your expertise with the wider Matillion ETL community and help improve our product even more.

Let’s look at how quick and easy it is to export and import Shared Jobs with Matillion ETL. 

Export a Shared Job from your Matillion ETL instance

If you want to contribute a Shared Job to Matillion Exchange, you will need to export it from your Matillion instance. For our purposes here, we will assume that you have already created your Shared Job and it is ready for export. (For more information on creating a Shared Job, check out our blog series). 

In this example, we have a Shared Job that was created to send a message to Slack using Python. To export the job, first go into ‘Manage Shared Jobs’ under the ‘Project’ menu. 

This opens a pop-up window with all of the Shared Jobs in my instance. Under ‘User-Defined’ locate the job to be shared. Once this is selected, all we have to do is click ‘Export’. 

From this new screen, we can confirm the job to be exported by ticking the checkbox next to it (note that you can export more than one job at a time. At this stage, you can also change the downloaded file name. By clicking ‘Export,’ the file(s) will be downloaded to your local machine. 

The new .melt file can be found in your download folder under the filename shown earlier. This file is now ready to be uploaded to Matillion Exchange. 

Import a Shared Job into your Matillion ETL instance

Now let’s look at how you can import a Shared Job into your Matillion ETL instance. In this example, we will show you how to import a job for Slowly Changing Dimensions-Type 2 from Matillion Exchange. A Matillion ETL job that handles Slowly Changing Dimensions can be a challenging and time-consuming job to build. This is a great example of a Shared Job where you can leverage the great work of other Matillion ETL users to be more efficient with your development time. The Shared Job will also provide a template that you can adjust in order for it to meet your specific use case. 

We have already downloaded the job from Matillion Exchange and saved it locally so that it is ready for importing. To get started, open your Matillion ETL instance – we always recommend importing new jobs into your Dev instance – and once again navigate to ‘Manage Shared Jobs’ from the ‘Project’ menu. To import the job, click on the ‘Import’ button.  

From the next screen, you can browse for the file in your local machine to then import it. 

When you click ‘OK’, you will be able to see the folder structure containing your Shared Job. When you are ready to import, click ‘OK’. 

If no issues were detected, you can click OK and you will then be able to view your new Shared Job in your ‘Manage Shared Jobs’ window.

If you click on the  Shared Job, you can see more details about it on the right-hand side of your screen. And If you click OK you will be ready to use your new Shared Job(s). 

Using your new Shared Job

To use this Shared Job, create a new Orchestration Job. In this example, we will create a job called ‘SCD_Job’. Once in the Orchestration Job, you can drag the “Type II” job from the ‘Shared Jobs’ panel onto the canvas and connect it up with the Start component. By clicking into the “Type II” component, you can view the properties and parameters that need to be updated and configure them accordingly. 

Editing a Shared Job

In some cases, Shared Jobs will get you most of the way there but you might want to make some edits and tweaks to it. Shared Jobs are essentially templated jobs and therefore can provide you with some flexibility. To dive into a Shared Job and make the changes you need, you can unpack the job. 

Head over to the ‘Shared Jobs’ panel and find your new Shared Job – “Type II Dimension”. Right-click on the job and select ‘Unpack’. 

This will make the job(s) available in your Explorer so you can configure and use them as needed. 

Ready to try out some Shared Jobs? Register for Matillion Exchange today!