Skip to main content

New in Matillion ETL Release 1.58 and 1.59: MergeManager

 

Want the very best Matillion ETL experience? Each new version of Matillion is better than the last. Make sure you are on the latest version to take advantage of the new features, new components, and improvements. Today, we’re excited to talk about MergeManager, available in Matillion ETL releases 1.58 and 1.59.

 

Within Matillion ETL, customers have the option to use Git to implement source control.

 

As with all forms of source control, managing the changes from a multitude of different sources, possibly changing the same underlying components, is a challenge.

 

Until now, the onus has been on the so-called integration manager to determine where changes have been made, which changes should be kept, and which ones should be discarded–or, in some cases, manually replayed into the codebase following the merging of a branch.

 

With the current implementation of Git in Matillion ETL, the assistance provided to the integration manager was somewhat limited. When attempting to merge in branches containing conflicting changes, the UI simply reports that the conflict exists and allows you to  either retain the changes in our branch or overwrite our branch with the changes from the incoming (their) branch.

 

In many cases, the integration manager would need to take investigative action outside of Matillion ETL to determine what the conflicting changes were before returning to Matillion to complete the merge process.

 

Now in Matillion ETL: A narrative user interface

With Releases 1.58 and 1.59, Matillion ETL now provides an enhanced user interface to help when merging conflicting branches.

How does MergeManager work?

The following example shows how  you can resolve  a merge conflict using the functionality provided with release 1.58 of the Matillion ETL product.

 

For this example, we use a very simple pair of transformation jobs, shown below. Various properties associated with the Aggregation  component are modified and committed to parallel development branches in order to intentionally create a merge conflict situation.

 

In addition, we  assume that the project being used has had a Git repository initialized and that the customer is familiar with the operations required to check out, commit, and merge branches within the Git integration feature in Matillion ETL.

 

 

Following the changes, commits, and merge of one of the branches, we have a Git-enabled project with merged / unmerged branches as shown below:

 

With the dev-changes branch already merged, we now wish to incorporate the ux-changes branch into master:

 

 

As expected, due to the conflicting changes in the ux-changes branch, the system detects a merge conflict and we receive the warning:

 

 

So far in the process we have seen no difference in how  the Matillion ETL handles the merge conflict compared to earlier versions. The first improvement in version 1.58 is the addition of a View option to the resolution screen showing information on the  jobs raising the merge conflict:

 

The View action item, when selected, shows the conflicting changes within the components of the selected job.

 

So, in this case, when we view t_job1, we see the following narrative conflicts view: 

 

 

By default, the user interface shows only the conflicts found between the branches. But by selecting the SHOW FULL VIEW, you can review all aspects of all components within the relevant job.

 

Having reviewed the cause of the conflicts for the selected job, we can return to the previous screen in order to resolve the conflict by selecting Ours or Theirs from the dropdown options in the Choice column (with the same resolution level that exists  in recent releases of Matillion ETL).

 

Upcoming enhancements: What’s next for MergeManager

Going forward, we are working to enhance the conflicts screen even further to allow you to select changes at a more granular level within the job. With these additional enhancements, you will be able to adjust settings from an Ours / Theirs selection dropdown within the conflicts  screen, as depicted below:

 

 

The current plan is to have this improved functionality available in early 2022.

 

We are also looking at how this feature can be applied to other aspects of the Matillion ETL product such as identification of differences between the current job and the last committed version of the same job in Git.

Upgrade to the latest version of Matillion

 

Ready to upgrade? For more information on how to upgrade, check out our blog on How to Update Matillion – Best Practices.

 

See Matillion ETL for yourself

Want to see how the latest functionality in Matillion ETL can help your enterprise? Get a demo to see what’s new.

 

Get a demo