Maintaining SCD Data Warehouse With Matillion ETL for Amazon Redshift

  • Richard Thelwell
  • April 27, 2016

In the latest video in our Amazon Redshift tutorial series, we walk you through an example of a Slowly Changing Dimension using Matillion ETL for Amazon Redshift.

Our Matillion ETL for Amazon Redshift customers often require the ability to maintain Slowly Changing Dimensions (SCD), in particular reference to Type 6/Hybrid SCD. You can read more about Slowly Changing Dimensions in this article from the Kimball Group.

Slowly Changing Dimension is becoming an increasingly common customer requirement as their data warehousing projects reach greater levels of maturity. It allows improved visibility of the transition of data over time in a data warehouse, not just offering a snapshot of the current data image.

We’ve created a short video walk through to show you how to maintain Slowly Changing Dimensions in Matillion ETL for Amazon Redshift. You can watch this video below. 

Detect Changes component

There are a number of features in Matillion ETL for Amazon Redshift aimed at simplifying this maintenance of Slowly Changing Dimensions. One of the most important, is the ‘Detect Changes’ component.

Screenshot image of the "Detect Changes" component in maintaining SCD in Matillion's ETL Redshift.

This component allows you to compare two input flows and detect if rows are identical, new, deleted or changed. This component supports a number of Data Warehousing use cases such as simplifying development of Slowly Changing Dimensions.

You can find out more about the Detect Changes component on our Redshift support portal.

Try it out for yourself

If you would like to work through an example job, showing you how to maintain Slowly Changing Dimension data in Matillion ETL for Amazon Redshift, you can find a further tutorial on our Redshift support site.

Screenshot image of the initial population of SCD in Matillion's ETL Redshift

This tutorial, complete with screenshots, provides the step-by-step actions you need to recreate the simple, repeatable and scaleable approach to storing SCD data via Matillion ETL Redshift.

Want to find out more?

You can subscribe to the Matillion Youtube channel for more great videos like this. We have a dedicated Redshift Tutorial Series playlist.

Alternatively, download our free guide below, containing all of the Amazon Redshift advice and best-practices that you need to follow.

Optimizing Amazon Redshift Free Ebook