Learn How to Use Python in Matillion ETL on Matillion Academy

Python has become central to creating data pipelines, helping advance and accelerate data from source systems into data integration tools such as the Matillion platform. Python is a popular coding language because it is easy to learn, versatile, flexible, and can be used for various types of projects. Although Matillion is a low code solution, many data engineers use the platform for scripting part of their ELT workflows — transformation, orchestration, or both — in order to accommodate specific business logic or organizational needs that can get them to data insights faster. 

Matillion’s platform encourages the use of Python and gives enthusiasts of the programming language a way to create the data pipelines they need to make their data useful. Along with our previously released best practices guide, Don’t use Python Script component for transformations, we now have a course on Matillion Academy dedicated to supporting Python users on our platform. 

Python scripts for ELT workflows

The Python Script component allows Matillion users to integrate their own Python scripts into their Matillion ETL jobs. This is commonly used to set variable values based on specific business rules, connect to other cloud data platform services and use the power of Python libraries to achieve business results.

The Matillion platform was developed to make data engineers and developers more productive by delivering components for orchestration and transformation that allow for faster development and increased efficiency. However, we recognize that different data teams have different needs and many data engineers truly enjoy coding within their data workflows. 

This course will cover the best ways to take advantage of the intuitive GUI components that Matillion provides while remaining flexible enough to support the modern data team’s desire for low code and high code preferences.

The course modules will cover the following: 

• Introduction to Python within Matillion ETL
• Introduction to the Python Script component
• Basic Python Script component exercise
• Managing Grid Variables in Python
• Dynamically Applying Business Logic using Python
• Working with Libraries using the Python Script component

Take the course for Python today

Python provides a powerful functionality when utilized by a developer well-versed in manual coding. Matillion likewise, is very powerful at simplifying complex tasks. When keeping best practices in mind, the two can complement each other with the efficiency and power needed to build out ELT pipelines at scale.

Start the course