Fixed flow in focus: Simplifying data pipelines with static inputs

Matillion's Fixed Flow component is a useful tool for generating static input data or using variables as input. Its key advantages are:

• It allows you to create simple static mappings by generating fixed lines of input data.
• You can use variables as input, making it easy to dynamically change the input data based on different environments or configurations.
• It is helpful for testing and debugging purposes, as you can quickly generate test data without relying on external sources.

The Fixed Flow component provides flexibility in defining input data for your Matillion pipelines. By generating fixed input lines or using variables, you can streamline data transformations and simplify the mapping process, especially for straightforward use cases. Its ability to leverage variables makes it a valuable component for maintaining consistency across different environments and configurations.

In my example I will use Matillion’s Fixed Flow component to take the XML data from a Python Pushdown component and load the XML into a table in Snowflake. My use case is executing an XML API call using the Python Pushdown component and loading the XML retrieved into a Project Variable called XML. 

The Python code is quite straightforward. It gets executed in Snowflake, and Matillion provides great integration with variables and python. Notice the last line context.updateVariable('xml', response.text) where I update the Matillion variable xml with the XML response data from the API call.

Using the `xml` variable, I will use the Fixed Flow Component to pass the XML to a Rewrite Table Component and load the XML into a Variant data type column in a Snowflake Table. Here is the configuration of the Fixed Flow Component, pretty simple right!

Connect a Rewrite Table Component to the Fixed Flow Component and your XML from the XML API call is now in a Snowflake Table ready to be parsed and flattened.

Follow me on LinkedIn for more tips and tricks with Matillion!

If you're already a Data Productivity Cloud user, you can download these pipelines from the Matillion Exchange and run them yourself. You can also get started for free or book a demo to explore everything Matillion has to offer.