How to build an orchestration pipeline in the Data Productivity Cloud

Data transformation is the cornerstone of refining and enhancing raw data into valuable insights. With Matillion's Data Productivity Cloud, crafting effective data transformation pipelines becomes an intuitive journey, tailored for both data engineers and analysts. Here’s a quick walkthrough of the process to help you get started.

 

1. The process starts in the Pipelines pane, where you can use the Add menu to create an Orchestration pipeline. After naming your pipeline, you’ll be directed to the Designer canvas.  

2. From here, you can go to the Components pane to choose your first component. For example, if your goal is to load data from Excel into your cloud data platform, you would select the Excel Query connector under Orchestration > Load > Connectors in the menu. 

3. After you’ve dragged your chosen component onto the canvas, make sure to connect your component to the Start component in the canvas.

4. Next, you’ll configure it in the Component Properties pane. Here, you can enter any credentials for the source system, select data you want to query, or customize any relevant settings. 

5. With your components in place, you can now validate the pipeline to make sure everything is configured correctly. Green checks mean no issues, and red Xs mean you should take another look at how the component is configured.

6. Finally, once the pipeline is validated, click Run to execute the orchestration.

It’s that easy! Now you can build a simple orchestration, you can start experimenting and customizing to see the full power of the Data Productivity Cloud. And don’t forget to check out this instructional blog on building transformation pipelines, which you can use to manipulate data directly inside your cloud data warehouse.

Sam Wortman
Sam Wortman

Senior Product Marketing Manager