How to build an orchestration pipeline in the Data Productivity Cloud

Data orchestration is a crucial aspect of modern data management, ensuring seamless coordination and execution of tasks within your data integration process. In Matillion's Data Productivity Cloud, you can use orchestration pipelines to move data from source systems into your cloud data warehouse, manage data objects and resources to improve pipeline performance, and coordinate data integration tasks like loading and transformation.  

Here’s a quick walkthrough of the process to help you get started.

1. The process starts in the Pipelines pane, where you can use the Add menu to create an Orchestration pipeline. After naming your pipeline, you’ll be directed to the Designer canvas.  

2. From here, you can go to the Components pane to choose your first component. For example, if your goal is to load data from Excel into your cloud data platform, you would select the Excel Query connector under Orchestration > Load > Connectors in the menu. 

3. After you’ve dragged your chosen component onto the canvas, make sure to connect your component to the Start component in the canvas.

4. Next, you’ll configure it in the Component Properties pane. Here, you can enter any credentials for the source system, select data you want to query, or customize any relevant settings. 

5. With your components in place, you can now validate the pipeline to make sure everything is configured correctly. Green checks mean no issues, and red Xs mean you should take another look at how the component is configured.

6. Finally, once the pipeline is validated, click Run to execute the orchestration.

It’s that easy! Now you can build a simple orchestration, you can start experimenting and customizing to see the full power of the Data Productivity Cloud. And don’t forget to check out this instructional blog on building transformation pipelines, which you can use to manipulate data directly inside your cloud data warehouse.