Matillion ETL 1.64 & Snowflake Zero Copy Clone Spotlight
Leverage Snowflake Zero Copy Clone and Matillion ETL 1.64 to streamline testing and reduce cost
What is Snowflake Zero Copy Clone and why it matters
Matillion ETL 1.64 adds even more benefits for Snowflake customers already using the Zero Copy Clone feature, or perhaps considering to. Our latest release now enables users to also delete clones from within Matillion ETL as part of the clean-up stage after job execution.
Snowflake clones offer a great way to save cost and time because when they are created, no actual data is copied into the new database, but instead pointers to the live data is used. This means that only changed data is maintained in a separate store, resulting in reduced storage requirements (cost) and time taken to configure the cloned environment.
Image: Initial support for Snowflake Zero Copy Clone with Matillion ETL 1.62.
How Matillion ETL 1.64 leverages Snowflake Zero Copy Clone
With Matillion ETL 1.64 we’ve made it super easy to not just create your cloned environments on the fly when executing a job, but also letting you choose how to handle the clean-up stage after the job completes. You can now select to have your cloned database automatically deleted if the job runs successfully, fails, or in both cases.
Matillion ETL provides a simple way for Snowflake users to leverage Zero Copy Clone functionality and run jobs against “as live” data in a cloned environment. This provides the benefit of confidence that your job is running correctly against real live data, without taking the risk of actually changing any live data – while also reducing the overall cost of testing.
This is phase 3 of 4 and the full feature will be available for 1.65. (1.62 was our first incarnation of Zero Copy Clone support – providing support for cloned environments. For example one can create a clone for testing purposes on a new environment against live data, without actually changing the live data). Matillion ETL 1.64 adds more options to the easy-to-use “run job as clone” right-click menu option, by adding options for the clean up stage after a job completes. (See below)
Not only does this feature keep parity with Snowflake features, but is also a great way to save our customers time and cost when testing their data workflows.
Future releases of Matillion will further enhance the usability of these new cloning features with the option to schedule job execution and have these jobs run in a cloned environment to assist with automated testing.
The existing API is also being extended to provide the option to run jobs within a cloned environment. This will allow customers to incorporate this additional capability within scripted pipelines as part of their data testing processes.
Step by Step – Overview Video
Use of assert to verify / assist with testing
Try Zero Copy Clone for yourself
All the benefits of Snowflake’s Zero Copy Clone (reduced storage needs, reduced risk to live data, reduced maintenance overhead to maintain multiple environments, etc.) combined with the ease of use of executing Matillion ETL jobs against cloned environments generated on the fly for testing or production purposes. Upgrade to Matillion ETL version 1.64 to try it for yourself today!