Autodesk makes software that helps customers to design and make a better world for all. Its technology spans architecture, engineering, construction, product design, manufacturing, media and entertainment, empowering innovators everywhere to solve challenges big and small.

Like most modern data-savvy organizations, Autodesk is on a journey to fully exploit the data that the business generates, converting it into new opportunities for growth and long-term sustainability. This means that the data team is under continuous pressure to transform data faster, make it available more quickly, and build solutions that operate in near real time.

The Autodesk Data Platform is the largest data lake at Autodesk, used by people across the enterprise in a self-serve manner to obtain insight into product usage from instrumentation data being captured and stored on the cloud. Users from across the business can perform analytics on shared data sets, or can combine their own data to satisfy advanced requirements. The Autodesk data team needs to support a diverse group of consumers who need the ability to build something unique to their own needs, but on a common platform.

Without this important instrumentation data, users wouldn’t be able to produce accurate scoring models, negatively impacting the ability to generate quality new leads. Says Mark Kidwell, Chief Data Architect, Autodesk Data Platform: “Our Salesforce salespeople would just be sitting there waiting for more stuff to do, people to call.”

After evaluating several technologies against their use cases, Autodesk chose Snowflake as their common enterprise data platform of choice. The data team then embarked on a data integration project that took nearly a year to implement. Because Snowflake was an integral part of their data engine and tool stack, Autodesk needed an ETL solution that would facilitate both rapid implementation and frictionless adoption, making the most of the native power and features of Snowflake.

Early on, Autodesk identified a need to simultaneously support two approaches to data pipeline automation: a “low-code” solution to cater to their more advanced needs as well as a “no-code” managed service. Autodesk couldn’t compromise on some of the sophistication offered by custom coded solutions, even though a key criteria for a new solution was ease of use, to facilitate rapid adoption and faster time to value. This dual challenge quickly eliminated ETL providers such as Informatica, which was based on older ETL technology and too hard to use for all but the most technical users. In fact, many ETL tools Autodesk considered relied too much on users acquiring (and maintaining) scripting skills. Matillion ETL’s visual design approach clearly offered a better alternative to a wider range of users. Today, Autodesk’s data integration team of about 10 people uses Matillion ETL for Snowflake extensively, citing the ease of use and flexibility to support both ELT and ETL paradigms as some of the key reasons why.

The Autodesk data team historically used a high proportion of highly technical staff to do complex coding work on the data lake. However, the company soon realized that it could not sustain that level of staffing and maintenance. The team needed to take a no-code approach for the bulk of data ingestion, combined with a low-code approach for more complex data transformation work.

Matillion ETL combines both ease of use and support for complex transformations, by hiding the complexity unless required by the user. This elegant approach is a key enabler in overcoming the dual challenge of simultaneously providing ease of use and sophistication. The switch to Matillion ETL also enabled Autodesk to attract and quickly onboard new talent more familiar with modern visual design paradigms such as drop-down boxes and drag-and-drop features.

• Using Snowflake and Matillion ETL, Autodesk has been able to achieve two important business data goals
• The new solution has enabled the data team to provide transformed, analytics-ready data that data scientists, such as financial analysts using the financial reporting platform and data mart, can use in models to more accurately and effectively score leads and predict churn and customer losses. 
• Autodesk has vastly improved the velocity and quality of enterprise data by retiring legacy technology that was responsible for 90 percent of their data quality issues. The previous experience, built around Spark and legacy ingestion tooling, was difficult to use and error prone. By adopting Matillion ETL, Autodesk simplified their entire process of ingestion, transformation, and analytics.

Autodesk is continuing to expand its adoption of Snowflake to allow more users than ever access to useful data. The company is now exploring the possibility of making Matillion available as a platform-level offering for any staff member who wants to adjust their own data, in the same way the data team and data scientists do today. By increasing the availability of data and analytics across the business, Autodesk hopes to increase the value of timely, quality enterprise data availability across the organization.