
5X
improvement
in data processing speed
>50%
policies migrated
to Snowflake within 6 months
60
seconds
average run job time
About
For more than 150 years, Pacific Life has helped millions of individuals and families with their financial needs through a wide range of life insurance products, annuities, and mutual funds, and offers a variety of investment products and services to individuals, businesses, and pension plans. Pacific Life offers innovative products and services that provide value and financial security for current and future generations. Pacific Life counts more than half of the 100 largest U.S. companies as its clients and has been named one of the 2021 World’s Most Ethical Companies® by the Ethisphere Institute.
Company:
Location:
Industry:
Employees:
Product:
Use Case:
Website:
Pacific Life
Newport Beach, CA
Insurance and Financial Services
~4,000 employees
Matillion ETL for Snowflake
Cloud Data Warehouse, Reporting Analytics
For more than 150 years, Pacific Life has helped millions of individuals and families with their financial needs through a wide range of life insurance products, annuities, and mutual funds, and offers a variety of investment products and services to individuals, businesses, and pension plans. Pacific Life offers innovative products and services that provide value and financial security for current and future generations. Pacific Life counts more than half of the 100 largest U.S. companies as its clients and has been named one of the 2021 World’s Most Ethical Companies® by the Ethisphere Institute.
Company: Location: Industry: Employees: Product: Use Case: Website: | Pacific Life Newport Beach, CA Insurance and Financial Services ~4,000 employees Matillion ETL for Snowflake Cloud Data Warehouse, Reporting Analytics |
Challenge
As a financial services company for more than 150 years, Pacific Life recognizes the value of improved access, reliability, and analysis in regard to its data. From improvement of actuarial risk models, to sales and marketing operations, to business intelligence, a modern data architecture that delivers faster processing of data and one that can scale as demand increases is essential.
Realizing that its legacy on-premises data architecture was not able to scale with its growth and digital transformation, Pacific Life migrated to a cloud data platform that improved and optimized its data processing capabilities, was able to scale up or down as needed, and ensured access to higher quality data to more people within the company
As a financial services company for more than 150 years, Pacific Life recognizes the value of improved access, reliability, and analysis in regard to its data. From improvement of actuarial risk models, to sales and marketing operations, to business intelligence, a modern data architecture that delivers faster processing of data and one that can scale as demand increases is essential.
Realizing that its legacy on-premises data architecture was not able to scale with its growth and digital transformation, Pacific Life migrated to a cloud data platform that improved and optimized its data processing capabilities, was able to scale up or down as needed, and ensured access to higher quality data to more people within the company
Solution
Pacific Life chose Snowflake as its cloud data platform due to its optimization for speed and performance, including its ability to scale up to meet peak loads and then scale back down for fast and affordable data queries. Pacific Life chose Matillion ETL for data integration and transformation to ensure a smooth and rapid migration of data from its on-prem databases to Snowflake and superior ongoing management of its data environment.
The team built a scalable metadata-driven data ingestion framework using Matillion’s pushdown optimization, which delivers high performance processing within the Snowflake environment that is optimized based on the size of the Snowflake warehouse. This allowed source-to-target mapping specifications to be ingested in Snowflake and enabled reusability of Matillion jobs to dynamically leverage parameterized variables to load data across Load, Transform, and Publish layers. Matillion’s data pipeline features they relied on included Git integration, data observability, validation of critical data elements (CDEs), and audit logs for traceability.
The first phase of the project included two major waves over 12 months. The first wave defined the key architectural components, operationalized the foundational data platform, and ingested data from their retail annuity data domain into Snowflake. The second wave implemented integration with finance, hedging and risk management functions. By automating the data integration for finance systems, Pacific Life was able to launch a new relationship with a key business partner.
Pacific Life chose Snowflake as its cloud data platform due to its optimization for speed and performance, including its ability to scale up to meet peak loads and then scale back down for fast and affordable data queries. Pacific Life chose Matillion ETL for data integration and transformation to ensure a smooth and rapid migration of data from its on-prem databases to Snowflake and superior ongoing management of its data environment.
The team built a scalable metadata-driven data ingestion framework using Matillion’s pushdown optimization, which delivers high performance processing within the Snowflake environment that is optimized based on the size of the Snowflake warehouse. This allowed source-to-target mapping specifications to be ingested in Snowflake and enabled reusability of Matillion jobs to dynamically leverage parameterized variables to load data across Load, Transform, and Publish layers. Matillion’s data pipeline features they relied on included Git integration, data observability, validation of critical data elements (CDEs), and audit logs for traceability.
The first phase of the project included two major waves over 12 months. The first wave defined the key architectural components, operationalized the foundational data platform, and ingested data from their retail annuity data domain into Snowflake. The second wave implemented integration with finance, hedging and risk management functions. By automating the data integration for finance systems, Pacific Life was able to launch a new relationship with a key business partner.
Results
- Achieved a 5X improvement in data processing speed, with an average job run-time of less than 60 seconds.
- Launched a production environment in 3 months, versus 12+ months anticipated with legacy solution.
- Implemented continuous integration and processing of data with high availability in the cloud at scale to meet increasing demands of the business.
- Migrated more than half of its policies to Snowflake within six months.
- Gained the ability to add new data sources very quickly with Matillion instead of months with legacy solution.
- Achieved a 5X improvement in data processing speed, with an average job run-time of less than 60 seconds.
- Launched a production environment in 3 months, versus 12+ months anticipated with legacy solution.
- Implemented continuous integration and processing of data with high availability in the cloud at scale to meet increasing demands of the business.
- Migrated more than half of its policies to Snowflake within six months.
- Gained the ability to add new data sources very quickly with Matillion instead of months with legacy solution.
Benefits
Once Pacific Life decided on Snowflake and Matillion, implementing the solution into production went very smoothly, aided greatly by Matillion ETL’s ability to continuously integrate and process data at scale within Snowflake. Within three months, Pacific Life had its production environment up and running; and within six months, the migration of over half of the hedging and valuation data for policies, which is essential for development and maintenance of Pacific Life’s actuarial risk models, had been completed.
Pacific Life’s Data Management team can now ingest, transform, and publish data at least five times faster than tools used previously. The average run time of jobs is less than 60 seconds, making trusted data available faster than ever before to the business.
The process was also greatly accelerated by Matillion’s solution architects, who helped Pacific Life’s data team quickly ramp up on key capabilities of Matillion ETL, including building of data pipelines and developing a framework that supported loading of information into different layers, and reading and working with metadata directly from Matillion.
Once Pacific Life decided on Snowflake and Matillion, implementing the solution into production went very smoothly, aided greatly by Matillion ETL’s ability to continuously integrate and process data at scale within Snowflake. Within three months, Pacific Life had its production environment up and running; and within six months, the migration of over half of the hedging and valuation data for policies, which is essential for development and maintenance of Pacific Life’s actuarial risk models, had been completed.
Pacific Life’s Data Management team can now ingest, transform, and publish data at least five times faster than tools used previously. The average run time of jobs is less than 60 seconds, making trusted data available faster than ever before to the business.
The process was also greatly accelerated by Matillion’s solution architects, who helped Pacific Life’s data team quickly ramp up on key capabilities of Matillion ETL, including building of data pipelines and developing a framework that supported loading of information into different layers, and reading and working with metadata directly from Matillion.
What's next?
Pacific Life plans to continue the modernization of its data environment to improve decision making, as well as the overall operations of its business. It plans to migrate the entire spectrum of their company data to Snowflake with Matillion for an everexpanding slate of use cases – sales and marketing, operations, product development, analytics, and business intelligence, with the goal of becoming a more data-driven organization that supports workflows to a more distributed workforce.
The company also plans to leverage Matillion’s GitHub integration, along with Matillion APIs, to help them implement a more DataOps-focused approach to data management with agility, data quality, and data governance
Pacific Life plans to continue the modernization of its data environment to improve decision making, as well as the overall operations of its business. It plans to migrate the entire spectrum of their company data to Snowflake with Matillion for an everexpanding slate of use cases – sales and marketing, operations, product development, analytics, and business intelligence, with the goal of becoming a more data-driven organization that supports workflows to a more distributed workforce.
The company also plans to leverage Matillion’s GitHub integration, along with Matillion APIs, to help them implement a more DataOps-focused approach to data management with agility, data quality, and data governance
In deciding on a data integration and transformation solution, we considered a number of different tools from open source to custom built solutions. We chose Matillion due to its cloud-native architecture, ease of deployment, adaptability, and smaller ramp-up curve. We have achieved 5X improvement in our data processing speed.Manzoor Mohideen Assistant Vice President, Data Management| Pacific Life
Featured Resources
Matillion earns 2023 Top Rated Award from TrustRadius
We are thrilled to be named a leader in data productivity by TrustRadius with a 2023 Top Rated Award for the fifth consecutive ...
BlogHow TUI Uses Snowflake and Matillion to Address an Analytic Information Gap
One of the world's leading tourism groups, the TUI group umbrella globally incorporates 1600 travel agencies, five airlines ...
Success StoryWestern Union
To support Western Union's customer-centric strategy, the data engineering team built a single source of truth for customer ...