Transforming Customer Support: How Edmund Optics Built a Game-Changing AI Chatbot with Matillion, Snap Analytics & Snowflake

Edmund Optics’ team of 50 global, highly-skilled support engineers with advanced science degrees frequently dealt with a high volume of repetitive and basic technical questions. This consumed valuable time and resources, with an average of 25 minutes spent per inquiry. With a catalog of 34,000-plus components, the volume of queries was at times overwhelming and challenging for the team to manage.

Edmund Optics partnered with Matillion, Snap Analytics, and Snowflake to build a technical support AI chatbot, significantly improving customer support efficiency and reducing time to value.

The Matillion pipeline was built in just two weeks, with the entire implementation completed within 10 weeks. The AI chatbot now provides consistent responses to common queries in seconds, freeing up skilled support engineers to focus on complex issues, while drastically reducing support metrics across the board.

Edmund Optics' global team of over 50 highly-skilled technical support engineers play a pivotal role in delivering expert guidance to customers navigating a diverse catalog of more than 34,000 optical components. These engineers, many of whom hold advanced science degrees and support customers in multiple languages, are best equipped to handle complex, high-value technical inquiries that require deep domain knowledge across a wide range of different markets and applications.

However, as customer demand and product variety grew, so too did the number of routine inquiries the technical support team found themselves dealing with. A substantial portion of time was being directed toward recurring questions that, while important, did not always require specialist insight. On average, engineers were spending around 25 minutes per query, limiting their ability to focus their expertise on more specialized and complex support cases.

This shift placed increasing pressure on operational efficiency and consistency. The team was committed to providing fast, accurate responses, but the growing volume of lower-complexity questions made it harder to maintain a uniformly high customer experience. Additionally, the comprehensive nature of the product catalog meant that new team members faced a steep learning curve, often taking up to six months to become fully proficient.

To continue scaling technical support without compromising quality or increasing costs unsustainably, Edmund Optics recognized the need for a solution that would streamline common inquiries, while preserving the team’s capacity to address more sophisticated customer needs.

The company’s legacy on-premises infrastructure presented an additional challenge. Whilst highly effective for many traditional workloads, the existing setup presented obstacles when trying to implement modern technologies – particularly AI-powered applications, which often work with large volumes of multi-modal data and require dynamic access to models. 

It was clear that a new, more flexible cloud-based approach was required. The company needed a secure platform that could combine and transform all kinds of data from a wide range of sources, and provide the performance and scalability needed for a computationally intensive Gen AI solution.

To address these challenges, Edmund Optics was introduced to Snap Analytics, beginning a fruitful partnership. Together, they developed an innovative technical support chatbot leveraging Retrieval-Augmented Generation (RAG) architecture. This architecture was executed with Matillion on 3,000-5,000 individual unstructured documents, some of which were hundreds of pages long. Critically, all data processing occurred securely within the Snowflake environment – a vital requirement given the company's risk-averse approach to external APIs.

Matillion and Snap Analytics designed a simple yet effective solution stack that included:

• Loading data from various sources and formatting it appropriately with Matillion
• Scanning PDFs for content extraction
• A Steamlit process to support the chatbot front-end
• Utilizing Matillion's new Chunk feature to process large documents when loading to the vector store
• Developing a pipeline to search vector storage, retrieve context, and generate responses

The solution implementation involved several key components:

Matillion Data Processing

• Extracting and staging raw data within Snowflake
• Implementing advanced features like generating embeddings, chunking text, and creating a vector store to support RAG processing
• Simplifying the integration and swapping of various LLM models and easily adjusting parameters in a low-code interface
• Streamlining the data pipeline development process, reducing months of work to just a few weeks

Snowflake’s key capabilities

• A container service within Snowflake to host the solution
• Easy-to-use LLM services through Cortex
• A secure environment ensuring all sensitive data remained within the Snowflake ecosystem

Snap Analytics chatbot capabilities 

• Provides responses to complex technical questions citing sources via RAG
• Translates responses into 9 languages
• Assists with scientific calculations for product applications

One of the biggest challenges was effectively chunking PDFs – breaking down large technical documents into manageable, semantically meaningful pieces. Snap Analytics leveraged Matillion's specialized features to overcome this obstacle, ensuring the chatbot could accurately process and retrieve information from complex technical documentation.

The migration to Matillion Data Productivity Cloud was seamless, and everything Edmund Optics had been doing with Matillion ETL was easily replicated in Data Productivity Cloud. With this migration, they were able to leverage advanced AI features, continuing to build on their existing infrastructure while utilizing the cloud's scalability.

Thanks to Matillion's low-code/no-code design and Snap Analytics' expertise, Edmund Optics was able to build the chatbot's pipelines in a way that is future-proof, scalable, and agile. The entire implementation was completed within 10 weeks, with the Matillion pipeline development taking just a few weeks of that time.

The implementation of the Matillion and Snap Analytics-powered AI chatbot marked a transformative shift in Edmund Optic's operations, delivering the following key outcomes:

Efficiency Gains

The chatbot now resolves repetitive technical queries within seconds, greatly reducing the 25-minute average previously required from support engineers. Engineers now have more time to focus on high-value, complex, challenging customer cases.

Accelerated Deployment

The entire solution was operational within 10 weeks, with the Matillion pipeline developed in just two weeks. This rapid timeline was made possible by avoiding additional infrastructure setup time and cost, thanks to Snowflake Cortex, Matillion Data Productivity Cloud, and Snap Analytics' expertise.

Streamlined Training

Centralizing knowledge within a single repository has significantly reduced the onboarding period for costly new hires. Engineers can now acquire the necessary skills more quickly, boosting productivity and improving onboarding efficiency, bringing value to customers and the wider business much faster.

Measurable Improvements

Since going into production in December, Edmund Optics has started to see significant improvements:

• An estimated 100 hours of engineering time saved per month
• More than 40% reduction in technical support response times
• Early signs of a 2-month reduction in onboarding time for new team members

Scalability

The chatbot enables Edmund Optics to manage increased query volumes without adding to its workforce, ensuring headcount neutrality while maintaining high service levels. This is particularly impactful as the team comprises scientists with advanced degrees – an exceptionally costly resource.

Enhanced Customer Experience

Customers benefit from faster, more consistent responses, reinforcing Edmund Optics’ reputation for exceptional technical support – a vital differentiator in their specialized market.

Continuous Improvement

A built-in feedback loop within the chatbot ensures that its knowledge base evolves, further enhancing its accuracy and reliability.

Through these advancements, Edmund Optics has optimized resource allocation, strengthened its customer service capabilities, and positioned itself for scalable growth. The success of this project not only addressed its immediate operational challenges, but also laid a foundation for future innovation.

Looking ahead, Edmund Optics plans to massively expand the amount of data loaded into Matillion and Snowflake by incorporating more data sources into its Snowflake environment. Building a Medallion architecture will allow for creation of a single source of truth in Snowflake. They also aim to leverage Matillion’s AI features to maintain headcount neutrality while scaling operations. Additionally, the team plans to develop further machine learning and AI models, with Matillion shaping and optimizing the underlying data. 

Edmund Optics has found the migration to the Data Productivity Cloud successful. Being a small team, they particularly benefit from the low-code, no-code element of Matillion’s platform, which enables them to greatly accelerate building pipelines.

For over 80 years Edmund Optics® has been a trusted provider of high-quality optical components and solutions, serving industries like Life Sciences, Biomedical, Industrial Inspection, Semiconductor, and R&D. The company employs 1,300+ people across 19 global locations and continues to grow. Edmund Optics manufacture and supply precision optics, imaging assemblies and photonics equipment, offering fast delivery on an inventory of millions of items from one of the world’s largest selections of optical components. Their 24/7 online chat connects customers with engineers for real-time support across industries, enhancing the customer experience anytime, anywhere.

Snap Analytics is a specialized data and AI consultancy that helps organizations maximize their data investments through innovative solutions. As a trusted partner of both Snowflake and Matillion, Snap Analytics brings deep technical expertise and practical implementation knowledge to help clients transform their data operations and achieve measurable business results.

Matillion is the intelligent data integration platform that empowers data teams to build and manage pipelines faster for AI and analytics - at scale. Matillion enables data teams and data professionals to take advantage of their data, AI and the cloud to build valuable data assets that can be used across a wide variety of applications from analytics to AI use cases. Its unified, AI-powered data integration platform enables data teams to spend less time on manual, repetitive, gritty data engineering work, and more time creating business impact. By removing data friction from data integration workflows, the platform empowers data teams to supercharge productivity.