Navigating the intersection of Academia and Industry: A PhD Placement Experience at Matillion

Matillion recently welcomed a PhD student who completed a three-month placement within the Data Science Team and is excited to share their experience. 

About the student

I am a recent PhD student at the University of St Andrews, where I specialise in applying a data-led approach to studying glaciers in Greenland and their response to climate change. During my final year, I had the privilege of a three-month placement with Matillion, where I worked closely with their Data Science team. Although it might seem unrelated to my glacier research, the core essence at Matillion is DATA. My PhD, funded by the Natural Environment Research Council, encourages such placements to shape us as versatile researchers capable of impactful contributions beyond academia. This opportunity at Matillion allowed me to pause my doctoral work for three months, applying and honing skills I could later integrate into my PhD endeavours. This blog recounts my enriching experience and insights gained during this placement.

What I got out of it

During my placement, I experienced a significant shift from individual work in academia to collaborative endeavours at Matillion. The welcoming atmosphere since day one and the comprehensive Onboarding Academy laid a strong foundation. Unlike academia, where projects are often independent, collaborative efforts were encouraged at Matillion. This change in approach fostered a supportive environment where teamwork prevailed over competition. 

The experience enriched my skill set remarkably. I delved into new software like Matillion ETL and Matillion Data Productivity Cloud (DPC), honed data handling abilities, and deepened my understanding of machine learning and AI. Beyond the narrow scope of my PhD, these skills highlighted the practicalities often constrained by academic research. Matillion's industry environment contrasted academia’s rigid requirements of newness and exhaustive prior research, allowing for quicker and more effective work and fostering a culture that prioritises action, problem-solving, and idea development. 

This shift in mindset, emphasising teamwork and action-driven approaches, was the most impactful learning during my tenure at Matillion, overshadowing even the technical skills gained. 

Learning Matillion ETL from scratch

When I joined Matillion, one of the things they wanted from me was a fresh perspective on Matillion ETL. What was it like to come in from the outside, having never used the product, and how quickly could I get up to speed and use it effectively for my own project? 

To give some background, I have been working with large datasets for a number of years and have usually done my data preparation work in either Matlab or R Studio (two popular packages in universities). I also have some experience using Python. However, I had never used a dedicated ETL tool like Matillion’s. 

I started with the Matillion Academy course “Building a Data Warehouse,” which gave me an excellent introduction to the product and its main tools. With the team's help, I delved into the datasets I'd be using. This hands-on exploration helped me become comfortable with the product within about a week. This learning curve was notably faster than my experience with other tools, where it took me around 2 to 3 months to feel proficient.

As a result, I managed all the pre-processing for my project in Matillion ETL - from importing required data to processing and exporting it for machine learning models. Achieving this in such a short time underscores the product's intuitive, step-by-step approach. Towards the end of my placement, I also got to use DPC, which seems to elevate the ease of use offered by Matillion ETL, and that's exciting.

The Project

The project I participated in aimed to develop a model to forecast customer usage of the Matillion ETL product at an account level, employing telemetry data on past product usage for model development and deployment for Redshift. I was able to develop a model that performed very well, and I am confident I would be able to develop a more advanced, complex model with more time at Matillion. The findings enabled us to reconsider the model type we want to develop in the future. They furthered our understanding of the important factors in forecasting usage at an account level. 

Learnings

At the end of the placement, I felt adept with the software and familiar with Matillion’s operational methods. I generated genuinely valuable insights that I presented to the Product team in my final week. My project has laid a strong foundation for further work and the potential development of a more intricate model. I believe it proved beneficial for the team to have a current academic perspective contributing to creative problem-solving. Diverse thinking is crucial in any successful team, especially noticeable in the Data Science Team, where we all learn from each other's unique skill sets and problem-solving approaches.

Conclusion

In retrospect, my time at Matillion was more than just a professional stint; it was a bridge connecting academia with the practical world of data science. The collaborative sprint, the emphasis on action-driven problem solving, and the invaluable exposure to cutting-edge technologies like Matillion ETL and the Data Productivity Cloud significantly broadened my horizons. 

The Manager’s Perspective

Having a PhD student join our Matillion Data Science Team was invaluable. Their unique academic background in glaciers and climate change brought a fresh approach to problem-solving, emphasising simpler, effective modelling techniques like the time-series approach for usage forecasting. Their adaptability across fields highlighted the versatility of data analysis skills.

Their independent work and emphasis on simplicity reshaped our problem-solving approach, impacting our team positively. This individual's contributions showcased the importance of diverse mindsets in preventing biases in AI applications.

They showcased the broad applicability of data science skills, diversifying our team and offering a new recruitment avenue. Their impact was significant, enriching our team dynamics. We wish them continued success in their pursuits.

Discover your own intersection of academic depth and practical industry applications through placement opportunities at Matillion. Stay updated with our careers page to explore and embark on a journey that bridges academia with real-world impact.