4 Data Management & Integration Trends to Watch in 2023

If you're wondering what's next for data management, you're not alone. In this constantly evolving data landscape, only organizations that are able to anticipate new tech trends and make future-proof plans continue to remain competitive. 

For the third year in a row, I had the pleasure to discuss the future of data management with our very own Ciaran Dynes and Paul Lacey – in a webinar now available for streaming, 3rd Annual Data Management Trends in 2023. We may not have a crystal ball, but our predictions for last year didn't turn out too bad.  

In this blog post, I'll recap our 2022 predictions and share our thoughts on what 2023 has in store for data integration.  

Scorecard for 2022 predictions: How did we do?

First, let's revisit our predictions for last year and see how they panned out:   

The return of change data capture

Prediction: 2022 would be the year that change data capture (CDC) would take off.  

What happened: CDC provides high-fidelity information about changes within a database, making it useful for data science, analytics, and root cause analysis.

Supporting stat: 60% of those surveyed by IDC say it's becoming easier to capture real-time streaming data, indicating a strong interest in CDC technology. 

Google Trends: CDC did indeed have a resurgence in popularity last year.  

Matillion grade: B+  

Operational analytics move into the CDP

Prediction: There will be an increasing trend of people moving their data into cloud-native platforms and the growth of applications being built on top of the CDPs. We thought we’d see companies start building operational analytics inside the CDPs and that the data would be pushed back into the operational systems as needed. 

What happened: People are looking towards the future of cloud data warehouses, which aligns with the prediction. 

Google Trends: There was an increase in the number of people searching for cloud data warehouses over operational data stores. 

Matillion grade: A  

The year of data ops

Prediction: Data ops will become more mainstream, and companies of all sizes will adopt the data ops practices for automation, testing, and deployment measures for data projects and platforms. 

Google Trends: The data suggests that the interest in the term data ops has remained relatively flat, and there hasn't been a significant increase in its adoption. 

What happened: We’re curious if the lack of interest in data ops is a matter of the term being too new or being dominated by other voices. We plan to continue monitoring the trend and see if the era of data ops is yet to come. 

Matillion grade: B-  

Data mesh debate heats up

Prediction: Data mesh and data fabric will rise in popularity.  

Google Trends: There is a significant increase in interest in these technologies over the past year as more and more people are thinking about decentralized architectures and management paradigms in their work with data. 

Matillion grade: B 

Side note: There is some confusion and lack of consistency in the definition of data mesh, with some people using it to mean real-time API or decentralized management while others are using it to mean data as a product. The definition of data fabric is somewhat different, being more focused on the technologies used to create a federated data plane.  

Cost optimization takes off in the cloud

Prediction: Finding ways to optimize the cost of cloud platforms and resources will become a hot topic in 2022; there will be a lot of growth in this area. 

What happened: The trend has indeed been moving in this direction. Gartner added spin-offs as a critical capability to the Gartner® Magic Quadrant™ for Data Integration Tools for 2022. A recent study by McKinsey also suggests that cloud costs can grow between 20 to 30% per year, and only 15% of companies have a sufficient understanding of cloud unit economics, indicating that there is still much to be learned in terms of managing and cloud costs. 

Google Trends: Trending upward. 

Matillion grade: A  

Predictions for 2023

As we approach a different economic climate, we will see a great shift in data management this year. Watch the video for a deeper dive.  

The year of data productivity

We predict that the market will no longer reward growth at all costs but instead reward efficiency. Data teams will be faced with the challenge of doing more with less, and there will be high scrutiny on data productivity and the management of data costs.  

The right mix of applications, data platforms, integration, and transformation will be necessary to get data to business users quickly and with fewer resources. Automation will also be key to freeing up time for new tasks and projects. 

Our advice: Find the right platforms and tools and focus on automation to help increase productivity.  

Cloud native

We predict that cloud-native technology is an emerging trend that will make this the year of cloud native. ("Cloud native" is defined as the use of cloud technology to achieve greater scale, security, and oversight).  

Cloud continues to grow. By 2025 more than 95% of new digital workloads will be deployed on cloud-native platforms. We’re excited about this trend and believe that it’s an important one to keep an eye on. 

We think this will be a great year to get your head in the clouds!   

Streaming is the new batch

We predict the trend and growth of streaming technology in the data analytics world. No, we’re not talking about Netflix. 

Organizations are increasingly looking for real-time and automated solutions.  The growth in this area is being driven by introducing new capabilities and features by technology partners such as Snowflake and Databricks. 

The economic situation will drive the adoption of streaming as businesses need to make faster decisions. One caution: Not all data can be made into a stream, so it’s important to be selective and laser-focused when investing in streaming technology.  

Research conducted in conjunction with IDC shows that 80% of those surveyed planned to invest in new streaming data capabilities, with the main barrier being developer shortages.  

Generative AI

There is a lot of buzz around generative AI. We predict that AI will play a role in repeatable process automation but will not take over all jobs. The technology has its limits and the output still requires human validation. 

Watch the 3rd Annual Data Management Trends in 2023 on demand to hear our full conversation.