An Agents of Data Special: Digging into Generative AI

With more than 15 years of experience at the heart of tech and AI, Eduardo brings a refreshingly pragmatic perspective on how organizations can move beyond buzzwords to deliver real value.

The Democratization of AI

While the underlying technology isn’t new transformers, a neural network architecture introduced in 2017 that uses attention to understand relationships in data, generative AI has achieved something previous AI implementations couldn’t: democratization.

Unlike narrow machine learning models trained for specific tasks like fraud detection or customer segmentation, generative AI offers a universal interface that anyone can use, from cooking dinner to finding purpose in life.

Data: The Foundation That Still Matters

Despite all the excitement around AI capabilities, Eduardo emphasizes a critical truth: "You cannot do AI if you don't have a solid data foundation."

This principle has been core to AWS since its inception, with Jeff Bezos noting data as their most valuable asset nearly 20 years ago.

The lesson for businesses is clear: Success with generative AI isn't about the model itself, but about the quality, timeliness and organization of the data you feed it. 

As Eduardo explains: "Sometimes we focus too much on the model itself, but the problem might be on our side, do we actually have the right data to implement the use case?"

Three Pillars of Enterprise AI

Eduardo identifies three main areas where organizations are successfully implementing generative AI:

• Customer Experience: Enhanced chatbots and virtual assistants that actually work, unlike their frustrating predecessors that always required escalation to human agents.
• Productivity: Code generation and documentation tools that are transforming software development, plus Retrieval-Augmented Generation (RAG) systems that help business users find insights within vast amounts of company data.
• Automation: The emerging frontier of AI agents that can orchestrate complex workflows end-to-end, reducing manual, repetitive processes.

RAG: More Complex Than It Appears

When discussing RAG implementation, Eduardo uses a compelling analogy: "It's like going to an exam. You are the LLM, and you don't know the answer to every single question. The teacher says, 'If you don't know the question, you can open the book, you can try to read the answer, and then you augment your reply."

The challenge scales dramatically with enterprise data volumes. 

Eduardo's practical advice for RAG success includes:

• Test with data volumes close to production scale, not just 5-10% of documents
• Experiment extensively with chunking strategies – how you break documents into pieces
• Don't rely solely on semantic search; combine with keyword matching and question rewriting
• Treat your vector database strategy with the same rigor as traditional data governance

The Agent Evolution

Moving beyond static RAG implementations, Eduardo sees AI agents as the next wave. 

AI agents combine improved reasoning capabilities with the ability to plan, take actions through tool integration and maintain both short and long-term memory. Gartner predicts that by 2028, 30% of business processes will be run by AI agents: a figure Eduardo thinks could be conservative.

Getting Started: Focus on Foundations

For data leaders looking to integrate AI, Eduardo's advice is straightforward: understand the technology before chasing buzzwords, cautioning businesses against trying to be AI-first, before they realize what it actually does and where it could fit within the business. 

Key recommendations include:

• Invest in team upskilling and cross-functional collaboration between IT and business
• Break down traditional silos between technical and business teams
• Start with clear use cases that match your data quality and organizational readiness
• Remember that 99% of the time, it's about processes, people and skills, not the technology.

The Bigger Picture

Perhaps most inspiring is Eduardo's vision of AI as an equalizer:

He points to small startups with just 10-15 people competing with established giants, democratizing access to sophisticated capabilities.

The conversation reinforces a central theme from the Agents of Data series: while the technology is transformative, success still depends on solid data foundations, clear business understanding, and thoughtful implementation. 

As the AI landscape continues evolving at breakneck speed, organizations that focus on these fundamentals will be best positioned to capture real value from generative AI.

Ready to explore how agentic AI can transform your data engineering workflows? Discover Maia, Matillion's agentic data team, and see how automation can help your team focus less on firefighting pipelines and more on driving business impact

Looking to catch the full episode?

Also available on major podcast platforms and Matillion's YouTube.