Why Experimentation in Data Analytics is Key to Success

  • Richard Thelwell
  • October 1, 2015

data analytics experimentation It’s perhaps the most famous observation in data analytics—a remark that has triggered untold billions in additional sales revenues, and led to the world’s first serious attempts to harvest point-of-sale data, and turn it into actionable insights.

After presenting the results of a three-month data analytics trial to retailer Tesco’s main board of directors in 1994, Edwina Dunn and her husband Clive Humby—who together headed a small 30-employee data analytics company called DunnHumby—were told by Tesco’s then chairman, Lord MacLaurin:

“What scares me about this is that you know more about my customers after three months, than I know after 30 years.”

The result? A national rollout, the launch of the Tesco Clubcard (in order to link point-of-sale data to customer demographic data), and a doubling of Tesco’s market share in just twelve months.

And, for Edwina Dunn and Clive Humby, the start of a relationship with Tesco and other large consumer-facing companies that would ultimately see their fledgling data analytics business—founded four years earlier in a Chiswick kitchen—valued at over £1 billion.

Perhaps the most famous data analytics experiment in history had just begun.

Success through serendipity

Chance and pure undirected experimentation plays a huge part in scientific discovery. Penicillin, stainless steel, the microwave oven, even Viagra: all were the result of chance discoveries.

data analytics success
Success can often comes from experimentation, and a degree of good fortune.

In each case, scientists were experimenting, to be sure. But those experiments promptly took a different—and more profitable—path when something unexpected turned up.

So too with data analytics.

An early unexpected finding was the now famous correlation, discovered through basket analysis, that when a consumer suddenly started to purchases baby nappies (‘diapers’ in America), beer purchases too were likely to rocket.

Laughingly dismissed as spurious, it turned out to have more than a grain of truth: stay-at-home new parents were indeed buying beer—because going out to bars and pubs was suddenly not an option.

Freedom to find

Savvy companies—think IBM, Apple and Google—have long understood the importance of allowing smart people freedom in which to explore interesting possibilities.

Google even formalised this, with its famous ‘20% rule’, which actively encouraged employees to pursue blue-sky projects. Google Mail and AdSense, for instance, both started out as ‘20%’ projects.

And in any business, data analytics projects should have the same freedom of scope. Just in case a future blockbuster, or surge in sales, lies lurking in the data, ready to be found.

But how best to manage this?

Which isn’t to say, of course, that data analytics projects should simply be allowed to drift, targeted only on seeing what turns up.

Of course not. Normal project disciplines and processes should apply.

But it’s sensible to supplement these disciplines and processes with a few explicit freedoms, giving data analytics teams and individuals carte blanche to look beyond the obvious.

data analytics freedom
Giving employees the freedom to experiment could be the key to your data analytics success

And of course, today’s new tools and techniques—think machine learning, for instance—make it even easier to start looking for correlations where none had previously been expected.

Experimentation starts here

To encourage a sense of experimentation, try the following tactics:

  • Push your data analytics specialists to deliberately investigate (and try) these new tools and techniques.
  • The boundaries of data analytics are expanding fast: push your people to stay ‘current’, and invest in training.
  • Build a little slack into project plans—but make it clear that it’s for experimentation, not slacking off.
  • Build a short Exploratory Data Analysis phase into every project, in which unexpected insights might be uncovered.

The bottom line

Of course, there’s no guarantee that experimentation will deliver, either on a given data analytics project, or at all.

Lady Luck doesn’t always turn up trumps.

But if you don’t try, and see what experimentation can deliver, then one thing is certain—you certainly won’t succeed.

For all the information you need to know about implementing a successful data analytics strategy in your business, download our free complete guide below