Big data: Use the right analytics for the business problem

Click here to view original web page at

Big Data analytics isn’t always the magical stuff it’s cracked up to be. With data, as with other things, sometimes less is more.

I have often encountered this in my own work. Prospective clients often come to me with an idea of how they want to deal with their data, from a technical standpoint. The prospect has a particular technology or process in mind and will accept no alternatives. Or they’ve decided exactly what steps they want to take. All they want to know from me is how much it will cost to do it their way. The short answer: doing it their way usually costs way more than necessary to get the information they need.

Now, when I tell a prospect that the original plan could cost, say, $2 million, but I can suggest an alternative that will cost only a tenth of that and produce better results, you’d think I’d get at least a big, fat “thank you” in response. And often, that’s just what happens. But not always.

It’s not that I have some secret cheap technology to offer. It’s a matter of knowing how to get the most from what’s available. One way to get the most value from data is to avoid using more data than you need to address your business problem.

Some business challenges truly require a lot of data. A retailer who needs tailored product suggestions for each of several million customers can get value from purchasing history, browsing history, real-time location data and more about each and every one of them. But if the same retailer wants to know what percentage of social media comments on a particular product are negative, a very small data sample is sufficient.

Many business problems can be addressed with less data than expected. Perhaps only 10% as much, or 1%, or even less. Since handling smaller data sets reduces hardware requirements, software licensing fees and data management expenses, costs go down. Smaller datasets are easier to clean and analyze, and they leave analysts with more options for analytic processes, creating opportunities to derive better quality results.

So, what’s the problem? Pride.

Some folks get a person thrill out of bragging about the big, honking data sources they’ve tackled. Some get a lot of attention for that. And some get money and job security out of it, too. And when the business needs those big, honking data sources, that’s appropriate. What’s not appropriate is the proliferation of cases where pride (and maybe a dose of ignorance) drives people to use a lot more data than necessary. If this is happening in your business, you’re wasting money, and lots of it.

So, what can you do? First, focus on the problem you need to solve first, and save technology details for later. Then, grill your staff. Ask questions:

Don’t look for Big Data solutions. Look for “data-that-addresses-my-problem” solutions. Spend less on hardware, programming and IT, but perhaps a little more to train your people in sampling and analytic methods.

Avoid Big Data breakdown. Give your people a better chance to take pride in the quality of analysis, instead of the quantity.

Source: thoughtsoncloud