Tag Archives: Big Data

“Big Data” Shifts Business Trends

The New Information Age

We are just beginning to wrap our heads around the vast amounts of information available to us in this new information age. With computer processor speed doubling every 18 months along with increasing storage capacity, we can afford the means to assess data in its entirety. This phenomena is known as “Big Data” which refers to our new found ability to crunch a vast quantity of information, analyze it instantly, and draw some astonishing conclusions.

Collecting information is just the beginning of creating the ability of artificial intelligence as we saw way back in Stanley Kubrick’s 2001 Space Odyssey

HAL 9000 (a computer) decides what’s best…………….

Gone Are The Days Of Traditional Statistical Analysis

In the old days, statisticians would have a hypothesis and test it with a random sample of a population. The greater the size sample of the population being measured, the more accuracy of the sample being a representation of the entire populace. This would reinforce the notion of “causality”, that being, we as humans have a yearning desire to try to understand the world around us. What causes things to happen? For example, say we have a theory that someone buys a home because they eat eggs in the morning.  As you can imagine, there’s a fair amount of people who get up in the morning, fix themselves some eggs for breakfast and go off into their day. And, of people who purchase homes, it can be said that there may be a high degree that they also had eggs for breakfast. Thus, eating eggs causes home sales? So, is it realistic to prove this preposterous hypothesis with a statistical analysis? However, if one was to gather a random sample of new homeowners, there would be proof that would suggest that egg consumption yields new home buying behavior. This type analysis is known as causality as opposed to correlation and takes the form of trying to understand a phenomena, coming up with a hypothesis, and proving it through statistic analysis.

Causality vs. Correlation Analysis

Statistical Analysis vs Causality

Some things are best left to try not to understand












Within a correlation model of understanding, we are able to assess just pure raw data and observe correlations without trying to understand the cause of an event. What this enables businesses to do, is to view highly correlated events and capitalize from their relatedness. In other words, if a given situation yields outcomes with a high degree of correlation, then why bother trying to understand why?

In the book, Big Data – A Revolution That Will Transform How We Live, Work, and Think the author explains how retailers are just beginning to understand their abilities to assess data for what its worth and understand the correlations of relationships. An illustration of this is how Walmart now sells Pop Tarts during times that precede a natural disaster. They observed that during times where people are preparing for hurricanes and/or a natural disaster, that not only did typical disaster items such as batteries, flash lights, and water sales spike but so did Pop Tarts sales. They then began to merchandise disaster related items and Pop Tarts in stores in order to make it easier for their customers to shop. Now who would have hypothesized that Pop Tarts would be a necessary item for preparing for a Tornado or Hurricane? In hind sight, it maybe easy to explain because of the product ease of use for their customers. However, if one removes the urge to try to understand what causes certain correlated events and just accepts the raw data and that these correlations exist, then a new era of marketing has begun using this strategy.

However, there are many ways to view the world, and one must be cautious as they develop ways to assess given circumstances of events. “Big Data” as some may have us to believe is not the cure all, but can be used in significant ways to observe the patterns of our society. This can be another great tool in ones tool box to be able to predict market trends, product sales, and any other activities if the right data is obtainable and accurate.(Read More – Big Data News Roundup: Correlation vs. Causation)

Built in Wordpress — Fueled by The Contrado Group