Why are companies becoming disillusioned with Big Data?
It is known that if you want to get an IT budget approved, there are certain phrases you need to throw into the proposal which hit such a nerve that the Chief Financial Officer will immediately approve the budget. One such phrase is “Big Data”.
However, this phrase usage is wearing thin. It has been on top of the Gartner Hype Cycle since 2013 and is now falling into the trough of disillusionment as Big Data was touted as the ability to solve all of the company’s ills. To date, it has not.
In 2014, according to the IDC, $125 billion was spent by companies on hardware, services and software to deal with Big Data however at CeBIT 2015, SurveyMonkey’s CEO Dave Glodberg, highlighted that “[big data] has not delivered on massive promises” which were made by IT departments, and so the projected are starting to fall out of favour.
Why is Big Data failing ?
It would easy to dismiss Big Data as failing due to too much data which is being collected, in too many silos and these silos are still not connected to each other. However, there is another side to Big Data which Dave Goldberg believes is missing from the discussion and so Dave talks about “Implicit Data” and “Explicit Data” and both are needed.
Implicit Data vs. Explicit Data
Implicit Data is what we currently think of when we talk about Big Data. It is data that is collected such as clicking on a mouse, listening to song, entering and analysing search queries. We get this type of data on a massive scale, we measure everything and it is this type of data that concerns people about their security and data privacy.
However Implicit data gets it wrong.
The credit card companies analyses everyone’s purchasing habits and suddenly discovers expenses such as teeth-whitening, gym membership and hotel rooms. Therefore an assumption is made based on this Implicit data that this person is getting divorced. The data doesn’t not take into account surrounding circumstances such as upcoming business travel.
Shopping engines recommend products based on historical implicit data purchases. However, if shopping for a gift for a friend, this would throw off the personal recommendation engine.
“More data doesn’t lead to better information and this is how Implicit Data gets it wrong” David states and goes on to that “If you want to know how someone is feeling, what music they like – you need to ask them. Analysing their searches doesn’t lead you to the correct answer as much as asking.”
Asking is known as Explicit data.
Explicit data is data we get by asking questions and receiving answers.
The internet has allowed companies to get this type of data en-mass by asking the right questions to the right group of people.
How is Explicit Data used ?
Companies like Google realised that they were losing female staff which concerned the company. Instead of relying on Implicit data of the female search history, they surveyed the staff and realised they don’t have a female/ male issue they have a “new mother” issue. Female who were pregnant or thinking about becoming a mom did not like Google’s maternity policy. As soon as Google discovered this, they instantly fixed the issue and retain their female staff.
SurveyMonkey use Implicit and Explicit data in order to make decisions. Mining the data and asking customers directly allows the company to thrive. And they are not the only ones. SurveyMonkey in one day has 3 million survey responses answering 29 million questions which results in 80GB of new Data
The key message behind Dave’s talk is relying on Causation without Correlation doesn’t paint a true picture. Have the conversation with your customer and listen to what they have to say. “Encourage customers and employees to give you feedback. People feel powerful and loyal when their voices are heard”
