3 D’s of Big Data – effective Big Data strategy implementation – What a TV show, Sports Analytics and a best-selling book taught me
The world of everything is now abuzz with Big Data. By now everybody is aware of the 3 V’s of Big Data – Velocity (speed of arrival of data), Variety (types of data) and Volume (amount of data). Every organisation has either harvested the benefits of Big Data or in the process of implementing strategies to do so.
Over the summer, I spent a lot of my spare time reading books and watching TV. Three independent events got me thinking on what it takes to implement an effective Big Data & Analytics strategy. They were – reading the book Freakonomics, watching the TV show Storage Hunters and comparing the analytics provided during Wimbledon tennis against those provided in an American Baseball game (MLB).
In the book Freakonomics, the authors use largely publicly available information and combine that with their research to derive insights on how crime perpetrated in Chicago suburbs. On the TV show Storage Hunters, individuals bid on items in abandoned storage containers based on barely visible corners of items using a wide array of theories learnt from other similar containers. The deep analytics provided during Wimbledon help prepare ahead of the match whereas the deeper analytics provided during MLB help adjust during the game
So, here goes my 3 D’s (or more!) of an effective Big Data implementation based on my learning from the above:
Develop the End Game – a good Big Data implementation is based on a clear set of outcomes from the initial pilots. It is important to have a set of tangible insights the organisation wants to derive. Once this is implemented, it is critical to validate the efficacy and usefulness of the insights derived and constantly tune the models to derive better insights. The MLB coaches do this effectively as the game progresses to advise the pitcher & batter of what to do at any particular situation in the game.
Discover Data Sources – identify and assemble all possible data sources (internal to the organisation and externally available) that can contribute to the Big Data implementation. Avoid the deluge of data by aggressively deduping these data sources and discard all unwanted data to avoid skewing the insights. The Storage Hunters come to the auctions with a lot of data in their heads, most of which they might not have expected to be of use, but comes in handy to identify patterns.
Distribute Insights – once the insights have been derived and put to use in the organisation, explore the possibility of monetising the value of these insights by distributing it to other organisations. What might seem as “end insight” for one organisation might just be the “start insight” for another organisation. The book Freakonomics is based on utilising “end” insights provided by organisations and combining with other data they researched to arrive at insights in completely unrelated areas
In summary, to effectively tackle the 3 V’s of Big Data and derive maximum value, it is imperative to follow the 3 D’s
Author: Ramanan Ramakrishna, Director & GM – Emerging Business Group