Monday, December 22, 2014

Big data deployment interdependencies

Over the last few years since MapReduce/Hadoop spawned a whole new era of big data, we have seen many vendors, community driven tools, products, libraries explode on the market. To even the most ardent followers of the space and especially to a casual observer or a new-entrant to the big data field making sense of all the tools and technologies is a nightmare-- bad enough for many to consider their choice of profession. But fear no more. Below are two things that will help you make sense of the big data jungle out there. First is a table of the technologies and their purpose/function. And the second is the interdependency chart of the various technologies.

F UNCTIONAL CLASSIFICA TION OF EXISTING HIGHER - LEVEL B IG D ATA APPLICATIONS AND LIB RARIES
Table - Classification of Big Data Technologies

Big data deployment interdependencies
Big data deployment interdependencies


If you are a big data novice start with the table, figure out the technology options that are in-play for your situation. Once you have narrowed down the options, research some more. Hint- Infoivy.com is a good starting place.

Deciding the options to use is never easy but once you do have made the decision,  use the dependency chart to figure the deployment stack that you would need for the options chosen. Word of caution - not all stacks can be deployed - at least not yet. So, before you jump into the big data pit check with people who might know more than you but make your own decisions. Remember all decisions come up with a price -- the most expensive one being time especially if you are on a tight timeline. Choose wisely!

Credit:
Deciphering Big Data Stacks: An Overview of Big Data Tools by Tomislav Lipic, Karolj Skala, Enis Afgan