Sunday, June 22, 2014

Linux distributions to package Hadoop (Seems like it)

There are several commercial and community based Hadoop distributions out there but so far none of the major Linux flavors (Ubuntu, Debian, Fedora) has decided to bundle a Hadoop distribution. Imagine the ease for an end-user who can use simple command line tools to install and run Hadoop. No more wading through bureaucratic delays or lengthy purchase processes to get permission to install and run Hadoop.

Couple of the larger Hadoop companies control the fate of what people currently consider the "standard" Hadoop package. It will be interesting to see how the dynamics will change once the major Linux operating system vendors make Hadoop available as a base RPM. Doing some research I did find the following Fedora project https://fedoraproject.org/wiki/Changes/Hadoop - which leads me to suggest that effort is underway to do just that. If Fedora takes the lead, it is just a matter of time before other distributions do the same. Please note that I have not seen any official comment about this in the press so this could well be a false alarm.

Following is taken from the web page mentioned above and clearly shows the intent of the project. Cannot wait for this to become real for us all.

Detailed Description

Apache Hadoop is a widely used, increasingly complete big data platform, with a strong open source community and growing ecosystem. The goal is to package and integrate the core of the Hadoop ecosystem for Fedora, allowing for immediate use and creating a base for the rest of the ecosystem.

Benefit to Fedora

The Apache Hadoop software will be packaged and integrated with Fedora. The core of the Hadoop ecosystem will be available with Fedora and provide a base for additional packages.

Scope

  • Proposal owners:
  •    Note: target is Apache Hadoop 2.2.0
  •    Package all dependencies needed for Apache Hadoop 2.x
  •    Package the Apache Hadoop 2.x software
  • Other developers: N/A (not a System Wide Change)
  • Release engineering: N/A (not a System Wide Change)
  • Policies and guidelines: N/A (not a System Wide Change)
 Image - Courtesy of rubixdesignandrepair.com