Pushpalanka's Blog

   With this post I am hoping to share the procedure to set up Apache Hadoop in multi node and is a continuation of the post,  Hadoop Single Node Set-up . The given steps are to set up a two node cluster which can be then expanded to more nodes according to the volume of data. The unique capabilities of Hadoop can be well observed when performing on a BIG volume of data in a multi node cluster of commodity hardware.    It will be useful to have a general idea on the  HDFS(Hadoop Distributed File System) architecture which is the default data storage for Hadoop, before proceed to the set up, that we can well understand the steps we are following and what is happening at execution. In brief, it is a master-slave architecture where master act as the NameNode which manages file system namespace and slaves act as the DataNodes which manage the storage of each node. Also there are JobTrackers which are master nodes and TaskTrackers which are ...

With this post I am hoping to share the procedure to set up Apache Hadoop in single node. Hadoop is used in dealing with Big Data sets where deployment is happening on low-cost commodity hardware. It is a map-reduce framework which map segments of a job among the nodes in a cluster for execution. Though we will not see the exact power of Hadoop running it on single node, it is the first step towards the multi-node cluster. Single node set up is useful in getting familiar with the operations and debugging applications for accuracy, but the performance may be far low than what is achievable. I am sharing the steps to be followed on a Linux system, as it is supported as both a development and a  production platform for Hadoop. Win32 is supported only as a development platform and equivalent commands for the given Linux commands need to be followed. This  Hadoop document  includes the details for setting up Hadoop, in brief. I am here sharing a detailed guidance f...