Showing posts from July, 2013

JVM Garbage Collection and Optimizations

Overview When troubleshooting systems for performance-related issues, memory optimizations are a place that needs a deep analysis of what each system stores in the memory, how long those are stored, and access patterns. This post is to keep a note on the background information and valuable points to note in such an effort, specific to Java-based implementations as a deep understanding of the JVM behaviors is very beneficial in the process.

Java language provides much convenience to the developers by taking care of the memory management to a great extent letting the focus be on the rest of the logic. Still having a good understanding of how Java does this underneath, rationalize several best practices we follow in Java implementations and help design the programs better and think seriously on some aspects that can later lead to memory leaks and system stability in the long run. Java Garbage Collector has a big role in this been responsible for freeing up memory by removing memory gar…

Hadoop Multi Node Set Up

With this post I am hoping to share the procedure to set up Apache Hadoop in multi node and is a continuation of the post, Hadoop Single Node Set-up. The given steps are to set up a two node cluster which can be then expanded to more nodes according to the volume of data. The unique capabilities of Hadoop can be well observed when performing on a BIG volume of data in a multi node cluster of commodity hardware.    It will be useful to have a general idea on the HDFS(Hadoop Distributed File System) architecture which is the default data storage for Hadoop, before proceed to the set up, that we can well understand the steps we are following and what is happening at execution. In brief, it is a master-slave architecture where master act as the NameNode which manages file system namespace and slaves act as the DataNodes which manage the storage of each node. Also there are JobTrackers which are master nodes and TaskTrackers which are slave nodes.    This Hadoop document inc…