Comissioning Node

To add new nodes to the cluster:

1. Add the network addresses of the new nodes to the include file.

hdfs-site.xml
<property>
<name>dfs.hosts</name>
<value>/<hadoop-home>/conf/include</value>
</property>

mapred-site.xml
<property>
<name>mapred.hosts</name>
<value>/<hadoop-home>/conf/include</value>
</property>

Datanodes that are permitted to connect to the namenode are specified in a
file whose name is specified by the dfs.hosts property.

Includes file resides on the NameNodes local filesystem, and it contains a line for each DataNode, specified by network address (as reported by the DataNode; you can see what this is by looking at the NameNodes web UI).

If you need to specify multiple network addresses for a DataNode, put them on one line,  separated by whitespace.
eg :
slave01
slave02
slave03
…..

Similarly, TaskTrackers that may connect to the JobTracker are specified in a file whose name is specified by themapred.hosts property.

In most cases, there is one shared file, referred to as the include file, that both dfs.hosts and  mapred.hosts refer to, since nodes in the cluster run both DataNode and TaskTracker daemons.

2. Update the namenode with the new set of permitted datanodes using this

command:

% hadoop dfsadmin –refreshNodes

3. Update theJobTrackerwith the new set of permitted TaskTrackers using this command:

% hadoop mradmin –refreshNodes

4. Update theslavesfile with the new nodes, so that they are included in future

operations performed by the Hadoop control scripts.

5. Start the new DataNodesand TaskTrackers.

% hadoop dfsadmin -refreshNodes

% hadoop mradmin –refreshNodes

6. Check that the newDataNodesand TaskTrackers appear in the web UI.