Decomissioning Nodes

To remove nodes from the cluster:

1. Add the network addresses of the nodes to be decommissioned to the exclude file. Do not update the include file at this point.

hdfs-site.xml

<property>

<name>dfs.hosts.exclude</name>

<value>/<hadoop-home>/conf/exclude</value>

</property>

mapred-site.xml

<property>

<name>mapred.hosts.exclude </name>

<value>/<hadoop-home>/conf/exclude</value>

</property>

The decommissioning process is controlled by an exclude file, which for HDFS is set by the dfs.hosts.exclude property and for MapReduce by the mapred.hosts.exclude property. It is often the case that these properties refer to the same file.

The exclude file lists the nodes that are not permitted to connect to the cluster.

2. Update the Nameodeswith the new set of permitted DataNodes, using this

command:

% hadoop dfsadmin –refreshNodes

3. Update theJobTrackerwith the new set of permitted TaskTrackers using this command:

% hadoop mradmin –refreshNodes

4. Go to the web UI and check whether the admin state has changed to “Decommission In Progress” for the DataNodesbeing decommissioned. They will start copying their blocks to other DataNodesin the cluster.
5. When all the DataNodesreport their state as “Decommissioned,” all the blocks have been replicated. Shut down the decommissioned nodes.
6. Remove the nodes from the include file, and run:

% hadoop dfsadmin -refreshNodes

% hadoop mradmin –refreshNodes