Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
0.20.1, 0.20.2, 0.20.205.0
-
None
-
None
-
None
Description
The symptoms are the following:
a) start-all.sh is able to start both hadoop dfs and map-reduce processes, assuming same grid nodes are used for dfs and map-reduce
b) stop-all.sh stops map-reduce but fails to stop dfs processes (datanode tasks on grid nodes)
Instead, the warning message 'no datanode to stop' is seen for all data nodes.
c) The 'pid' files for datanode processes do not exist therefore the only way to stop datanode processes is to manually execute kill commands.
The root cause of the issue appears to be in hadoop startup scripts. start-all.sh is really two parts:
1. start-dfs.sh : Start namenode and datanodes
2. start-mapred.sh: Jobtracker and task trackers.
In this case, running start-dfs.sh did as expected and created the pid files for different datanodes. However, start-mapred.sh script did end up forcing another rsync from master to slaves, effectively wiping out the pid files stored under "pid" directory.