[HADOOP-2899] [HOD] hdfs:///mapredsystem directory not cleaned up after deallocation

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.16.0
• Fix Version/s: 0.17.0
• Component/s: contrib/hod
• Hadoop Flags: Reviewed
• Release Note: The mapred system directory generated by HOD is cleaned up at cluster deallocation time.
• Assignee: Hemanth Yamijala
• Reporter: Luca Telloli
• Created: 26/Feb/08 17:39
• Updated: 21/May/08 20:05
• Resolved: 20/Mar/08 09:18

Description

Each submitted job creates a hdfs:///mapredsystem directory, created by (I guess) the hodring process. Problem is that it's not cleaned up at the end of the process; a use case would be:

• user A allocates a cluster, the hodring is svrX, so a /mapredsystem/srvX directory is created

• user A deallocates the cluster, but that directory is not cleaned up

• user B allocates a cluster, and the first node chosen as hodring is svrX, so hodring tries to write hdfs:///mapredsystem but it fails

• allocation succeeds, but there's no hodring running; looking at
  0-jobtracker/logdir/hadoop.log under the temporary directory I can read:

2008-02-26 17:28:42,567 WARN org.apache.hadoop.mapred.JobTracker: Error starting tracker: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=B, access=WRITE, inode="mapredsystem":hadoop:supergroup:rwxr-xr-x

I guess a possible solution would be to clean up those directories during the deallocation process.