[MAPREDUCE-7227] Fix job staging directory residual problem in a big yarn cluster composed of multiple independent hdfs clusters

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.6.0, 2.7.0, 3.1.2
• Component/s: applicationmaster, mrv2
• Target Version/s: 3.5.0
• Assignee: Yuan Luo
• Reporter: Yuan Luo
• Created: 31/Jul/19 09:47
• Updated: 04/Jan/24 09:33

Description

Our yarn cluster is made up of some independent hdfs cluster, the 'default.FS' in every hdfs cluster is different.when user submit job to yarn cluster, if the 'default.FS'  between client and nodemanager  is inconsistent, then the job staging dir can't be cleanup by AppMaster. Because it will produce two job staging dirs in our conditions by client and appmaster. So we can modify AppMaster through client's ‘default.FS’ to create job staging dir.