[MAPREDUCE-6111] Hadoop users' staging directories should be under a user folder

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.5.0, 2.4.1
• Component/s: mrv2
• Labels: BB2015-05-TBR
• Assignee: Unassigned
• Reporter: Jian Fang
• Created: 26/Sep/14 18:27
• Updated: 06/May/15 03:35

Description

Right now, Hadoop puts all users' staging directories under /tmp/hadoop-yarn/staging/, for example /tmp/hadoop-yarn/staging/hadoop for user hadoop, but the directory /tmp/hadoop-yarn/staging is also used for other purpose. For example, /tmp/hadoop-yarn/staging/history/ is used to hold finished JHS files. The shared parent /tmp/hadoop-yarn/staging makes it difficult to track all users' folders without adding extra logic to exclude other known folders.

As a result, we should move all users' folders to a user sub-folder, i.e., /tmp/hadoop-yarn/staging/user/. In this case, user hadoop's staging folder becomes /tmp/hadoop-yarn/staging/user/hadoop/.staging.