Details
-
Bug
-
Status: Patch Available
-
Major
-
Resolution: Unresolved
-
2.5.0, 2.4.1
-
None
Description
Right now, Hadoop puts all users' staging directories under /tmp/hadoop-yarn/staging/, for example /tmp/hadoop-yarn/staging/hadoop for user hadoop, but the directory /tmp/hadoop-yarn/staging is also used for other purpose. For example, /tmp/hadoop-yarn/staging/history/ is used to hold finished JHS files. The shared parent /tmp/hadoop-yarn/staging makes it difficult to track all users' folders without adding extra logic to exclude other known folders.
As a result, we should move all users' folders to a user sub-folder, i.e., /tmp/hadoop-yarn/staging/user/. In this case, user hadoop's staging folder becomes /tmp/hadoop-yarn/staging/user/hadoop/.staging.