Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-2925

[HOD] Create mapred system directory using a naming convention that will avoid clashes in multi-user shared cluster scenario.

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.16.0
    • Fix Version/s: 0.16.1
    • Component/s: contrib/hod
    • Labels:
      None

      Description

      Currently, HOD generates the name of the mapredsystem directory using the name /mapredsystem/hostname-of-jobtracker.

      In HADOOP-2899, we ran into a scenario where this naming convention could lead to problems in case dfs permissions are enabled. While the bug should ideally be addressed in Hadoop M/R, it will be better that HOD does not generate names that can potentially clash across runs. One way to solve the problem is to do what HOD already does for local log and temp directories - name it using username.torque-job-id, which is going to be pretty unique mostly.

        Attachments

        1. 2925.1.patch
          8 kB
          Hemanth Yamijala
        2. 2925.2.patch
          12 kB
          Hemanth Yamijala
        3. 2925.patch
          4 kB
          Hemanth Yamijala

          Activity

            People

            • Assignee:
              yhemanth Hemanth Yamijala
              Reporter:
              yhemanth Hemanth Yamijala

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment