Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2327

MapTask doesn't need to put username information in SpillRecord

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This is an amendment to MAPREDUCE-2096 that's found in Yahoo's 0.20.100 branch.

      This bug causes task failures in the following case:

      • Cluster is not set up with LinuxTaskController (ie not secured cluster)
      • Job submitter is not the same as the user running the TT
      • Map output is more than one spill's worth

      The issue is that UserGroupInformation's view of the current user is the job submitter, but on disk the spill files will be owned by the TT user. SecureIO will then fail when constructing the spill record.

      1. mr-2327.txt
        0.7 kB
        Todd Lipcon
      2. mapreduce-2327.txt
        5 kB
        Todd Lipcon
      3. mapreduce-2327.txt
        5 kB
        Todd Lipcon
      4. mapreduce-2327.txt
        5 kB
        Todd Lipcon

        Issue Links

          Activity

          Todd Lipcon created issue -
          Todd Lipcon made changes -
          Field Original Value New Value
          Attachment mr-2327.txt [ 12471252 ]
          Todd Lipcon made changes -
          Assignee Todd Lipcon [ tlipcon ]
          Fix Version/s 0.22.0 [ 12314184 ]
          Priority Major [ 3 ] Blocker [ 1 ]
          Description This is an amendment to MAPREDUCE-2096 that's found in Yahoo's 0.20.100 branch. This is an amendment to MAPREDUCE-2096 that's found in Yahoo's 0.20.100 branch.

          This bug causes task failures in the following case:
          - Cluster is not set up with LinuxTaskController (ie not secured cluster)
          - Job submitter is not the same as the user running the TT
          - Map output is more than one spill's worth

          The issue is that UserGroupInformation's view of the current user is the job submitter, but on disk the spill files will be owned by the TT user. SecureIO will then fail when constructing the spill record.
          Todd Lipcon made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Todd Lipcon made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Todd Lipcon made changes -
          Link This issue is blocked by MAPREDUCE-2445 [ MAPREDUCE-2445 ]
          Todd Lipcon made changes -
          Attachment mapreduce-2327.txt [ 12477073 ]
          Todd Lipcon made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Todd Lipcon made changes -
          Attachment mapreduce-2327.txt [ 12477614 ]
          Todd Lipcon made changes -
          Attachment mapreduce-2327.txt [ 12479210 ]
          Tom White made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Konstantin Shvachko made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development