Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7033

Map outputs implicitly rely on permissive umask for shuffle

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1.0, 3.0.1
    • Component/s: mrv2
    • Labels:
      None

      Description

      Map tasks do not explicitly set the permissions of their output files for shuffle. In a secure cluster the shuffle service is running as a different user than the map task, so the output files require group readability in order to serve up the data during the shuffle phase. If the user's UNIX umask is too restrictive (e.g.: 077) then the map task's file.out and file.out.index permissions can be too restrictive to allow the shuffle handler to access them.

        Attachments

        1. MAPREDUCE-7033.002.patch
          6 kB
          Jason Lowe
        2. MAPREDUCE-7033.001.patch
          2 kB
          Jason Lowe

          Issue Links

            Activity

              People

              • Assignee:
                jlowe Jason Lowe
                Reporter:
                jlowe Jason Lowe
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: