Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3264

mapreduce.job.user.name needs to be set automatically

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: mrv2
    • Labels:
      None

      Description

      Currently in MR2 I have to manually specify mapreduce.job.user.name for each job. It's not picking it up from the security infrastructure, at least when running with DefaultContainerExecutor. This is obviously incorrect.

      1. MAPREDUCE-3264.patch
        0.9 kB
        Arun C Murthy

        Activity

        Hide
        Arun C Murthy added a comment -

        We need to investigate further, but maybe you can set it to $

        {user}

        in mapred-site.xml or better, mapred-default.xml?

        Show
        Arun C Murthy added a comment - We need to investigate further, but maybe you can set it to $ {user} in mapred-site.xml or better, mapred-default.xml?
        Hide
        Arun C Murthy added a comment -

        One possible, simple fix.

        Show
        Arun C Murthy added a comment - One possible, simple fix.
        Hide
        Arun C Murthy added a comment -

        Vinod - can we do better?

        Show
        Arun C Murthy added a comment - Vinod - can we do better?
        Hide
        Todd Lipcon added a comment -

        Seems a little hacky, but +1 for now unless Vinod has a better solution.

        Show
        Todd Lipcon added a comment - Seems a little hacky, but +1 for now unless Vinod has a better solution.
        Hide
        Arun C Murthy added a comment -

        Quick-fix for now. We should improve this soon.

        Show
        Arun C Murthy added a comment - Quick-fix for now. We should improve this soon.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Let's commit it as is.

        IIRC, the MR AM uses this config name for getting user-name. We can fix it correctly by making RM pass the user-name to AM and making AM use that, but I'll create a separate ticket for doing that way.

        +1 for the patch, for now.

        Show
        Vinod Kumar Vavilapalli added a comment - Let's commit it as is. IIRC, the MR AM uses this config name for getting user-name. We can fix it correctly by making RM pass the user-name to AM and making AM use that, but I'll create a separate ticket for doing that way. +1 for the patch, for now.
        Hide
        Mahadev konar added a comment -

        Just committed this. Thanks Arun.

        Show
        Mahadev konar added a comment - Just committed this. Thanks Arun.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1163 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1163/)
        MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev)

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189616
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1163 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1163/ ) MAPREDUCE-3264 . mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189616 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Commit #70 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/70/)
        MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) - Merging r1189616 from trunk

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189617
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #70 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/70/ ) MAPREDUCE-3264 . mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) - Merging r1189616 from trunk mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189617 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-0.23-Commit #69 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/69/)
        MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) - Merging r1189616 from trunk

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189617
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #69 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/69/ ) MAPREDUCE-3264 . mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) - Merging r1189616 from trunk mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189617 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1241 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1241/)
        MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev)

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189616
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1241 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1241/ ) MAPREDUCE-3264 . mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189616 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1180 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1180/)
        MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev)

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189616
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1180 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1180/ ) MAPREDUCE-3264 . mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189616 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Commit #66 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/66/)
        MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) - Merging r1189616 from trunk

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189617
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #66 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/66/ ) MAPREDUCE-3264 . mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) - Merging r1189616 from trunk mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189617 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Hide
        Vinod Kumar Vavilapalli added a comment -

        We can fix it correctly by making RM pass the user-name to AM and making AM use that, but I'll create a separate ticket for doing that way.

        Created MAPREDUCE-3280 for this.

        Show
        Vinod Kumar Vavilapalli added a comment - We can fix it correctly by making RM pass the user-name to AM and making AM use that, but I'll create a separate ticket for doing that way. Created MAPREDUCE-3280 for this.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #52 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/52/)
        MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) - Merging r1189616 from trunk

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189617
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #52 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/52/ ) MAPREDUCE-3264 . mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) - Merging r1189616 from trunk mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189617 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #873 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/873/)
        MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev)

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189616
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #873 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/873/ ) MAPREDUCE-3264 . mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189616 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #64 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/64/)
        MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) - Merging r1189616 from trunk

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189617
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #64 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/64/ ) MAPREDUCE-3264 . mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) - Merging r1189616 from trunk mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189617 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #845 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/845/)
        MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev)

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189616
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #845 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/845/ ) MAPREDUCE-3264 . mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189616 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml

          People

          • Assignee:
            Arun C Murthy
            Reporter:
            Todd Lipcon
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development