Hadoop Common
  1. Hadoop Common
  2. HADOOP-7987

Support setting the run-as user in unsecure mode

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0, 0.23.0, 0.24.0
    • Fix Version/s: 1.0.1, 0.23.1, 0.24.0, 2.0.0-alpha
    • Component/s: security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Some applications need to be able to perform actions (such as launch MR jobs) from map or reduce tasks. In earlier unsecure versions of hadoop (20.x), it was possible to do this by setting user.name in the configuration. But in 20.205 and 1.0, when running in unsecure mode, this does not work. (In secure mode, you can do this using the kerberos credentials).

      1. HADOOP-7987.trunk.patch
        3 kB
        Jitendra Nath Pandey
      2. HADOOP-7987.branch-1.patch
        3 kB
        Jitendra Nath Pandey

        Activity

        Hide
        Harsh J added a comment -

        If secure impersonation covered this need and retained a simple, still-controlled notion of security that was present before this change even for insecure mode clusters, why did this need to get added?

        Show
        Harsh J added a comment - If secure impersonation covered this need and retained a simple, still-controlled notion of security that was present before this change even for insecure mode clusters, why did this need to get added?
        Hide
        Matt Foley added a comment -

        1.0.1 was a lineal predecessor of 1.1.0.

        Show
        Matt Foley added a comment - 1.0.1 was a lineal predecessor of 1.1.0.
        Hide
        Matt Foley added a comment -

        Closed upon release 1.0.1.

        Show
        Matt Foley added a comment - Closed upon release 1.0.1.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #970 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/970/)
        HADOOP-7987. Support setting the run-as user in unsecure mode.

        jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235945
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #970 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/970/ ) HADOOP-7987 . Support setting the run-as user in unsecure mode. jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235945 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #172 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/172/)
        Merged r1235945 from trunk for HADOOP-7987.

        jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235948
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #172 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/172/ ) Merged r1235945 from trunk for HADOOP-7987 . jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235948 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #150 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/150/)
        Merged r1235945 from trunk for HADOOP-7987.

        jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235948
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #150 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/150/ ) Merged r1235945 from trunk for HADOOP-7987 . jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235948 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #937 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/937/)
        HADOOP-7987. Support setting the run-as user in unsecure mode.

        jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235945
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #937 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/937/ ) HADOOP-7987 . Support setting the run-as user in unsecure mode. jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235945 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Commit #434 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/434/)
        Merged r1235945 from trunk for HADOOP-7987.

        jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235948
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #434 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/434/ ) Merged r1235945 from trunk for HADOOP-7987 . jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235948 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1601 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1601/)
        HADOOP-7987. Support setting the run-as user in unsecure mode.

        jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235945
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1601 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1601/ ) HADOOP-7987 . Support setting the run-as user in unsecure mode. jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235945 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Commit #409 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/409/)
        Merged r1235945 from trunk for HADOOP-7987.

        jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235948
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #409 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/409/ ) Merged r1235945 from trunk for HADOOP-7987 . jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235948 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1657 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1657/)
        HADOOP-7987. Support setting the run-as user in unsecure mode.

        jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235945
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1657 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1657/ ) HADOOP-7987 . Support setting the run-as user in unsecure mode. jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235945 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-0.23-Commit #418 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/418/)
        Merged r1235945 from trunk for HADOOP-7987.

        jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235948
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #418 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/418/ ) Merged r1235945 from trunk for HADOOP-7987 . jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235948 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1584 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1584/)
        HADOOP-7987. Support setting the run-as user in unsecure mode.

        jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235945
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1584 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1584/ ) HADOOP-7987 . Support setting the run-as user in unsecure mode. jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1235945 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserFromEnv.java
        Hide
        Jitendra Nath Pandey added a comment -

        The javadoc warnings have existed in trunk for a while now.

        Show
        Jitendra Nath Pandey added a comment - The javadoc warnings have existed in trunk for a while now.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12511871/HADOOP-7987.trunk.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        -1 javadoc. The javadoc tool appears to have generated 7 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/530//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/530//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511871/HADOOP-7987.trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 7 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/530//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/530//console This message is automatically generated.
        Hide
        Jitendra Nath Pandey added a comment -

        Same patch applies to trunk without conflicts, just file paths changes.

        Show
        Jitendra Nath Pandey added a comment - Same patch applies to trunk without conflicts, just file paths changes.
        Hide
        Devaraj Das added a comment -

        +1

        Show
        Devaraj Das added a comment - +1
        Hide
        Jitendra Nath Pandey added a comment -

        A patch is uploaded.

        Show
        Jitendra Nath Pandey added a comment - A patch is uploaded.
        Hide
        Owen O'Malley added a comment -

        Actually, this is a pretty different solution in practice, although it allows a lot of the same abilities. This is just a change to UGI that lets you override the current user lookup in the HadoopLoginModule. Obviously, the change only works in non-secure mode and probably should be done with an environment variable such as HADOOP_USER.

        Show
        Owen O'Malley added a comment - Actually, this is a pretty different solution in practice, although it allows a lot of the same abilities. This is just a change to UGI that lets you override the current user lookup in the HadoopLoginModule. Obviously, the change only works in non-secure mode and probably should be done with an environment variable such as HADOOP_USER.
        Hide
        Todd Lipcon added a comment -

        I had started a thread on this same topic in September '10, where Owen vetoed the idea. See "hadoop.job.ugi backwards compatibility":
        http://markmail.org/thread/irzcppjxy35hwybj
        and http://markmail.org/message/qegxf3yeq74an5hq

        Would be interesting to see if his opinion has changed.

        Show
        Todd Lipcon added a comment - I had started a thread on this same topic in September '10, where Owen vetoed the idea. See "hadoop.job.ugi backwards compatibility": http://markmail.org/thread/irzcppjxy35hwybj and http://markmail.org/message/qegxf3yeq74an5hq Would be interesting to see if his opinion has changed.

          People

          • Assignee:
            Jitendra Nath Pandey
            Reporter:
            Devaraj Das
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development