Hadoop Common
  1. Hadoop Common
  2. HADOOP-8561

Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.3-alpha, 0.23.6, 1.1.2
    • Component/s: security
    • Labels:
      None
    • Target Version/s:

      Description

      To solve the problem for an authenticated user to type hadoop shell commands in a web console, we can introduce an HADOOP_PROXY_USER environment variable to allow proper impersonation in the child hadoop client processes.

      1. hadoop-8561.patch
        4 kB
        Yu Gao
      2. hadoop-8561-branch-1.patch
        4 kB
        Yu Gao
      3. hadoop-8561-branch-2.patch
        4 kB
        Yu Gao
      4. hadoop-8561-v2.patch
        4 kB
        Luke Lu

        Issue Links

          Activity

          Hide
          Robert Joseph Evans added a comment -

          This would be really good for testing as well. We have seen issues with HFTP tokens being broken only for proxy users, but were not testing it properly. This should make that testing a lot simpler in the future. +1 for the idea.

          Show
          Robert Joseph Evans added a comment - This would be really good for testing as well. We have seen issues with HFTP tokens being broken only for proxy users, but were not testing it properly. This should make that testing a lot simpler in the future. +1 for the idea.
          Show
          Todd Lipcon added a comment - We achieved this in Hue with a simple wrapper around FsShell: http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/com.cloudera.hue/sudo-shell/1.2.0-cdh3u0/com/cloudera/hue/SudoFsShell.java?av=f
          Hide
          Owen O'Malley added a comment -

          I'm not against making an environment variable/property to set the user, but we might as well use the one we already have and enable HADOOP_USER_NAME in secure mode to mean act as a proxy for the given user.

          Show
          Owen O'Malley added a comment - I'm not against making an environment variable/property to set the user, but we might as well use the one we already have and enable HADOOP_USER_NAME in secure mode to mean act as a proxy for the given user.
          Hide
          Luke Lu added a comment -

          We'd also like to use proxy user in "semi" secure mode as well.

          Show
          Luke Lu added a comment - We'd also like to use proxy user in "semi" secure mode as well.
          Hide
          Luke Lu added a comment -

          @Owen, I'm fine with repurposing HADOOP_USER_NAME and for proxy user (better auditing and access control even without kerbero), though it's an incompatible change. One of the reasons we added HADOOP_PROXY_USER is to preserve the original semantics for HADOOP_USER_NAME.

          Show
          Luke Lu added a comment - @Owen, I'm fine with repurposing HADOOP_USER_NAME and for proxy user (better auditing and access control even without kerbero), though it's an incompatible change. One of the reasons we added HADOOP_PROXY_USER is to preserve the original semantics for HADOOP_USER_NAME.
          Hide
          Daryn Sharp added a comment -

          I kind of like Todd's approach. Maybe we should consider adding a sudo command to FsShell so it's not a separate utility. Using an env makes me a bit squeamish since it may introduce an unexpected attack vector.

          Show
          Daryn Sharp added a comment - I kind of like Todd's approach. Maybe we should consider adding a sudo command to FsShell so it's not a separate utility. Using an env makes me a bit squeamish since it may introduce an unexpected attack vector.
          Hide
          Matt Foley added a comment -

          Moved to 1.2.0 upon release of 1.1.0.

          Show
          Matt Foley added a comment - Moved to 1.2.0 upon release of 1.1.0.
          Hide
          Luke Lu added a comment -

          This approach has added benefit of working with clients (like HBase shell) not written in Java.

          Using an env makes me a bit squeamish since it may introduce an unexpected attack vector.

          It won't do anything for ordinary users. An admin web app of course needs to do a few things sanitize the input to disallow fork/exec etc.

          Show
          Luke Lu added a comment - This approach has added benefit of working with clients (like HBase shell) not written in Java. Using an env makes me a bit squeamish since it may introduce an unexpected attack vector. It won't do anything for ordinary users. An admin web app of course needs to do a few things sanitize the input to disallow fork/exec etc.
          Hide
          Luke Lu added a comment -

          The patches lgtm. +1 pending jenkins.

          Show
          Luke Lu added a comment - The patches lgtm. +1 pending jenkins.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12551912/hadoop-8561.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1701//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1701//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12551912/hadoop-8561.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1701//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1701//console This message is automatically generated.
          Hide
          Luke Lu added a comment -

          Need to merge with HADOOP-9035

          Show
          Luke Lu added a comment - Need to merge with HADOOP-9035
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12561097/hadoop-8561-v2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1878//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1878//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12561097/hadoop-8561-v2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1878//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1878//console This message is automatically generated.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk-Commit #3128 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3128/)
          HADOOP-8561. Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes. (Yu Gao via llu) (Revision 1422429)

          Result = SUCCESS
          llu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1422429
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestProxyUserFromEnv.java
          Show
          Hudson added a comment - Integrated in Hadoop-trunk-Commit #3128 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3128/ ) HADOOP-8561 . Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes. (Yu Gao via llu) (Revision 1422429) Result = SUCCESS llu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1422429 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestProxyUserFromEnv.java
          Hide
          Luke Lu added a comment -

          Committed to trunk and branch-

          {2,1,1.1}

          . Thanks Yu!

          Show
          Luke Lu added a comment - Committed to trunk and branch- {2,1,1.1} . Thanks Yu!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Yarn-trunk #67 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/67/)
          HADOOP-8561. Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes. (Yu Gao via llu) (Revision 1422429)

          Result = SUCCESS
          llu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1422429
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestProxyUserFromEnv.java
          Show
          Hudson added a comment - Integrated in Hadoop-Yarn-trunk #67 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/67/ ) HADOOP-8561 . Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes. (Yu Gao via llu) (Revision 1422429) Result = SUCCESS llu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1422429 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestProxyUserFromEnv.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1256 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1256/)
          HADOOP-8561. Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes. (Yu Gao via llu) (Revision 1422429)

          Result = FAILURE
          llu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1422429
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestProxyUserFromEnv.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1256 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1256/ ) HADOOP-8561 . Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes. (Yu Gao via llu) (Revision 1422429) Result = FAILURE llu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1422429 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestProxyUserFromEnv.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1287 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1287/)
          HADOOP-8561. Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes. (Yu Gao via llu) (Revision 1422429)

          Result = SUCCESS
          llu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1422429
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestProxyUserFromEnv.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1287 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1287/ ) HADOOP-8561 . Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes. (Yu Gao via llu) (Revision 1422429) Result = SUCCESS llu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1422429 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestProxyUserFromEnv.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #470 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/470/)
          HADOOP-8561. Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes (Yu Gao via tgraves) (Revision 1424698)

          Result = UNSTABLE
          tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1424698
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestProxyUserFromEnv.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #470 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/470/ ) HADOOP-8561 . Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes (Yu Gao via tgraves) (Revision 1424698) Result = UNSTABLE tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1424698 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestProxyUserFromEnv.java

            People

            • Assignee:
              Yu Gao
              Reporter:
              Luke Lu
            • Votes:
              0 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development