Hadoop Common
  1. Hadoop Common
  2. HADOOP-6832

Provide a web server plugin that uses a static user for the web UI

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We need a simple plugin that uses a static user for clusters with security that don't want to authenticate users on the web UI.

      1. static-web-user.patch
        5 kB
        Owen O'Malley
      2. h-6382.patch
        5 kB
        Owen O'Malley
      3. h-6832.patch
        6 kB
        Owen O'Malley
      4. h-6832.patch
        6 kB
        Owen O'Malley
      5. hadoop-6832.txt
        11 kB
        Todd Lipcon
      6. h-6832.patch
        10 kB
        Owen O'Malley

        Issue Links

          Activity

          Hide
          Todd Lipcon added a comment -

          I think we should make this a default servlet filter, and add a configuration value like hadoop.http.staticusermapping.user that determines the username. We can default this to Dr.Who, or perhaps webuser or nobody to be more standard-looking. We can also support dfs.web.ugi as a deprecated configuration.

          Let me know if you want me to pick this up where you left off, Owen.

          Show
          Todd Lipcon added a comment - I think we should make this a default servlet filter, and add a configuration value like hadoop.http.staticusermapping.user that determines the username. We can default this to Dr.Who, or perhaps webuser or nobody to be more standard-looking. We can also support dfs.web.ugi as a deprecated configuration. Let me know if you want me to pick this up where you left off, Owen.
          Hide
          Owen O'Malley added a comment -

          Here's the trivial update.

          Show
          Owen O'Malley added a comment - Here's the trivial update.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12457792/h-6382.patch
          against trunk revision 1031422.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/31//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/31//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/31//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12457792/h-6382.patch against trunk revision 1031422. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/31//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/31//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/31//console This message is automatically generated.
          Hide
          Devaraj Das added a comment -

          Owen, was wondering whether you want to accomodate Todd's comment here?

          Show
          Devaraj Das added a comment - Owen, was wondering whether you want to accomodate Todd's comment here?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12457792/h-6382.patch
          against trunk revision 1071364.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/270//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/270//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/270//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12457792/h-6382.patch against trunk revision 1071364. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/270//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/270//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/270//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          I will update Owen's patch

          Show
          Todd Lipcon added a comment - I will update Owen's patch
          Hide
          Owen O'Malley added a comment -

          I forgot about this one. I'll go ahead and do it.

          Show
          Owen O'Malley added a comment - I forgot about this one. I'll go ahead and do it.
          Hide
          Owen O'Malley added a comment -

          Actually, looking back, since it is already committed to 0.20.203.0, let's go ahead and commit this to trunk and file a new jira. Does that make sense Todd?

          Show
          Owen O'Malley added a comment - Actually, looking back, since it is already committed to 0.20.203.0, let's go ahead and commit this to trunk and file a new jira. Does that make sense Todd?
          Hide
          Todd Lipcon added a comment -

          Actually nearly complete with the new patch for trunk. Since this was committed to 203 even though it hadn't been +1ed here, I don't see why that should hold this back from being fixed.

          Show
          Todd Lipcon added a comment - Actually nearly complete with the new patch for trunk. Since this was committed to 203 even though it hadn't been +1ed here, I don't see why that should hold this back from being fixed.
          Hide
          Owen O'Malley added a comment -

          Adds the configuration variable and makes it part of the default list.

          Show
          Owen O'Malley added a comment - Adds the configuration variable and makes it part of the default list.
          Hide
          Owen O'Malley added a comment -

          Forgot to use the default user.

          Show
          Owen O'Malley added a comment - Forgot to use the default user.
          Hide
          Todd Lipcon added a comment -

          Patch similar to Owen's but with some improvements:

          • doesn't rely on static variables to configure the static user
          • adds unit tests
          • documents the new configuration
          Show
          Todd Lipcon added a comment - Patch similar to Owen's but with some improvements: doesn't rely on static variables to configure the static user adds unit tests documents the new configuration
          Hide
          Owen O'Malley added a comment -

          thanks for the test case, todd.

          incorporated todd's suggestions, although i'm not wild about doing the username lookup for each request.

          The default user is already in the code and doesn't need to be in the configuration file too.

          Using a non-unix userid means the chance for false positives is much lower. Besides, dr. who is fun.

          Show
          Owen O'Malley added a comment - thanks for the test case, todd. incorporated todd's suggestions, although i'm not wild about doing the username lookup for each request. The default user is already in the code and doesn't need to be in the configuration file too. Using a non-unix userid means the chance for false positives is much lower. Besides, dr. who is fun.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12479664/hadoop-6832.txt
          against trunk revision 1124406.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/473//testReport/
          Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/473//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/473//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12479664/hadoop-6832.txt against trunk revision 1124406. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/473//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/473//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/473//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          incorporated todd's suggestions, although i'm not wild about doing the username lookup for each request.

          What do you mean by "username lookup" here? The conf lookup only happens when the HttpServer starts. The "new User()" construction does happen for every request, but that's hardly heavy lifting (eg we do the same thing on every RPC)

          The default user is already in the code and doesn't need to be in the configuration file too.

          The same could be said for every other default throughout all of common, HDFS, and MR, no? This is our status quo.

          Using a non-unix userid means the chance for false positives is much lower. Besides, dr. who is fun.

          Apparently most of our very-confused customers who've filed tickets about this don't agree. To quote one operator:

          One thing I'd like to add is the fact that DrWho.Tardis are the default
          user.group for Hadoop is horrible and confusing. For my $0.02 on this as
          a sys-admin I want a useful error message that doesn't make me go digging
          though the Hadoop Jira for 15 minutes
          
          Show
          Todd Lipcon added a comment - incorporated todd's suggestions, although i'm not wild about doing the username lookup for each request. What do you mean by "username lookup" here? The conf lookup only happens when the HttpServer starts. The "new User()" construction does happen for every request, but that's hardly heavy lifting (eg we do the same thing on every RPC) The default user is already in the code and doesn't need to be in the configuration file too. The same could be said for every other default throughout all of common, HDFS, and MR, no? This is our status quo. Using a non-unix userid means the chance for false positives is much lower. Besides, dr. who is fun. Apparently most of our very-confused customers who've filed tickets about this don't agree. To quote one operator: One thing I'd like to add is the fact that DrWho.Tardis are the default user.group for Hadoop is horrible and confusing. For my $0.02 on this as a sys-admin I want a useful error message that doesn't make me go digging though the Hadoop Jira for 15 minutes
          Hide
          Owen O'Malley added a comment -

          What do you mean by "username lookup" here?

          +      this.username = conf.getInitParameter(USERNAME_KEY);
          +      this.user = new User(username);
          

          I agree it isn't huge, which is why I left it in, but it also isn't adding any value.

          The same could be said for every other default throughout all of common, HDFS, and MR, no?

          No, actually. We do it in a lot of places, but it leads to lots of confusion. Furthermore, this isn't a framework default, but a plugin default. I don't think it is appropriate to put into the default configuration file.

          Apparently most of our very-confused customers who've filed tickets about this don't agree.

          The issue isn't the default value of "dr.who". Having a magic value of "webuser" will have exactly the same problem with trying to figure out where it is coming from. Furthermore, if they actually have a real webuser account, it could create problems. I guess we could use "default.web.user" or something.

          Show
          Owen O'Malley added a comment - What do you mean by "username lookup" here? + this .username = conf.getInitParameter(USERNAME_KEY); + this .user = new User(username); I agree it isn't huge, which is why I left it in, but it also isn't adding any value. The same could be said for every other default throughout all of common, HDFS, and MR, no? No, actually. We do it in a lot of places, but it leads to lots of confusion. Furthermore, this isn't a framework default, but a plugin default. I don't think it is appropriate to put into the default configuration file. Apparently most of our very-confused customers who've filed tickets about this don't agree. The issue isn't the default value of "dr.who". Having a magic value of "webuser" will have exactly the same problem with trying to figure out where it is coming from. Furthermore, if they actually have a real webuser account, it could create problems. I guess we could use "default.web.user" or something.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12479679/h-6832.patch
          against trunk revision 1124456.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/474//testReport/
          Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/474//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/474//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12479679/h-6832.patch against trunk revision 1124456. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/474//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/474//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/474//console This message is automatically generated.
          Hide
          Chris Douglas added a comment -

          +1 on the merged h-6832.patch.

          I committed this. Thanks Owen and Todd!

          Using default.web.user would be more descriptive, but at least searching for "hadoop dr.who" yields relevant, whinging results.

          Show
          Chris Douglas added a comment - +1 on the merged h-6832.patch . I committed this. Thanks Owen and Todd! Using default.web.user would be more descriptive, but at least searching for "hadoop dr.who" yields relevant, whinging results.
          Hide
          Todd Lipcon added a comment -

          I am still strongly against "dr who". I understand it's some kind of tradition to put Dr Who references throughout our code base. Unfortunately the users aren't in on the joke. I will open another JIRA to remove these references.

          Show
          Todd Lipcon added a comment - I am still strongly against "dr who". I understand it's some kind of tradition to put Dr Who references throughout our code base. Unfortunately the users aren't in on the joke. I will open another JIRA to remove these references.
          Hide
          Chris Douglas added a comment -

          nod Discussing that in a separate thread makes sense.

          Show
          Chris Douglas added a comment - nod Discussing that in a separate thread makes sense.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #610 (See https://builds.apache.org/hudson/job/Hadoop-Common-trunk-Commit/610/)
          HADOOP-6832. Add an authentication plugin using a configurable static user
          for the web UI. Contributed by Owen O'Malley and Todd Lipcon

          cdouglas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1125043
          Files :

          • /hadoop/common/trunk/src/java/org/apache/hadoop/http/lib
          • /hadoop/common/trunk/CHANGES.txt
          • /hadoop/common/trunk/src/java/org/apache/hadoop/http/lib/package.html
          • /hadoop/common/trunk/src/test/core/org/apache/hadoop/http/lib
          • /hadoop/common/trunk/src/test/core/org/apache/hadoop/http/lib/TestStaticUserWebFilter.java
          • /hadoop/common/trunk/src/java/core-default.xml
          • /hadoop/common/trunk/src/java/org/apache/hadoop/http/lib/StaticUserWebFilter.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #610 (See https://builds.apache.org/hudson/job/Hadoop-Common-trunk-Commit/610/ ) HADOOP-6832 . Add an authentication plugin using a configurable static user for the web UI. Contributed by Owen O'Malley and Todd Lipcon cdouglas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1125043 Files : /hadoop/common/trunk/src/java/org/apache/hadoop/http/lib /hadoop/common/trunk/CHANGES.txt /hadoop/common/trunk/src/java/org/apache/hadoop/http/lib/package.html /hadoop/common/trunk/src/test/core/org/apache/hadoop/http/lib /hadoop/common/trunk/src/test/core/org/apache/hadoop/http/lib/TestStaticUserWebFilter.java /hadoop/common/trunk/src/java/core-default.xml /hadoop/common/trunk/src/java/org/apache/hadoop/http/lib/StaticUserWebFilter.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #694 (See https://builds.apache.org/hudson/job/Hadoop-Common-trunk/694/)
          HADOOP-6832. Add an authentication plugin using a configurable static user
          for the web UI. Contributed by Owen O'Malley and Todd Lipcon

          cdouglas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1125043
          Files :

          • /hadoop/common/trunk/src/java/org/apache/hadoop/http/lib
          • /hadoop/common/trunk/CHANGES.txt
          • /hadoop/common/trunk/src/java/org/apache/hadoop/http/lib/package.html
          • /hadoop/common/trunk/src/test/core/org/apache/hadoop/http/lib
          • /hadoop/common/trunk/src/test/core/org/apache/hadoop/http/lib/TestStaticUserWebFilter.java
          • /hadoop/common/trunk/src/java/core-default.xml
          • /hadoop/common/trunk/src/java/org/apache/hadoop/http/lib/StaticUserWebFilter.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #694 (See https://builds.apache.org/hudson/job/Hadoop-Common-trunk/694/ ) HADOOP-6832 . Add an authentication plugin using a configurable static user for the web UI. Contributed by Owen O'Malley and Todd Lipcon cdouglas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1125043 Files : /hadoop/common/trunk/src/java/org/apache/hadoop/http/lib /hadoop/common/trunk/CHANGES.txt /hadoop/common/trunk/src/java/org/apache/hadoop/http/lib/package.html /hadoop/common/trunk/src/test/core/org/apache/hadoop/http/lib /hadoop/common/trunk/src/test/core/org/apache/hadoop/http/lib/TestStaticUserWebFilter.java /hadoop/common/trunk/src/java/core-default.xml /hadoop/common/trunk/src/java/org/apache/hadoop/http/lib/StaticUserWebFilter.java

            People

            • Assignee:
              Owen O'Malley
              Reporter:
              Owen O'Malley
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development