Hadoop Common
  1. Hadoop Common
  2. HADOOP-7119

add Kerberos HTTP SPNEGO authentication support to Hadoop JT/NN/DN/TT web-consoles

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.20.205.0, 0.22.1, 0.23.0
    • Component/s: security
    • Labels:
      None
    • Environment:

      all

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Adding support for Kerberos HTTP SPNEGO authentication to the Hadoop web-consoles

      Description

      Currently the JT/NN/DN/TT web-consoles don't support any form of authentication.

      Hadoop RPC API already supports Kerberos authentication.

      Kerberos enables single sign-on.

      Popular browsers (Firefox and Internet Explorer) have support for Kerberos HTTP SPNEGO.

      Adding support for Kerberos HTTP SPNEGO to Hadoop web consoles would provide a unified authentication mechanism and single sign-on for Hadoop web UI and Hadoop RPC.

      1. hadoop-7119-022.patch
        168 kB
        Benoy Antony
      2. hadoop-7119-022.patch
        167 kB
        Benoy Antony
      3. spnego-20-security4.patch
        167 kB
        Sanjay Radia
      4. spnego-20-security3.patch
        166 kB
        Sanjay Radia
      5. spnego-20-security2.patch
        161 kB
        Sanjay Radia
      6. spnego-20-security.patch
        158 kB
        Sanjay Radia
      7. HADOOP-7119v6.patch
        245 kB
        Alejandro Abdelnur
      8. HADOOP-7119v5.patch
        246 kB
        Alejandro Abdelnur
      9. HADOOP-7119v4-amendment.patch
        59 kB
        Aaron T. Myers
      10. HADOOP-7119v4.patch
        203 kB
        Alejandro Abdelnur
      11. HADOOP-7119v3.patch
        203 kB
        Alejandro Abdelnur
      12. ha-common-02.patch
        18 kB
        Alejandro Abdelnur
      13. ha-common-01.patch
        18 kB
        Alejandro Abdelnur
      14. ha-commons.patch
        15 kB
        Alejandro Abdelnur

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-22-branch #106 (See https://builds.apache.org/job/Hadoop-Common-22-branch/106/)
          HADOOP-7119. Add Kerberos HTTP SPNEGO authentication support to Hadoop JT/NN/DN/TT web-consoles. Contributed by Alejandro Abdelnur and Benoy Antony. (Revision 1346222)

          Result = SUCCESS
          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346222
          Files :

          • /hadoop/common/branches/branch-0.22/common/CHANGES.txt
          • /hadoop/common/branches/branch-0.22/common/src/docs/src/documentation/content/xdocs/HttpAuthentication.xml
          • /hadoop/common/branches/branch-0.22/common/src/docs/src/documentation/content/xdocs/site.xml
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/AuthenticationFilterInitializer.java
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/README.txt
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/client
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/client/AuthenticatedURL.java
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/client/AuthenticationException.java
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/client/Authenticator.java
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/client/KerberosAuthenticator.java
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/client/PseudoAuthenticator.java
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/server
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/server/AuthenticationFilter.java
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/server/AuthenticationToken.java
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/server/KerberosAuthenticationHandler.java
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/server/PseudoAuthenticationHandler.java
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/util
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/util/KerberosName.java
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/util/Signer.java
          • /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/util/SignerException.java
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/KerberosTestUtils.java
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/client
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/client/TestAuthenticatedURL.java
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/client/TestKerberosAuthenticator.java
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/client/TestPseudoAuthenticator.java
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/server
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/server/TestAuthenticationFilter.java
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/server/TestAuthenticationToken.java
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/server/TestKerberosAuthenticationHandler.java
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/server/TestPseudoAuthenticationHandler.java
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/util
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/util/TestKerberosName.java
          • /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/util/TestSigner.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-22-branch #106 (See https://builds.apache.org/job/Hadoop-Common-22-branch/106/ ) HADOOP-7119 . Add Kerberos HTTP SPNEGO authentication support to Hadoop JT/NN/DN/TT web-consoles. Contributed by Alejandro Abdelnur and Benoy Antony. (Revision 1346222) Result = SUCCESS shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346222 Files : /hadoop/common/branches/branch-0.22/common/CHANGES.txt /hadoop/common/branches/branch-0.22/common/src/docs/src/documentation/content/xdocs/HttpAuthentication.xml /hadoop/common/branches/branch-0.22/common/src/docs/src/documentation/content/xdocs/site.xml /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/AuthenticationFilterInitializer.java /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/README.txt /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/client /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/client/AuthenticatedURL.java /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/client/AuthenticationException.java /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/client/Authenticator.java /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/client/KerberosAuthenticator.java /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/client/PseudoAuthenticator.java /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/server /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/server/AuthenticationFilter.java /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/server/AuthenticationToken.java /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/server/KerberosAuthenticationHandler.java /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/server/PseudoAuthenticationHandler.java /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/util /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/util/KerberosName.java /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/util/Signer.java /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/security/authentication/util/SignerException.java /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/KerberosTestUtils.java /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/client /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/client/TestAuthenticatedURL.java /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/client/TestKerberosAuthenticator.java /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/client/TestPseudoAuthenticator.java /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/server /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/server/TestAuthenticationFilter.java /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/server/TestAuthenticationToken.java /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/server/TestKerberosAuthenticationHandler.java /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/server/TestPseudoAuthenticationHandler.java /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/util /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/util/TestKerberosName.java /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/security/authentication/util/TestSigner.java
          Hide
          Konstantin Shvachko added a comment -

          I just committed this to branch 0.22.1. Thank you Benoy.

          Show
          Konstantin Shvachko added a comment - I just committed this to branch 0.22.1. Thank you Benoy.
          Hide
          Benoy Antony added a comment -

          Fixed FIndBugs warnings - 17 of them

          Show
          Benoy Antony added a comment - Fixed FIndBugs warnings - 17 of them
          Hide
          Benoy Antony added a comment -

          This has the patch for 22

          Show
          Benoy Antony added a comment - This has the patch for 22
          Hide
          Tsz Wo Nicholas Sze added a comment -

          The patch committed introduced findbugs warnigns (HADOOP-7576), javac warnings (HADOOP-8119), javadoc warnings (HADOOP-7898) and broke mvn eclipse:eclipse (HADOOP-7567). The javac and findbugs warnings were catched by this Jenkins report.

          > +1, the latest patch looks good to me. I'll commit this shortly.

          Hi Aaron, why the patch was committed?

          Show
          Tsz Wo Nicholas Sze added a comment - The patch committed introduced findbugs warnigns ( HADOOP-7576 ), javac warnings ( HADOOP-8119 ), javadoc warnings ( HADOOP-7898 ) and broke mvn eclipse:eclipse ( HADOOP-7567 ). The javac and findbugs warnings were catched by this Jenkins report . > +1, the latest patch looks good to me. I'll commit this shortly. Hi Aaron, why the patch was committed?
          Hide
          Matt Foley added a comment -

          Closed upon release of 0.20.205.0

          Show
          Matt Foley added a comment - Closed upon release of 0.20.205.0
          Hide
          Aaron T. Myers added a comment -

          Looks like the branch-0.20-security back-port of this JIRA missed the changes to core-default.xml that went into trunk. I've filed HADOOP-7665 to address this.

          Show
          Aaron T. Myers added a comment - Looks like the branch-0.20-security back-port of this JIRA missed the changes to core-default.xml that went into trunk. I've filed HADOOP-7665 to address this.
          Hide
          Aaron T. Myers added a comment -

          I've filed HADOOP-7645 to get the Kerberos tests addressed.

          Show
          Aaron T. Myers added a comment - I've filed HADOOP-7645 to get the Kerberos tests addressed.
          Hide
          Aaron T. Myers added a comment -

          Sanjay, did you file a JIRA to add the Kerberos tests to branch-0.20-security?

          Show
          Aaron T. Myers added a comment - Sanjay, did you file a JIRA to add the Kerberos tests to branch-0.20-security?
          Hide
          Sanjay Radia added a comment -

          committed to branch-0.20-security to go into 20.205

          Show
          Sanjay Radia added a comment - committed to branch-0.20-security to go into 20.205
          Hide
          Sanjay Radia added a comment -

          Last patch has the site stuff. Ignore my previous comment about apt and forrest; I misunderstood Alejandro.

          Show
          Sanjay Radia added a comment - Last patch has the site stuff. Ignore my previous comment about apt and forrest; I misunderstood Alejandro.
          Hide
          Sanjay Radia added a comment -

          updated patch
          Still working on site.xml - this is in apt and will need to be ported to forrest for 20.

          Show
          Sanjay Radia added a comment - updated patch Still working on site.xml - this is in apt and will need to be ported to forrest for 20.
          Hide
          Aaron T. Myers added a comment -

          Regarding the missing testcases, those are the Kerberos ones, a KDC setup is required to run them, ideally we should have them in, but if pressed with time I think is OK to commit as it is (exact working code from trunk) and open a JIRA to add them before the next maintenance release of the 2xx branch. Or, for the 205 release add the Kerberos testcases to the exclude list in the build.

          It seems fine to me to do that as a follow-up JIRA. Could someone please file that?

          What is missing are the docs (changes to the xdocs/site.xml and the new hadoop-xdocs/HttpAuthentication.xml file), I think those should go in.

          Agreed.

          I should also mention that Alejandro and I tested this patch manually yesterday (with the addition of AuthenticationFilterInitializer) and it worked like a charm, both from curl and in Firefox. So, +1 for the back-port once the above are addressed.

          Show
          Aaron T. Myers added a comment - Regarding the missing testcases, those are the Kerberos ones, a KDC setup is required to run them, ideally we should have them in, but if pressed with time I think is OK to commit as it is (exact working code from trunk) and open a JIRA to add them before the next maintenance release of the 2xx branch. Or, for the 205 release add the Kerberos testcases to the exclude list in the build. It seems fine to me to do that as a follow-up JIRA. Could someone please file that? What is missing are the docs (changes to the xdocs/site.xml and the new hadoop-xdocs/HttpAuthentication.xml file), I think those should go in. Agreed. I should also mention that Alejandro and I tested this patch manually yesterday (with the addition of AuthenticationFilterInitializer) and it worked like a charm, both from curl and in Firefox. So, +1 for the back-port once the above are addressed.
          Hide
          Alejandro Abdelnur added a comment -

          Regarding the missing testcases, those are the Kerberos ones, a KDC setup is required to run them, ideally we should have them in, but if pressed with time I think is OK to commit as it is (exact working code from trunk) and open a JIRA to add them before the next maintenance release of the 2xx branch. Or, for the 205 release add the Kerberos testcases to the exclude list in the build.

          What is missing are the docs (changes to the xdocs/site.xml and the new hadoop-xdocs/HttpAuthentication.xml file), I think those should go in.

          Besides this +1 (not binding).

          Show
          Alejandro Abdelnur added a comment - Regarding the missing testcases, those are the Kerberos ones, a KDC setup is required to run them, ideally we should have them in, but if pressed with time I think is OK to commit as it is (exact working code from trunk) and open a JIRA to add them before the next maintenance release of the 2xx branch. Or, for the 205 release add the Kerberos testcases to the exclude list in the build. What is missing are the docs (changes to the xdocs/site.xml and the new hadoop-xdocs/HttpAuthentication.xml file), I think those should go in. Besides this +1 (not binding).
          Hide
          Sanjay Radia added a comment -

          Updated path with AuthenticationFilterInitializer.java

          Show
          Sanjay Radia added a comment - Updated path with AuthenticationFilterInitializer.java
          Hide
          Aaron T. Myers added a comment -

          Sanjay, also note that some of tests which are part of the back-port require Kerberos infrastructure to be present in order to run. Under the maven build system, these tests will not be run by default, unless you provide the option "-PtestKerberos" to mvn. We'll probably need to do something similar in ant for the back-port.

          Show
          Aaron T. Myers added a comment - Sanjay, also note that some of tests which are part of the back-port require Kerberos infrastructure to be present in order to run. Under the maven build system, these tests will not be run by default, unless you provide the option "-PtestKerberos" to mvn. We'll probably need to do something similar in ant for the back-port.
          Hide
          Alejandro Abdelnur added a comment -

          Sanjay, your patch is missing the AuthenticationFilterInitializer.java file

          Show
          Alejandro Abdelnur added a comment - Sanjay, your patch is missing the AuthenticationFilterInitializer.java file
          Hide
          Matt Foley added a comment -

          Re-opened for 0.20.205 target. Not yet committed.

          Show
          Matt Foley added a comment - Re-opened for 0.20.205 target. Not yet committed.
          Hide
          Alejandro Abdelnur added a comment -

          Sanjay, regarding the patch for 0.20.205.0, the docs changes are missing (changes to the xdocs/site.xml and the new hadoop-xdocs/HttpAuthentication.xml file)

          Show
          Alejandro Abdelnur added a comment - Sanjay, regarding the patch for 0.20.205.0, the docs changes are missing (changes to the xdocs/site.xml and the new hadoop-xdocs/HttpAuthentication.xml file)
          Hide
          Sanjay Radia added a comment -

          Ported patch to 20-security (after the renames of HADOOP-7579)

          Show
          Sanjay Radia added a comment - Ported patch to 20-security (after the renames of HADOOP-7579 )
          Hide
          Mahadev konar added a comment -

          @alejandro thanks. Will comment!

          Show
          Mahadev konar added a comment - @alejandro thanks. Will comment!
          Hide
          Alejandro Abdelnur added a comment -

          @Mahadev, done HADOOP-7560, please comment there regarding module names.

          Show
          Alejandro Abdelnur added a comment - @Mahadev, done HADOOP-7560 , please comment there regarding module names.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #761 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/761/)
          Follow-up to HADOOP-7119 - removing two files which were moved.

          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1159806
          Files :

          • /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/KerberosName.java
          • /hadoop/common/trunk/hadoop-common/src/test/java/org/apache/hadoop/security/TestKerberosName.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #761 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/761/ ) Follow-up to HADOOP-7119 - removing two files which were moved. atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1159806 Files : /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/KerberosName.java /hadoop/common/trunk/hadoop-common/src/test/java/org/apache/hadoop/security/TestKerberosName.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #760 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/760/)
          HADOOP-7119. add Kerberos HTTP SPNEGO authentication support to Hadoop JT/NN/DN/TT web-consoles. (Alejandro Abdelnur via atm)

          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1159804
          Files :

          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/util
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/server
          • /hadoop/common/trunk/hadoop-common/src/main/docs/src/documentation/content/xdocs/HttpAuthentication.xml
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/annonymous/index.html
          • /hadoop/common/trunk/hadoop-alfredo/src/site/apt/Configuration.apt.vm
          • /hadoop/common/trunk/hadoop-alfredo/src/site/apt/index.apt.vm
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/simple
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/client
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/KerberosTestUtils.java
          • /hadoop/common/trunk/hadoop-alfredo/src/site
          • /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/User.java
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java
          • /hadoop/common/trunk/hadoop-common/src/test/java/org/apache/hadoop/security/TestAuthenticationFilter.java
          • /hadoop/common/trunk/hadoop-alfredo/src/examples
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/annonymous
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org/apache/hadoop/alfredo/examples/RequestLoggerFilter.java
          • /hadoop/common/trunk/hadoop-alfredo/src/site/site.xml
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java
          • /hadoop/common/trunk/pom.xml
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/server/AuthenticationToken.java
          • /hadoop/common/trunk/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/server/TestKerberosAuthenticationHandler.java
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/client/PseudoAuthenticator.java
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/index.html
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/server/KerberosAuthenticationHandler.java
          • /hadoop/common/trunk/hadoop-alfredo/src/site/apt/BuildingIt.apt.vm
          • /hadoop/common/trunk/hadoop-alfredo/src/site/apt
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/kerberos/index.html
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/util
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo
          • /hadoop/common/trunk/hadoop-alfredo/README.txt
          • /hadoop/common/trunk/hadoop-alfredo/src
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo
          • /hadoop/common/trunk/hadoop-alfredo/BUILDING.txt
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org/apache/hadoop/alfredo/examples/WhoServlet.java
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/kerberos
          • /hadoop/common/trunk/hadoop-alfredo/src/test/resources/krb5.conf
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/client/TestAuthenticatedURL.java
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/client/Authenticator.java
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/resources/log4j.properties
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org/apache
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-alfredo/pom.xml
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/WEB-INF/web.xml
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/client/AuthenticatorTestCase.java
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org/apache/hadoop/alfredo/examples/WhoClient.java
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/server/PseudoAuthenticationHandler.java
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/util/KerberosName.java
          • /hadoop/common/trunk/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/util/TestSigner.java
          • /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java
          • /hadoop/common/trunk/hadoop-common/src/main/docs/src/documentation/content/xdocs/site.xml
          • /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/HadoopKerberosName.java
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/server/TestAuthenticationToken.java
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/WEB-INF
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/util/SignerException.java
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/client/TestPseudoAuthenticator.java
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/client/AuthenticatedURL.java
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache
          • /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java
          • /hadoop/common/trunk/hadoop-alfredo
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org/apache/hadoop/alfredo/examples
          • /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/AuthenticationFilterInitializer.java
          • /hadoop/common/trunk/hadoop-alfredo/src/test
          • /hadoop/common/trunk/hadoop-alfredo/src/site/apt/Examples.apt.vm
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/util/Signer.java
          • /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/server/TestAuthenticationFilter.java
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org/apache/hadoop/alfredo
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/server/AuthenticationHandler.java
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/server
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/util/TestKerberosName.java
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/simple/index.html
          • /hadoop/common/trunk/hadoop-alfredo/src/test/resources
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/client/TestKerberosAuthenticator.java
          • /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenIdentifier.java
          • /hadoop/common/trunk/hadoop-common/src/main/resources/core-default.xml
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/client
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/client/KerberosAuthenticator.java
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/pom.xml
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/client/AuthenticationException.java
          • /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/server/AuthenticationFilter.java
          • /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/resources
          • /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/server/TestPseudoAuthenticationHandler.java
          • /hadoop/common/trunk/hadoop-alfredo/src/main
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #760 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/760/ ) HADOOP-7119 . add Kerberos HTTP SPNEGO authentication support to Hadoop JT/NN/DN/TT web-consoles. (Alejandro Abdelnur via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1159804 Files : /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/util /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/server /hadoop/common/trunk/hadoop-common/src/main/docs/src/documentation/content/xdocs/HttpAuthentication.xml /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/annonymous/index.html /hadoop/common/trunk/hadoop-alfredo/src/site/apt/Configuration.apt.vm /hadoop/common/trunk/hadoop-alfredo/src/site/apt/index.apt.vm /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/simple /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/client /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/KerberosTestUtils.java /hadoop/common/trunk/hadoop-alfredo/src/site /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/User.java /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java /hadoop/common/trunk/hadoop-common/src/test/java/org/apache/hadoop/security/TestAuthenticationFilter.java /hadoop/common/trunk/hadoop-alfredo/src/examples /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/annonymous /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org/apache/hadoop/alfredo/examples/RequestLoggerFilter.java /hadoop/common/trunk/hadoop-alfredo/src/site/site.xml /hadoop/common/trunk/hadoop-alfredo/src/main/java /hadoop/common/trunk/pom.xml /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/server/AuthenticationToken.java /hadoop/common/trunk/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/server/TestKerberosAuthenticationHandler.java /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/client/PseudoAuthenticator.java /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/index.html /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/server/KerberosAuthenticationHandler.java /hadoop/common/trunk/hadoop-alfredo/src/site/apt/BuildingIt.apt.vm /hadoop/common/trunk/hadoop-alfredo/src/site/apt /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/kerberos/index.html /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/util /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo /hadoop/common/trunk/hadoop-alfredo/README.txt /hadoop/common/trunk/hadoop-alfredo/src /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache /hadoop/common/trunk/hadoop-alfredo/src/test/java/org /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo /hadoop/common/trunk/hadoop-alfredo/BUILDING.txt /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org/apache/hadoop/alfredo/examples/WhoServlet.java /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/kerberos /hadoop/common/trunk/hadoop-alfredo/src/test/resources/krb5.conf /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/client/TestAuthenticatedURL.java /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/client/Authenticator.java /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/resources/log4j.properties /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org/apache /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-alfredo/pom.xml /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/WEB-INF/web.xml /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/client/AuthenticatorTestCase.java /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org/apache/hadoop/alfredo/examples/WhoClient.java /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/server/PseudoAuthenticationHandler.java /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/util/KerberosName.java /hadoop/common/trunk/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/util/TestSigner.java /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java /hadoop/common/trunk/hadoop-common/src/main/docs/src/documentation/content/xdocs/site.xml /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/HadoopKerberosName.java /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/server/TestAuthenticationToken.java /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/WEB-INF /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/util/SignerException.java /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/client/TestPseudoAuthenticator.java /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/client/AuthenticatedURL.java /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java /hadoop/common/trunk/hadoop-alfredo /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org/apache/hadoop/alfredo/examples /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/AuthenticationFilterInitializer.java /hadoop/common/trunk/hadoop-alfredo/src/test /hadoop/common/trunk/hadoop-alfredo/src/site/apt/Examples.apt.vm /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org/apache/hadoop /hadoop/common/trunk/hadoop-alfredo/src/test/java /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/util/Signer.java /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/server/TestAuthenticationFilter.java /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/java/org/apache/hadoop/alfredo /hadoop/common/trunk/hadoop-alfredo/src/examples/src /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/server/AuthenticationHandler.java /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main /hadoop/common/trunk/hadoop-alfredo/src/main/java/org /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/server /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/util/TestKerberosName.java /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/webapp/simple/index.html /hadoop/common/trunk/hadoop-alfredo/src/test/resources /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/client/TestKerberosAuthenticator.java /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenIdentifier.java /hadoop/common/trunk/hadoop-common/src/main/resources/core-default.xml /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/client /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/client/KerberosAuthenticator.java /hadoop/common/trunk/hadoop-alfredo/src/examples/pom.xml /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/client/AuthenticationException.java /hadoop/common/trunk/hadoop-alfredo/src/main/java/org/apache/hadoop/alfredo/server/AuthenticationFilter.java /hadoop/common/trunk/hadoop-alfredo/src/examples/src/main/resources /hadoop/common/trunk/hadoop-alfredo/src/test/java/org/apache/hadoop/alfredo/server/TestPseudoAuthenticationHandler.java /hadoop/common/trunk/hadoop-alfredo/src/main
          Hide
          Aaron T. Myers added a comment -

          I've just committed this. Thanks a lot for the monster contribution, Alejandro!

          Show
          Aaron T. Myers added a comment - I've just committed this. Thanks a lot for the monster contribution, Alejandro!
          Hide
          Mahadev konar added a comment -

          @alejandro, makes sense to do it as follow on work. Mind opening a jira for that?

          Show
          Mahadev konar added a comment - @alejandro, makes sense to do it as follow on work. Mind opening a jira for that?
          Hide
          Aaron T. Myers added a comment -

          +1, the latest patch looks good to me. I'll commit this shortly.

          Show
          Aaron T. Myers added a comment - +1, the latest patch looks good to me. I'll commit this shortly.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12491008/HADOOP-7119v6.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 38 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          -1 javac. The applied patch generated 296 javac compiler warnings (more than the trunk's current 289 warnings).

          -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/60//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/60//artifact/trunk/target/newPatchFindbugsWarningshadoop-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/60//artifact/trunk/target/newPatchFindbugsWarningshadoop-alfredo.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/60//artifact/trunk/target/newPatchFindbugsWarningshadoop-hdfs.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/60//artifact/trunk/target/newPatchFindbugsWarningshadoop-annotations.html
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/60//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12491008/HADOOP-7119v6.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 38 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 296 javac compiler warnings (more than the trunk's current 289 warnings). -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/60//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/60//artifact/trunk/target/newPatchFindbugsWarningshadoop-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/60//artifact/trunk/target/newPatchFindbugsWarningshadoop-alfredo.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/60//artifact/trunk/target/newPatchFindbugsWarningshadoop-hdfs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/60//artifact/trunk/target/newPatchFindbugsWarningshadoop-annotations.html Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/60//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          'v6' takes care of documentation nits

          Show
          Alejandro Abdelnur added a comment - 'v6' takes care of documentation nits
          Hide
          Alejandro Abdelnur added a comment -

          @Mahadev, to make a module inside hadoop-common, we'll have to make hadoop-common a POM module and have a sub-module for all the stuff it is currently there (Java, scripts, docs, etc). In maven, an aggregator module (a POM module) does not produce JARs.

          It is possible, and it would make sense, but I'd see that as an incremental refactoring later on.

          Show
          Alejandro Abdelnur added a comment - @Mahadev, to make a module inside hadoop-common, we'll have to make hadoop-common a POM module and have a sub-module for all the stuff it is currently there (Java, scripts, docs, etc). In maven, an aggregator module (a POM module) does not produce JARs. It is possible, and it would make sense, but I'd see that as an incremental refactoring later on.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12490899/HADOOP-7119v5.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 38 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          -1 javac. The applied patch generated 285 javac compiler warnings (more than the trunk's current 278 warnings).

          -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/52//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/52//artifact/trunk/target/newPatchFindbugsWarningshadoop-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/52//artifact/trunk/target/newPatchFindbugsWarningshadoop-alfredo.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/52//artifact/trunk/target/newPatchFindbugsWarningshadoop-annotations.html
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/52//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490899/HADOOP-7119v5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 38 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 285 javac compiler warnings (more than the trunk's current 278 warnings). -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/52//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/52//artifact/trunk/target/newPatchFindbugsWarningshadoop-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/52//artifact/trunk/target/newPatchFindbugsWarningshadoop-alfredo.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/52//artifact/trunk/target/newPatchFindbugsWarningshadoop-annotations.html Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/52//console This message is automatically generated.
          Hide
          Mahadev konar added a comment -

          Alejandro,
          Just browsed through the patch. Any reason its not a sub module inside hadoop-common? How will the artifacts be published? Sorry I am not an expert on maven. Just trying to understand the artifact/directory structure.

          Show
          Mahadev konar added a comment - Alejandro, Just browsed through the patch. Any reason its not a sub module inside hadoop-common? How will the artifacts be published? Sorry I am not an expert on maven. Just trying to understand the artifact/directory structure.
          Hide
          Alejandro Abdelnur added a comment -

          v5, integrating patch amendement by ATM and using KerberosName to resolve name.

          Note on KerberosName, I've moved the class from hadoop-common to hadoop-alfredo removing the Configuration dependency and created a HadoopKerberosName in hadoop-common to be used as before.

          I've done this change to keep Alfredo independent of hadoop-common.

          Show
          Alejandro Abdelnur added a comment - v5, integrating patch amendement by ATM and using KerberosName to resolve name. Note on KerberosName , I've moved the class from hadoop-common to hadoop-alfredo removing the Configuration dependency and created a HadoopKerberosName in hadoop-common to be used as before. I've done this change to keep Alfredo independent of hadoop-common.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12490768/HADOOP-7119v4-amendment.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 30 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/47//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490768/HADOOP-7119v4-amendment.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 30 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/47//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          Clean-up patch which applies on top of HADOOP-7199v4.patch.

          Show
          Aaron T. Myers added a comment - Clean-up patch which applies on top of HADOOP-7199 v4.patch.
          Hide
          Aaron T. Myers added a comment -

          Patch looks pretty good, Alejandro. I think it's very close to being able to be committed.

          Awesome that you included tests which work if a Kerberos environment is available. Thanks for doing that. I ran all the tests (with and without Kerberos present) and they all passed. I also ran all of the maven goals and they all worked flawlessly. I also reviewed all of the code (though none of the code to build the project). The following are the comments from that review:

          1. In KerberosAuthenticationHandler.authenticate, you determine the user's name by always taking the first component of the fully Kerberos principal name. Hadoop (and MIT Kerberos) allow for one to configure arbitrary rules to perform this mapping. In order to be compatible with Hadoop in this respect, I would think that Alfredo would also need to perform this mapping.
          2. README.txt has a few errors (documentation location, user mailing list.)
          3. Typo in BuildingIt.apt.vm: "can be used to change de default"
          4. In BuildingIt.apt.vm, you seem to indicate that if one changes the default values for alfredo.test.ker
            beros.server.principal or alfredo.test.kerberos.client.principal that one must include the realm part of
            the principal name. In fact, doing so will cause the tests to fail.
          5. In all of the documentation you include the author's name. The Hadoop projects deliberately do not inc
            lude author tags in the source.

          While I was going through the code I found a number of little things (typos mostly, and style stuff) that could use clean-up. I'll attach a patch shortly which should be applied on top of HADOOP-7119v4.patch. This seemed like the easiest way for you to review those changes.

          Show
          Aaron T. Myers added a comment - Patch looks pretty good, Alejandro. I think it's very close to being able to be committed. Awesome that you included tests which work if a Kerberos environment is available. Thanks for doing that. I ran all the tests (with and without Kerberos present) and they all passed. I also ran all of the maven goals and they all worked flawlessly. I also reviewed all of the code (though none of the code to build the project). The following are the comments from that review: In KerberosAuthenticationHandler.authenticate, you determine the user's name by always taking the first component of the fully Kerberos principal name. Hadoop (and MIT Kerberos) allow for one to configure arbitrary rules to perform this mapping. In order to be compatible with Hadoop in this respect, I would think that Alfredo would also need to perform this mapping. README.txt has a few errors (documentation location, user mailing list.) Typo in BuildingIt.apt.vm: "can be used to change de default" In BuildingIt.apt.vm, you seem to indicate that if one changes the default values for alfredo.test.ker beros.server.principal or alfredo.test.kerberos.client.principal that one must include the realm part of the principal name. In fact, doing so will cause the tests to fail. In all of the documentation you include the author's name. The Hadoop projects deliberately do not inc lude author tags in the source. While I was going through the code I found a number of little things (typos mostly, and style stuff) that could use clean-up. I'll attach a patch shortly which should be applied on top of HADOOP-7119 v4.patch. This seemed like the easiest way for you to review those changes.
          Hide
          Alejandro Abdelnur added a comment -

          'v4' rebased to trunk's HEAD

          Show
          Alejandro Abdelnur added a comment - 'v4' rebased to trunk's HEAD
          Hide
          Alejandro Abdelnur added a comment -

          HDFS and MR don't need further patching (it was build wiring, which is now unnecessary because of Mavenization)

          Show
          Alejandro Abdelnur added a comment - HDFS and MR don't need further patching (it was build wiring, which is now unnecessary because of Mavenization)
          Hide
          Alejandro Abdelnur added a comment -

          'v3' patch brings Alfredo codebase into Hadoop. and wires Hadoop common (there are 2 other JIRAs, one for MR and one for HDFS to follow up with their wiring there).

          Show
          Alejandro Abdelnur added a comment - 'v3' patch brings Alfredo codebase into Hadoop. and wires Hadoop common (there are 2 other JIRAs, one for MR and one for HDFS to follow up with their wiring there).
          Hide
          Alejandro Abdelnur added a comment -

          I'm working on an update for this patch that would bring all Alfredo code into Hadoop.

          Show
          Alejandro Abdelnur added a comment - I'm working on an update for this patch that would bring all Alfredo code into Hadoop.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12469816/ha-common-02.patch
          against trunk revision 1071364.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/244//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/244//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/244//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12469816/ha-common-02.patch against trunk revision 1071364. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/244//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/244//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/244//console This message is automatically generated.
          Hide
          Eli Collins added a comment -

          How about contributing it to HttpComponents? Looks like it would be a relevant addition to their project, saves us from adding a generic library to core Hadoop and we can leverage their existing project/infrastructure.

          Show
          Eli Collins added a comment - How about contributing it to HttpComponents? Looks like it would be a relevant addition to their project, saves us from adding a generic library to core Hadoop and we can leverage their existing project/infrastructure.
          Hide
          Alejandro Abdelnur added a comment -

          As mentioned before one of the motivations of having this functionality independent of Hadoop is that it will be possible to use it independently of Hadoop releases. If Oozie/Hbase/Hive (just to name a few projects with HTTP endpoints) want to use it, they don't have to wait until Hadoop ships with this functionality and they could use it with other versions of Hadoop.

          If we can find a way to have this functionality in Hadoop but still be independent of Hadoop releases, I'm good.

          Could be a Hadoop sub-project? Could Hadoop depend on one of its sub-projects?

          Thoughts?

          Show
          Alejandro Abdelnur added a comment - As mentioned before one of the motivations of having this functionality independent of Hadoop is that it will be possible to use it independently of Hadoop releases. If Oozie/Hbase/Hive (just to name a few projects with HTTP endpoints) want to use it, they don't have to wait until Hadoop ships with this functionality and they could use it with other versions of Hadoop. If we can find a way to have this functionality in Hadoop but still be independent of Hadoop releases, I'm good. Could be a Hadoop sub-project? Could Hadoop depend on one of its sub-projects? Thoughts?
          Hide
          eric baldeschwieler added a comment -

          It would be great to get SPNEGO support checked into hadoop, rather than picking up an external dependency for a small amount of just written code (assuming no preexisting code solves the problem). Especially given that this is security code and that it doesn't have a community beyond hadoop, having it in the project seems like a good idea.

          This isn't to say that others can't use it right away, but any modification to hadoop consoles won't ship until hadoop does a release anyway, so there is no reason not to add the functionality in "alfredo" into hadoop common with this patch, is there?

          Show
          eric baldeschwieler added a comment - It would be great to get SPNEGO support checked into hadoop, rather than picking up an external dependency for a small amount of just written code (assuming no preexisting code solves the problem). Especially given that this is security code and that it doesn't have a community beyond hadoop, having it in the project seems like a good idea. This isn't to say that others can't use it right away, but any modification to hadoop consoles won't ship until hadoop does a release anyway, so there is no reason not to add the functionality in "alfredo" into hadoop common with this patch, is there?
          Hide
          Rajiv Chittajallu added a comment -

          bq (when I reviewed the security work last summer I found several root exploits and data confidentiality exploits that the Y paranoids missed in their review.)

          Yahoo! central paranoid team reviewed the architecture. Not the code. The community is responsible of the code. Trust on the code base builds on how active the community is reviewing the code and fixing it. Its based on javax.security.sasl and any extensions are added in Hadoop but not released as an external component.

          Right now there isn't an http.filter of web consoles. The patch provides one. Its probably might be acceptable to others in adding this to contrib.

          Show
          Rajiv Chittajallu added a comment - bq (when I reviewed the security work last summer I found several root exploits and data confidentiality exploits that the Y paranoids missed in their review.) Yahoo! central paranoid team reviewed the architecture. Not the code. The community is responsible of the code. Trust on the code base builds on how active the community is reviewing the code and fixing it. Its based on javax.security.sasl and any extensions are added in Hadoop but not released as an external component. Right now there isn't an http.filter of web consoles. The patch provides one. Its probably might be acceptable to others in adding this to contrib.
          Hide
          Todd Lipcon added a comment -

          (In the case of the existent Kerberos code in Hadoop, I know it was reviewed by the paranoids at Y!.)

          I think it's important that both outside "security experts" and also "framework experts" review code – for example, when I reviewed the security work last summer I found several root exploits and data confidentiality exploits that the Y paranoids missed in their review.

          Security is never perfect. The more eyes the better. Who those eyes happen to be is probably less important so long as they're not malicious.

          Show
          Todd Lipcon added a comment - (In the case of the existent Kerberos code in Hadoop, I know it was reviewed by the paranoids at Y!.) I think it's important that both outside "security experts" and also "framework experts" review code – for example, when I reviewed the security work last summer I found several root exploits and data confidentiality exploits that the Y paranoids missed in their review. Security is never perfect. The more eyes the better. Who those eyes happen to be is probably less important so long as they're not malicious.
          Hide
          Allen Wittenauer added a comment -

          You have the answer yourself: if it is truly meant to be a generic framework, it would be good to get some outside eyes on it before we build something based on it. This way if there any changes that need to be made, they can be done earlier rather than later.

          The other thing to keep in mind is that I don't think any of us would particularly qualify as a security expert. (In the case of the existent Kerberos code in Hadoop, I know it was reviewed by the paranoids at Y!.)

          Show
          Allen Wittenauer added a comment - You have the answer yourself: if it is truly meant to be a generic framework, it would be good to get some outside eyes on it before we build something based on it. This way if there any changes that need to be made, they can be done earlier rather than later. The other thing to keep in mind is that I don't think any of us would particularly qualify as a security expert. (In the case of the existent Kerberos code in Hadoop, I know it was reviewed by the paranoids at Y!.)
          Hide
          Alejandro Abdelnur added a comment -

          Doing a audit/review of Alfredo sounds good (I didn't know Hadoop was doing audit/reviews on new dependencies, I guess it is just because is a security related component).

          The thing I don't get is why somebody outside of the community has to audit/review it? Shouldn't be the other way around?

          Said this, how do we get this audit/review started? It should be that difficult as Alfredo is quite simple and small (~1000 lines of code total without counting javadocs & testcases).

          Some useful background info:

          Alfredo was developed because I couldn't find an alternative already available. The closest thing I've found is the SourceForge SPNEGO project but it is LGPL (and I don't think they have source code avail).

          As I've mentioned before, the reasons for Alfredo to be a separate project and not part of the Hadoop patch were that other Hadoop related projects could start using it today without having to wait for a Hadoop release. Plus, it has applicability outside of Hadoop projects.

          Show
          Alejandro Abdelnur added a comment - Doing a audit/review of Alfredo sounds good (I didn't know Hadoop was doing audit/reviews on new dependencies, I guess it is just because is a security related component). The thing I don't get is why somebody outside of the community has to audit/review it? Shouldn't be the other way around? Said this, how do we get this audit/review started? It should be that difficult as Alfredo is quite simple and small (~1000 lines of code total without counting javadocs & testcases). Some useful background info: Alfredo was developed because I couldn't find an alternative already available. The closest thing I've found is the SourceForge SPNEGO project but it is LGPL (and I don't think they have source code avail). As I've mentioned before, the reasons for Alfredo to be a separate project and not part of the Hadoop patch were that other Hadoop related projects could start using it today without having to wait for a Hadoop release. Plus, it has applicability outside of Hadoop projects.
          Hide
          Allen Wittenauer added a comment -

          It would be good to have someone outside the community do a security review/audit of Alfredo. It is simply too young to be trusted.

          Until that happens, my comfort level with this patch is low.

          Show
          Allen Wittenauer added a comment - It would be good to have someone outside the community do a security review/audit of Alfredo. It is simply too young to be trusted. Until that happens, my comfort level with this patch is low.
          Hide
          Jakob Homan added a comment -

          I'm hoping to take a look at this later this week, but before this code goes in, we'll need to do a review of Alfredo as well. It's a new library that's not been reviewed or tested, but would become a crucial part of the security infrastructure, assuming this patch goes in.

          Show
          Jakob Homan added a comment - I'm hoping to take a look at this later this week, but before this code goes in, we'll need to do a review of Alfredo as well. It's a new library that's not been reviewed or tested, but would become a crucial part of the security infrastructure, assuming this patch goes in.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12469816/ha-common-02.patch
          against trunk revision 1064919.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/211//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/211//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/211//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12469816/ha-common-02.patch against trunk revision 1064919. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/211//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/211//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/211//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          wrong patch format

          Show
          Alejandro Abdelnur added a comment - wrong patch format
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12469793/ha-common-01.patch
          against trunk revision 1064919.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/210//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12469793/ha-common-01.patch against trunk revision 1064919. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/210//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          This patch, as suggested, makes use of hadoop.http.filter.initializers

          Show
          Alejandro Abdelnur added a comment - This patch, as suggested, makes use of hadoop.http.filter.initializers
          Hide
          Alejandro Abdelnur added a comment -

          Thanks for the comments/feedback.

          On not using hadoop.http.filter.initializers

          I've completely missed the existence of this. I'll refactor the patch to levarage it, it will make the patch much simpler and won't have to mock around with the build.

          On why dependent upon a cookie

          HTTP SPNEGO takes care of the authentication protocol only, it does not take care of keeping track of authenticated requests. Once the HTTP SPNEGO sequence is complete, you responsible for keeping track of authenticated user-agents. That is where the cookie comes into play, this is how most HTTP authentication mechanisms work and browsers support it.

          Furthermore, if you would not use cookies, you would initiate an HTTP SPNEGO authentication sequence on every request, which requires an extra HTTP round trip. This would be an expensive operation. And it would work only with HTTP GET requests (I believe this would not be an issue for today's Hadoop console).

          On why not using HttpComponents

          As far as I know, HttpComponents is client side only. This patch is using Alfredo server side authentication capabilities, not the client ones.

          I'l work in a new patch using hadoop.http.filter.initializers.

          Thanks.

          Show
          Alejandro Abdelnur added a comment - Thanks for the comments/feedback. On not using hadoop.http.filter.initializers I've completely missed the existence of this. I'll refactor the patch to levarage it, it will make the patch much simpler and won't have to mock around with the build. On why dependent upon a cookie HTTP SPNEGO takes care of the authentication protocol only, it does not take care of keeping track of authenticated requests. Once the HTTP SPNEGO sequence is complete, you responsible for keeping track of authenticated user-agents. That is where the cookie comes into play, this is how most HTTP authentication mechanisms work and browsers support it. Furthermore, if you would not use cookies, you would initiate an HTTP SPNEGO authentication sequence on every request, which requires an extra HTTP round trip. This would be an expensive operation. And it would work only with HTTP GET requests (I believe this would not be an issue for today's Hadoop console). On why not using HttpComponents As far as I know, HttpComponents is client side only. This patch is using Alfredo server side authentication capabilities, not the client ones. I'l work in a new patch using hadoop.http.filter.initializers. Thanks.
          Hide
          Owen O'Malley added a comment -

          I'm also concerned about not using the hadoop.http.filter.initializers and want to investigate the differences between Alfredo and HttpComponents. If HttpComponents works, that would be better since it is an Apache project and has a community supporting it.

          Show
          Owen O'Malley added a comment - I'm also concerned about not using the hadoop.http.filter.initializers and want to investigate the differences between Alfredo and HttpComponents. If HttpComponents works, that would be better since it is an Apache project and has a community supporting it.
          Hide
          Allen Wittenauer added a comment -

          Maybe I'm missing something, but why are we dependent upon a cookie? Also why are we using Alfredo when Apache has SPNEGO support in HttpComponents?

          Show
          Allen Wittenauer added a comment - Maybe I'm missing something, but why are we dependent upon a cookie? Also why are we using Alfredo when Apache has SPNEGO support in HttpComponents?
          Hide
          Jakob Homan added a comment -

          Is there a reason you're not going with the hadoop.http.filter.initializers configuration parameter designed for pluggable authentication? Hadoop already has this capability via this parameter and, having implemented it to use one company's SSO solution, it worked quite well.

          Show
          Jakob Homan added a comment - Is there a reason you're not going with the hadoop.http.filter.initializers configuration parameter designed for pluggable authentication? Hadoop already has this capability via this parameter and, having implemented it to use one company's SSO solution, it worked quite well.
          Hide
          Alejandro Abdelnur added a comment -

          Regarding previous comment by Hadoop QA, "-1 on tests included"

          The patch adds a filter in front of the JSP in hadoop-hdfs and hadoop-mapreduce.

          The logic of this filter is implemented in Alfredo and tested in Alfredo build.

          The manual steps for testing this patch set are:

          • build Hadoop
          • install Hadoop

          Testing Hadoop with pseudo/simple authentication:

          • start Hadoop
          • check that HTTP web-consoles work (as usual)
          • stop Hadoop
          • add, in core-site.xml, the property
            **hadoop.http.authentication.simple.anonymous.allowed=false
          • start Hadoop
          • try to access HTTP web-consoles, it will return 401 (unauthorized)
          • access HTTP web-consoles using the query string ?user.name=foo

          Testing Hadoop with kerberos authentication:

          • make sure KDC is running (assuming realm name is REALM)
          • create a principal HTTP/localhost@REALM, create ~/hadoop.keytab file with its credentials
          • make sure there are not kerberos credentials in the OS cache, run kdestroy
          • add, in core-site.xml, the following properties
            • hadoop.http.authentication.type=kerberos
            • hadoop.http.authentication.kerberos.principal=HTTP/localhost@REALM
            • hadoop.http.authentication.kerberos.keytab=$ {user.home}

              /hadoop.keytab

          • restart hadoop
          • try to access HTTP web-consoles, it will return 401 (unauthorized) or the browser will attempt to initiate a kerberos session
          • do kinit to initiate a kerberos session
          • access HTTP web-consoles using a browser that supports HTTP SPNEGO (Firefox or IE)
          Show
          Alejandro Abdelnur added a comment - Regarding previous comment by Hadoop QA, "-1 on tests included" The patch adds a filter in front of the JSP in hadoop-hdfs and hadoop-mapreduce. The logic of this filter is implemented in Alfredo and tested in Alfredo build. The manual steps for testing this patch set are: build Hadoop install Hadoop Testing Hadoop with pseudo/simple authentication: start Hadoop check that HTTP web-consoles work (as usual) stop Hadoop add, in core-site.xml, the property **hadoop.http.authentication.simple.anonymous.allowed=false start Hadoop try to access HTTP web-consoles, it will return 401 (unauthorized) access HTTP web-consoles using the query string ?user.name=foo Testing Hadoop with kerberos authentication: make sure KDC is running (assuming realm name is REALM) create a principal HTTP/localhost@REALM, create ~/hadoop.keytab file with its credentials make sure there are not kerberos credentials in the OS cache, run kdestroy add, in core-site.xml, the following properties hadoop.http.authentication.type=kerberos hadoop.http.authentication.kerberos.principal=HTTP/localhost@REALM hadoop.http.authentication.kerberos.keytab=$ {user.home} /hadoop.keytab restart hadoop try to access HTTP web-consoles, it will return 401 (unauthorized) or the browser will attempt to initiate a kerberos session do kinit to initiate a kerberos session access HTTP web-consoles using a browser that supports HTTP SPNEGO (Firefox or IE)
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12469645/ha-commons.patch
          against trunk revision 1064403.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/205//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/205//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/205//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12469645/ha-commons.patch against trunk revision 1064403. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/205//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/205//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/205//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          The attached patches (this JIRA & HDFS-1604 & MAPREDUCE-2287) implement authentication in the following manner:

          A HadoopAuthenticationFilter (A servlet filter) is configured in front of all Hadoop web console JSPs.

          This filter verifies if the incoming request is already authenticated (by the presence of a signed HTTP cookie).

          If the cookie is present, its signature is valid and its value didn't expire; then the request continues its way to the page invoked by the request.

          If the cookie is not present, it is invalid or it expired; then the request is delegated to an authenticator handler. The authenticator handler then is responsible for requesting/validating the user-agent for the user credentials. This may require one or more additional interactions between the authenticator handler and the user-agent (which will be multiple HTTP requests). Once the authenticator handler verifies the credentials and generates an authentication token, a signed cookie is returned to the user-agent for all subsequent invocations.

          The authenticator handler is pluggable and 2 implementations are provided out o the box: pseudo/simple and kerberos.

          The pseudo/simple authenticator handler is equivalent to the Hadoop pseudo/simple authentication. It trusts the value of the user.name query string parameter.

          The pseudo/simple authenticator handler supports an anonymous mode which accepts any request without requiring the user.name query string parameter to create the token (this is the default behavior, preserving the behavior of the Hadoop web-consoles before this patch).

          The kerberos authenticator handler implements the Kerberos HTTP SPNEGO implementation. This authenticator handler will generate a token only if a successful Kerberos HTTP SPNEGO interaction is performed between the user-agent and the authenticator. Browsers like Firefox and Internet Explorer support Kerberos HTTP SPNEGO.

          To use the kerberos authenticator handler an HTTP service kerberos principal is required (HTTP/$HOSTNAME@$REALM) and its credentials must be stored in a keytab file (most likely it would be the same keytab file used by the node for JT/NN/DN/TT credentials if Hadoop Kerberos authentication is on).

          To support an additional authentication mechanism, an authenticator handler implementation must be written. The authentication handler is a simple interface with 3 methods (init/destroy/authenticate).

          The HadoopAuthenticationFilter extends Alfredo AuthenticationFilter overriding the getConfiguration() method to load the configuration from Hadoop conf/ directory (via the Configuration class).

          Alfredo (http://cloudera.github.com/alfredo) is an HTTP client/server authentication framework. Alfredo is distributed under the Apache License, it is fully documented and it has comprehensive test cases.

          As the question may come, the motivation for doing the authentication framework as a separate project (Alfredo) was:

          • Other Hadoop related projects can start using it today without having to wait for a Hadoop release
          • It has applicability outside of Hadoop
          Show
          Alejandro Abdelnur added a comment - The attached patches (this JIRA & HDFS-1604 & MAPREDUCE-2287 ) implement authentication in the following manner: A HadoopAuthenticationFilter (A servlet filter) is configured in front of all Hadoop web console JSPs. This filter verifies if the incoming request is already authenticated (by the presence of a signed HTTP cookie). If the cookie is present, its signature is valid and its value didn't expire; then the request continues its way to the page invoked by the request. If the cookie is not present, it is invalid or it expired; then the request is delegated to an authenticator handler. The authenticator handler then is responsible for requesting/validating the user-agent for the user credentials. This may require one or more additional interactions between the authenticator handler and the user-agent (which will be multiple HTTP requests). Once the authenticator handler verifies the credentials and generates an authentication token, a signed cookie is returned to the user-agent for all subsequent invocations. The authenticator handler is pluggable and 2 implementations are provided out o the box: pseudo/simple and kerberos . The pseudo/simple authenticator handler is equivalent to the Hadoop pseudo/simple authentication. It trusts the value of the user.name query string parameter. The pseudo/simple authenticator handler supports an anonymous mode which accepts any request without requiring the user.name query string parameter to create the token (this is the default behavior, preserving the behavior of the Hadoop web-consoles before this patch). The kerberos authenticator handler implements the Kerberos HTTP SPNEGO implementation. This authenticator handler will generate a token only if a successful Kerberos HTTP SPNEGO interaction is performed between the user-agent and the authenticator. Browsers like Firefox and Internet Explorer support Kerberos HTTP SPNEGO. To use the kerberos authenticator handler an HTTP service kerberos principal is required (HTTP/$HOSTNAME@$REALM) and its credentials must be stored in a keytab file (most likely it would be the same keytab file used by the node for JT/NN/DN/TT credentials if Hadoop Kerberos authentication is on). To support an additional authentication mechanism, an authenticator handler implementation must be written. The authentication handler is a simple interface with 3 methods (init/destroy/authenticate). The HadoopAuthenticationFilter extends Alfredo AuthenticationFilter overriding the getConfiguration() method to load the configuration from Hadoop conf/ directory (via the Configuration class). Alfredo ( http://cloudera.github.com/alfredo ) is an HTTP client/server authentication framework. Alfredo is distributed under the Apache License, it is fully documented and it has comprehensive test cases. As the question may come, the motivation for doing the authentication framework as a separate project (Alfredo) was: Other Hadoop related projects can start using it today without having to wait for a Hadoop release It has applicability outside of Hadoop
          Hide
          Alejandro Abdelnur added a comment -

          Andreas,

          Glad to hear that Yahoo finds this feature very useful.

          At the moment I'm finishing a patch for the trunk that enables this functionality.

          Similarly to what you describe, the patch I'm working supports pluggable authentication mechanisms via a simple interface.

          Show
          Alejandro Abdelnur added a comment - Andreas, Glad to hear that Yahoo finds this feature very useful. At the moment I'm finishing a patch for the trunk that enables this functionality. Similarly to what you describe, the patch I'm working supports pluggable authentication mechanisms via a simple interface.
          Hide
          Andreas Neumann added a comment -

          I agree that this would be very useful.
          The Oozie team at Yahoo! is currently working on a patch to support Kerberos/SPNEGO (and an extensible framework to also allow other authentication methods).
          This code can be contributed to Hadoop so that all dependent projects can make use of it.
          Cheers -Andreas.

          Show
          Andreas Neumann added a comment - I agree that this would be very useful. The Oozie team at Yahoo! is currently working on a patch to support Kerberos/SPNEGO (and an extensible framework to also allow other authentication methods). This code can be contributed to Hadoop so that all dependent projects can make use of it. Cheers -Andreas.

            People

            • Assignee:
              Alejandro Abdelnur
              Reporter:
              Alejandro Abdelnur
            • Votes:
              0 Vote for this issue
              Watchers:
              28 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development