Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12082

Support multiple authentication schemes via AuthenticationFilter

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: security
    • Labels:
      None

      Description

      The requirement is to support LDAP based authentication scheme via Hadoop AuthenticationFilter. HADOOP-9054 added a support to plug-in custom authentication scheme (in addition to Kerberos) via AltKerberosAuthenticationHandler class. But it is based on selecting the authentication mechanism based on User-Agent HTTP header which does not conform to HTTP protocol semantics.

      As per RFC-2616

      • HTTP protocol provides a simple challenge-response authentication mechanism that can be used by a server to challenge a client request and by a client to provide the necessary authentication information.
      • This mechanism is initiated by server sending the 401 (Authenticate) response with ‘WWW-Authenticate’ header which includes at least one challenge that indicates the authentication scheme(s) and parameters applicable to the Request-URI.
      • In case server supports multiple authentication schemes, it may return multiple challenges with a 401 (Authenticate) response, and each challenge may use a different auth-scheme.
      • A user agent MUST choose to use the strongest auth-scheme it understands and request credentials from the user based upon that challenge.

      The existing Hadoop authentication filter implementation supports Kerberos authentication scheme and uses ‘Negotiate’ as the challenge as part of ‘WWW-Authenticate’ response header. As per the following documentation, ‘Negotiate’ challenge scheme is only applicable to Kerberos (and Windows NTLM) authentication schemes.
      SPNEGO-based Kerberos and NTLM HTTP Authentication
      Understanding HTTP Authentication

      On the other hand for LDAP authentication, typically ‘Basic’ authentication scheme is used (Note TLS is mandatory with Basic authentication scheme).
      http://httpd.apache.org/docs/trunk/mod/mod_authnz_ldap.html

      Hence for this feature, the idea would be to provide a custom implementation of Hadoop AuthenticationHandler and Authenticator interfaces which would support both schemes - Kerberos (via Negotiate auth challenge) and LDAP (via Basic auth challenge). During the authentication phase, it would send both the challenges and let client pick the appropriate one. If client responds with an ‘Authorization’ header tagged with ‘Negotiate’ - it will use Kerberos authentication. If client responds with an ‘Authorization’ header tagged with ‘Basic’ - it will use LDAP authentication.

      Note - some HTTP clients (e.g. curl or Apache Http Java client) need to be configured to use one scheme over the other e.g.

      Typically web browsers automatically choose an authentication scheme based on a notion of “strength” of security. e.g. take a look at the design of Chrome browser for HTTP authentication

      1. HADOOP-12082-branch-2-003.patch
        81 kB
        Hrishikesh Gadre
      2. HADOOP-12082-branch-2.8-002.patch
        81 kB
        Hrishikesh Gadre
      3. HADOOP-12082-branch-2-002.patch
        81 kB
        Hrishikesh Gadre
      4. HADOOP-12082-branch-2-001.patch
        81 kB
        Hrishikesh Gadre
      5. HADOOP-12082-branch-2.8-001.patch
        81 kB
        Hrishikesh Gadre
      6. HADOOP-12082-branch-2.8.patch
        81 kB
        Hrishikesh Gadre
      7. HADOOP-12082-branch-2.patch
        81 kB
        Hrishikesh Gadre
      8. HADOOP-12082-006.patch
        82 kB
        Hrishikesh Gadre
      9. HADOOP-12082-005.patch
        82 kB
        Hrishikesh Gadre
      10. HADOOP-12082-004.patch
        80 kB
        Hrishikesh Gadre
      11. HADOOP-12082-003.patch
        80 kB
        Hrishikesh Gadre
      12. HADOOP-12082-002.patch
        79 kB
        Hrishikesh Gadre
      13. HADOOP-12082-001.patch
        79 kB
        Hrishikesh Gadre
      14. HADOOP-12082.patch
        80 kB
        Hrishikesh Gadre
      15. hadoop-ldap-auth-v6.patch
        78 kB
        Hrishikesh Gadre
      16. hadoop-ldap-auth-v5.patch
        70 kB
        Hrishikesh Gadre
      17. hadoop-ldap-auth-v4.patch
        59 kB
        Hrishikesh Gadre
      18. hadoop-ldap-auth-v3.patch
        61 kB
        Hrishikesh Gadre
      19. hadoop-ldap-auth-v2.patch
        43 kB
        Hrishikesh Gadre
      20. hadoop-ldap.patch
        14 kB
        Hrishikesh Gadre
      21. multi-scheme-auth-support-poc.patch
        7 kB
        Hrishikesh Gadre

        Issue Links

          Activity

          Hide
          aw Allen Wittenauer added a comment -

          Is this going to get documented?

          Show
          aw Allen Wittenauer added a comment - Is this going to get documented?
          Hide
          benoyantony Benoy Antony added a comment -

          Hrishikesh Gadre, Could you please open a related jira to update the documentation so that users learn how to make use of the new AuthenticationHandlers ?
          It will be very useful.

          Show
          benoyantony Benoy Antony added a comment - Hrishikesh Gadre , Could you please open a related jira to update the documentation so that users learn how to make use of the new AuthenticationHandlers ? It will be very useful.
          Hide
          benoyantony Benoy Antony added a comment -

          Committed to trunk, branch-2 and branch-2.8.
          Thanks for the contribution, Hrishikesh Gadre.

          Show
          benoyantony Benoy Antony added a comment - Committed to trunk, branch-2 and branch-2.8. Thanks for the contribution, Hrishikesh Gadre .
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 1s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 54s Maven dependency ordering for branch
          +1 mvninstall 6m 28s branch-2 passed
          +1 compile 5m 32s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 30s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 26s branch-2 passed
          +1 mvnsite 1m 32s branch-2 passed
          +1 mvneclipse 0m 42s branch-2 passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 9s branch-2 passed
          +1 javadoc 1m 13s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 27s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 13s the patch passed
          +1 compile 5m 32s the patch passed with JDK v1.8.0_101
          +1 javac 5m 32s the patch passed
          +1 compile 6m 35s the patch passed with JDK v1.7.0_111
          +1 javac 6m 35s the patch passed
          +1 checkstyle 1m 32s root: The patch generated 0 new + 151 unchanged - 6 fixed = 151 total (was 157)
          +1 mvnsite 1m 44s the patch passed
          +1 mvneclipse 0m 52s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 41s the patch passed
          +1 javadoc 1m 23s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 38s the patch passed with JDK v1.7.0_111
          +1 unit 0m 16s hadoop-project in the patch passed with JDK v1.7.0_111.
          +1 unit 4m 29s hadoop-auth in the patch passed with JDK v1.7.0_111.
          +1 unit 8m 23s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 29s The patch does not generate ASF License warnings.
          102m 50s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-12082
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834823/HADOOP-12082-branch-2-003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 41c890b5dd8e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 5b7cbb5
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10886/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10886/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 1s The patch appears to include 4 new or modified test files. 0 mvndep 0m 54s Maven dependency ordering for branch +1 mvninstall 6m 28s branch-2 passed +1 compile 5m 32s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 30s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 26s branch-2 passed +1 mvnsite 1m 32s branch-2 passed +1 mvneclipse 0m 42s branch-2 passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 9s branch-2 passed +1 javadoc 1m 13s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 27s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 13s the patch passed +1 compile 5m 32s the patch passed with JDK v1.8.0_101 +1 javac 5m 32s the patch passed +1 compile 6m 35s the patch passed with JDK v1.7.0_111 +1 javac 6m 35s the patch passed +1 checkstyle 1m 32s root: The patch generated 0 new + 151 unchanged - 6 fixed = 151 total (was 157) +1 mvnsite 1m 44s the patch passed +1 mvneclipse 0m 52s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 41s the patch passed +1 javadoc 1m 23s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 38s the patch passed with JDK v1.7.0_111 +1 unit 0m 16s hadoop-project in the patch passed with JDK v1.7.0_111. +1 unit 4m 29s hadoop-auth in the patch passed with JDK v1.7.0_111. +1 unit 8m 23s hadoop-common in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 102m 50s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-12082 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834823/HADOOP-12082-branch-2-003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 41c890b5dd8e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 5b7cbb5 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10886/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10886/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          benoyantony Benoy Antony added a comment - - edited

          I agree Hrishikesh Gadre. I started a ReBuild just to be on the safe side.

          Show
          benoyantony Benoy Antony added a comment - - edited I agree Hrishikesh Gadre . I started a ReBuild just to be on the safe side.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Benoy Antony The unit test failure and whitespace related error are unrelated to my patch. Please let me know if anything required from my end.

          Show
          hgadre Hrishikesh Gadre added a comment - Benoy Antony The unit test failure and whitespace related error are unrelated to my patch. Please let me know if anything required from my end.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 58s Maven dependency ordering for branch
          +1 mvninstall 6m 34s branch-2 passed
          +1 compile 5m 37s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 31s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 27s branch-2 passed
          +1 mvnsite 1m 32s branch-2 passed
          +1 mvneclipse 0m 41s branch-2 passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 11s branch-2 passed
          +1 javadoc 1m 14s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 26s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 13s the patch passed
          +1 compile 5m 29s the patch passed with JDK v1.8.0_101
          +1 javac 5m 29s the patch passed
          +1 compile 6m 34s the patch passed with JDK v1.7.0_111
          +1 javac 6m 34s the patch passed
          +1 checkstyle 1m 29s root: The patch generated 0 new + 151 unchanged - 6 fixed = 151 total (was 157)
          +1 mvnsite 1m 42s the patch passed
          +1 mvneclipse 0m 53s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 39s the patch passed
          +1 javadoc 1m 23s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 38s the patch passed with JDK v1.7.0_111
          +1 unit 0m 15s hadoop-project in the patch passed with JDK v1.7.0_111.
          +1 unit 4m 24s hadoop-auth in the patch passed with JDK v1.7.0_111.
          -1 unit 8m 44s hadoop-common in the patch failed with JDK v1.7.0_111.
          +1 asflicense 0m 27s The patch does not generate ASF License warnings.
          112m 34s



          Reason Tests
          JDK v1.8.0_101 Failed junit tests hadoop.net.TestDNS
          JDK v1.8.0_101 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle
          JDK v1.7.0_111 Failed junit tests hadoop.net.TestDNS



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-12082
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834823/HADOOP-12082-branch-2-003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux e92c56346421 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 0205ad5
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10860/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10860/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_111.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10860/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10860/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 58s Maven dependency ordering for branch +1 mvninstall 6m 34s branch-2 passed +1 compile 5m 37s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 31s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 27s branch-2 passed +1 mvnsite 1m 32s branch-2 passed +1 mvneclipse 0m 41s branch-2 passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 11s branch-2 passed +1 javadoc 1m 14s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 26s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 13s the patch passed +1 compile 5m 29s the patch passed with JDK v1.8.0_101 +1 javac 5m 29s the patch passed +1 compile 6m 34s the patch passed with JDK v1.7.0_111 +1 javac 6m 34s the patch passed +1 checkstyle 1m 29s root: The patch generated 0 new + 151 unchanged - 6 fixed = 151 total (was 157) +1 mvnsite 1m 42s the patch passed +1 mvneclipse 0m 53s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 39s the patch passed +1 javadoc 1m 23s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 38s the patch passed with JDK v1.7.0_111 +1 unit 0m 15s hadoop-project in the patch passed with JDK v1.7.0_111. +1 unit 4m 24s hadoop-auth in the patch passed with JDK v1.7.0_111. -1 unit 8m 44s hadoop-common in the patch failed with JDK v1.7.0_111. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 112m 34s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.net.TestDNS JDK v1.8.0_101 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle JDK v1.7.0_111 Failed junit tests hadoop.net.TestDNS Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-12082 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834823/HADOOP-12082-branch-2-003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux e92c56346421 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 0205ad5 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10860/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10860/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10860/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10860/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Benoy Antony Here is the patch against branch-2

          Show
          hgadre Hrishikesh Gadre added a comment - Benoy Antony Here is the patch against branch-2
          Hide
          benoyantony Benoy Antony added a comment - - edited

          Yes, multi-scheme and ldap can be plugged in like any other AuthenticationHandler. It will be a good idea to update the relevant documentation. But let us do it in a different jira.

          Show
          benoyantony Benoy Antony added a comment - - edited Yes, multi-scheme and ldap can be plugged in like any other AuthenticationHandler. It will be a good idea to update the relevant documentation. But let us do it in a different jira.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Yuanbo Liu

          I guess it's hard for NameNode or ResourceManager's http server to take advantage of your work, since http server is a thread in NameNode or ResourceManager and the webapp is packaged into jar. It's not able to change web.xml unless the jar is replaced.

          I don't think so. The httpfs-site.xml file contains the configuration options for the authentication filter. Please refer to following links

          https://hadoop.apache.org/docs/r2.6.4/hadoop-hdfs-httpfs/ServerSetup.html
          https://hadoop.apache.org/docs/r2.6.4/hadoop-hdfs-httpfs/httpfs-default.html

          Specifically refer to following params for kerberos authentication,

          httpfs.authentication.type
          httpfs.authentication.kerberos.principal
          httpfs.authentication.kerberos.keytab

          As per this doc, the valid values for "httpfs.authentication.type" are simple OR kerberos. May be we need update the code to consider "ldap" and "multi-scheme" as valid values as well. The similar changes would probably work for ResourceManager's HTTP server. Benoy Antony what do you think?

          Show
          hgadre Hrishikesh Gadre added a comment - Yuanbo Liu I guess it's hard for NameNode or ResourceManager's http server to take advantage of your work, since http server is a thread in NameNode or ResourceManager and the webapp is packaged into jar. It's not able to change web.xml unless the jar is replaced. I don't think so. The httpfs-site.xml file contains the configuration options for the authentication filter. Please refer to following links https://hadoop.apache.org/docs/r2.6.4/hadoop-hdfs-httpfs/ServerSetup.html https://hadoop.apache.org/docs/r2.6.4/hadoop-hdfs-httpfs/httpfs-default.html Specifically refer to following params for kerberos authentication, httpfs.authentication.type httpfs.authentication.kerberos.principal httpfs.authentication.kerberos.keytab As per this doc, the valid values for "httpfs.authentication.type" are simple OR kerberos . May be we need update the code to consider "ldap" and "multi-scheme" as valid values as well. The similar changes would probably work for ResourceManager's HTTP server. Benoy Antony what do you think?
          Hide
          yuanbo Yuanbo Liu added a comment -

          Hrishikesh Gadre Thanks for your response.

          The jira addresses the requirement where more tha....

          Now I understand what issue this JIRA address. The delegation filter which I'm looking for is similar with your idea, the first auth is SPENGO auth, and the second is proxy auth.

          The authentication handler is configured as part of configuring Hadoop AuthenticationFilter. This is typically done via web.xml....

          I have went though Oozie configuration, also Configuration.md again. I guess it's hard for NameNode or ResourceManager's http server to take advantage of your work, since http server is a thread in NameNode or ResourceManager and the webapp is packaged into jar. It's not able to change web.xml unless the jar is replaced.
          So I think it's designed for third-party projects which depend on Hadoop-Auth, right?

          Show
          yuanbo Yuanbo Liu added a comment - Hrishikesh Gadre Thanks for your response. The jira addresses the requirement where more tha.... Now I understand what issue this JIRA address. The delegation filter which I'm looking for is similar with your idea, the first auth is SPENGO auth, and the second is proxy auth. The authentication handler is configured as part of configuring Hadoop AuthenticationFilter. This is typically done via web.xml.... I have went though Oozie configuration, also Configuration.md again. I guess it's hard for NameNode or ResourceManager's http server to take advantage of your work, since http server is a thread in NameNode or ResourceManager and the webapp is packaged into jar. It's not able to change web.xml unless the jar is replaced. So I think it's designed for third-party projects which depend on Hadoop-Auth, right?
          Hide
          benoyantony Benoy Antony added a comment -

          committed to branch-2.8. Thanks for the effort and contribution Hrishikesh Gadre . Could you please provide a patch for the branch-2 as well ?
          I cannot cherry-pick from branch-2.8 to branch-2 .

          Show
          benoyantony Benoy Antony added a comment - committed to branch-2.8. Thanks for the effort and contribution Hrishikesh Gadre . Could you please provide a patch for the branch-2 as well ? I cannot cherry-pick from branch-2.8 to branch-2 .
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Benoy Antony What would recommend regarding the error related to whitespaces? This file is not part of my patch...

          Show
          hgadre Hrishikesh Gadre added a comment - Benoy Antony What would recommend regarding the error related to whitespaces? This file is not part of my patch...
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 3m 8s Maven dependency ordering for branch
          +1 mvninstall 7m 7s branch-2.8 passed
          +1 compile 6m 42s branch-2.8 passed with JDK v1.8.0_101
          +1 compile 7m 11s branch-2.8 passed with JDK v1.7.0_111
          +1 checkstyle 1m 11s branch-2.8 passed
          +1 mvnsite 1m 37s branch-2.8 passed
          +1 mvneclipse 0m 42s branch-2.8 passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 18s branch-2.8 passed
          +1 javadoc 1m 25s branch-2.8 passed with JDK v1.8.0_101
          +1 javadoc 1m 33s branch-2.8 passed with JDK v1.7.0_111
          0 mvndep 0m 27s Maven dependency ordering for patch
          +1 mvninstall 1m 15s the patch passed
          +1 compile 7m 3s the patch passed with JDK v1.8.0_101
          +1 javac 7m 3s the patch passed
          +1 compile 7m 45s the patch passed with JDK v1.7.0_111
          +1 javac 7m 45s the patch passed
          +1 checkstyle 1m 20s root: The patch generated 0 new + 138 unchanged - 6 fixed = 138 total (was 144)
          +1 mvnsite 1m 49s the patch passed
          +1 mvneclipse 0m 52s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 3m 1s the patch passed
          +1 javadoc 1m 32s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 41s the patch passed with JDK v1.7.0_111
          +1 unit 0m 15s hadoop-project in the patch passed with JDK v1.7.0_111.
          +1 unit 4m 27s hadoop-auth in the patch passed with JDK v1.7.0_111.
          +1 unit 8m 4s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 29s The patch does not generate ASF License warnings.
          110m 51s



          Reason Tests
          JDK v1.8.0_101 Failed junit tests hadoop.ipc.TestIPC



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:5af2af1
          JIRA Issue HADOOP-12082
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834770/HADOOP-12082-branch-2.8-002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux c6f86d69b9d5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.8 / b823647
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10858/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10858/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10858/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 3m 8s Maven dependency ordering for branch +1 mvninstall 7m 7s branch-2.8 passed +1 compile 6m 42s branch-2.8 passed with JDK v1.8.0_101 +1 compile 7m 11s branch-2.8 passed with JDK v1.7.0_111 +1 checkstyle 1m 11s branch-2.8 passed +1 mvnsite 1m 37s branch-2.8 passed +1 mvneclipse 0m 42s branch-2.8 passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 18s branch-2.8 passed +1 javadoc 1m 25s branch-2.8 passed with JDK v1.8.0_101 +1 javadoc 1m 33s branch-2.8 passed with JDK v1.7.0_111 0 mvndep 0m 27s Maven dependency ordering for patch +1 mvninstall 1m 15s the patch passed +1 compile 7m 3s the patch passed with JDK v1.8.0_101 +1 javac 7m 3s the patch passed +1 compile 7m 45s the patch passed with JDK v1.7.0_111 +1 javac 7m 45s the patch passed +1 checkstyle 1m 20s root: The patch generated 0 new + 138 unchanged - 6 fixed = 138 total (was 144) +1 mvnsite 1m 49s the patch passed +1 mvneclipse 0m 52s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 3m 1s the patch passed +1 javadoc 1m 32s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 41s the patch passed with JDK v1.7.0_111 +1 unit 0m 15s hadoop-project in the patch passed with JDK v1.7.0_111. +1 unit 4m 27s hadoop-auth in the patch passed with JDK v1.7.0_111. +1 unit 8m 4s hadoop-common in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 110m 51s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.ipc.TestIPC Subsystem Report/Notes Docker Image:yetus/hadoop:5af2af1 JIRA Issue HADOOP-12082 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834770/HADOOP-12082-branch-2.8-002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux c6f86d69b9d5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.8 / b823647 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10858/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10858/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10858/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hgadre Hrishikesh Gadre added a comment - - edited

          Yuanbo Liu

          When you said LDAP based authentication, did you mean authentication filter which supports delegation?

          The jira addresses the requirement where more than one authentication mode need to be configured e.g. consider a case where some of the users can not be authenticated using kerberos since they reside outside the kerberos domain. So with this mechanism, you can configure two authentication schemes (a) SPNEGO scheme using kerberos auth and (b) BASIC scheme using LDAP auth. Hence the users which are outside kerberos domain can provide username/password to authenticate against LDAP (or Active Directory) server.
          Also the underlying framework is extensible so you can implement different mechanisms as well e.g. instead of LDAP auth - you can use a database backend for storing the credentials. For this - you will need to provide a different authentication handler implementation using LdapAuthenticationHandler.java as a reference.

          BTW HADOOP-9054 also targeted the same use-case, although that implementation was not conforming the HTTP protocol semantics. More explanation is available in the jira description.

          I was anticipating there were some descriptions about configuration work of core-site/hdfs-site, but there weren't.

          The authentication handler is configured as part of configuring Hadoop AuthenticationFilter. This is typically done via web.xml (or other web container/application specific means). e.g. following link provide steps to configure authentication handler for Oozie

          https://oozie.apache.org/docs/4.2.0/ENG_Custom_Authentication.html#Provide_Custom_Authentication_to_Oozie_Server

          Show
          hgadre Hrishikesh Gadre added a comment - - edited Yuanbo Liu When you said LDAP based authentication, did you mean authentication filter which supports delegation? The jira addresses the requirement where more than one authentication mode need to be configured e.g. consider a case where some of the users can not be authenticated using kerberos since they reside outside the kerberos domain. So with this mechanism, you can configure two authentication schemes (a) SPNEGO scheme using kerberos auth and (b) BASIC scheme using LDAP auth. Hence the users which are outside kerberos domain can provide username/password to authenticate against LDAP (or Active Directory) server. Also the underlying framework is extensible so you can implement different mechanisms as well e.g. instead of LDAP auth - you can use a database backend for storing the credentials. For this - you will need to provide a different authentication handler implementation using LdapAuthenticationHandler.java as a reference. BTW HADOOP-9054 also targeted the same use-case, although that implementation was not conforming the HTTP protocol semantics. More explanation is available in the jira description. I was anticipating there were some descriptions about configuration work of core-site/hdfs-site, but there weren't. The authentication handler is configured as part of configuring Hadoop AuthenticationFilter. This is typically done via web.xml (or other web container/application specific means). e.g. following link provide steps to configure authentication handler for Oozie https://oozie.apache.org/docs/4.2.0/ENG_Custom_Authentication.html#Provide_Custom_Authentication_to_Oozie_Server
          Hide
          yuanbo Yuanbo Liu added a comment - - edited

          Hrishikesh Gadre Thanks for your work.
          It seems a long time JIRA, and I doesn't catch up much context of this issue.
          When you said LDAP based authentication, did you mean authentication filter which supports delegation?
          If so, I'm looking forwards to your work, because it would help some proxy servers such as Knox to deal with more http requests which require proxy user.

          I'm also confused about Configuration.md. I was anticipating there were some descriptions about configuration work of core-site/hdfs-site, but there weren't. Could you elaborate how to configure a real Hadoop cluster so that users can use your new handlers LdapAuthenticationHandler, MultiSchemeAuthenticationHandler. I can't get the steps from test cases.

          Thanks again for your time, please let me know your thoughts.

          Show
          yuanbo Yuanbo Liu added a comment - - edited Hrishikesh Gadre Thanks for your work. It seems a long time JIRA, and I doesn't catch up much context of this issue. When you said LDAP based authentication, did you mean authentication filter which supports delegation? If so, I'm looking forwards to your work, because it would help some proxy servers such as Knox to deal with more http requests which require proxy user. I'm also confused about Configuration.md. I was anticipating there were some descriptions about configuration work of core-site/hdfs-site, but there weren't. Could you elaborate how to configure a real Hadoop cluster so that users can use your new handlers LdapAuthenticationHandler , MultiSchemeAuthenticationHandler . I can't get the steps from test cases. Thanks again for your time, please let me know your thoughts.
          Hide
          andrew.wang Andrew Wang added a comment -

          No real reason, I just backported it for you.

          Show
          andrew.wang Andrew Wang added a comment - No real reason, I just backported it for you.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Andrew Wang Is there a reason why we have not backported HADOOP-12859 to branch-2.8 ? This is causing problems in backporting this patch to branch-2.8

          Show
          hgadre Hrishikesh Gadre added a comment - Andrew Wang Is there a reason why we have not backported HADOOP-12859 to branch-2.8 ? This is causing problems in backporting this patch to branch-2.8
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Yes I just changed the method param name. The method name is still same.

          Show
          hgadre Hrishikesh Gadre added a comment - Yes I just changed the method param name. The method name is still same.
          Hide
          benoyantony Benoy Antony added a comment -

          Let's keep the same method names in all branches.

          Show
          benoyantony Benoy Antony added a comment - Let's keep the same method names in all branches.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Benoy Antony

          Any reason to not use primitive for enableStartTls and disableHostNameVerification ? I think , that will fix the checkstyle errors.

          No specific reason. BTW jenkins didn't report any checkstyle warnings for the patch against trunk (which had the identical code). I have resubmitted the patch by renaming the method parameter. Let's see if it fixes the issue.

          Show
          hgadre Hrishikesh Gadre added a comment - Benoy Antony Any reason to not use primitive for enableStartTls and disableHostNameVerification ? I think , that will fix the checkstyle errors. No specific reason. BTW jenkins didn't report any checkstyle warnings for the patch against trunk (which had the identical code). I have resubmitted the patch by renaming the method parameter. Let's see if it fixes the issue.
          Hide
          benoyantony Benoy Antony added a comment - - edited

          Hrishikesh Gadre, Any reason to not use primitive for enableStartTls and disableHostNameVerification ?
          I think , that will fix the checkstyle errors.

          The whitespace issues seem to occur on file that's not modified by your patch.

          Show
          benoyantony Benoy Antony added a comment - - edited Hrishikesh Gadre , Any reason to not use primitive for enableStartTls and disableHostNameVerification ? I think , that will fix the checkstyle errors. The whitespace issues seem to occur on file that's not modified by your patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 7m 2s branch-2 passed
          +1 compile 6m 9s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 38s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 32s branch-2 passed
          +1 mvnsite 1m 34s branch-2 passed
          +1 mvneclipse 0m 41s branch-2 passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 13s branch-2 passed
          +1 javadoc 1m 19s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 27s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 26s Maven dependency ordering for patch
          +1 mvninstall 1m 12s the patch passed
          +1 compile 6m 8s the patch passed with JDK v1.8.0_101
          +1 javac 6m 8s the patch passed
          +1 compile 6m 43s the patch passed with JDK v1.7.0_111
          +1 javac 6m 43s the patch passed
          +1 checkstyle 1m 30s root: The patch generated 0 new + 151 unchanged - 6 fixed = 151 total (was 157)
          +1 mvnsite 1m 44s the patch passed
          +1 mvneclipse 0m 54s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 48s the patch passed
          +1 javadoc 1m 28s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 37s the patch passed with JDK v1.7.0_111
          +1 unit 0m 15s hadoop-project in the patch passed with JDK v1.7.0_111.
          +1 unit 4m 23s hadoop-auth in the patch passed with JDK v1.7.0_111.
          +1 unit 11m 0s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 29s The patch does not generate ASF License warnings.
          109m 58s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-12082
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834301/HADOOP-12082-branch-2-001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 9c99b6349721 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / a3cbaf0
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10837/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10837/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10837/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 7m 2s branch-2 passed +1 compile 6m 9s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 38s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 32s branch-2 passed +1 mvnsite 1m 34s branch-2 passed +1 mvneclipse 0m 41s branch-2 passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 13s branch-2 passed +1 javadoc 1m 19s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 27s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 26s Maven dependency ordering for patch +1 mvninstall 1m 12s the patch passed +1 compile 6m 8s the patch passed with JDK v1.8.0_101 +1 javac 6m 8s the patch passed +1 compile 6m 43s the patch passed with JDK v1.7.0_111 +1 javac 6m 43s the patch passed +1 checkstyle 1m 30s root: The patch generated 0 new + 151 unchanged - 6 fixed = 151 total (was 157) +1 mvnsite 1m 44s the patch passed +1 mvneclipse 0m 54s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 48s the patch passed +1 javadoc 1m 28s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 37s the patch passed with JDK v1.7.0_111 +1 unit 0m 15s hadoop-project in the patch passed with JDK v1.7.0_111. +1 unit 4m 23s hadoop-auth in the patch passed with JDK v1.7.0_111. +1 unit 11m 0s hadoop-common in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 109m 58s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-12082 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834301/HADOOP-12082-branch-2-001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 9c99b6349721 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / a3cbaf0 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10837/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10837/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10837/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 43s Maven dependency ordering for branch
          +1 mvninstall 6m 30s branch-2.8 passed
          +1 compile 6m 13s branch-2.8 passed with JDK v1.8.0_101
          +1 compile 7m 17s branch-2.8 passed with JDK v1.7.0_111
          +1 checkstyle 1m 10s branch-2.8 passed
          +1 mvnsite 1m 39s branch-2.8 passed
          +1 mvneclipse 0m 41s branch-2.8 passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 10s branch-2.8 passed
          +1 javadoc 1m 15s branch-2.8 passed with JDK v1.8.0_101
          +1 javadoc 1m 28s branch-2.8 passed with JDK v1.7.0_111
          0 mvndep 0m 28s Maven dependency ordering for patch
          +1 mvninstall 1m 13s the patch passed
          +1 compile 6m 21s the patch passed with JDK v1.8.0_101
          +1 javac 6m 21s the patch passed
          +1 compile 7m 28s the patch passed with JDK v1.7.0_111
          +1 javac 7m 28s the patch passed
          -0 checkstyle 1m 11s root: The patch generated 2 new + 138 unchanged - 5 fixed = 140 total (was 143)
          +1 mvnsite 1m 43s the patch passed
          +1 mvneclipse 0m 51s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 40s the patch passed
          +1 javadoc 1m 24s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 40s the patch passed with JDK v1.7.0_111
          +1 unit 0m 12s hadoop-project in the patch passed with JDK v1.7.0_111.
          +1 unit 4m 16s hadoop-auth in the patch passed with JDK v1.7.0_111.
          +1 unit 8m 44s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          114m 42s



          Reason Tests
          JDK v1.8.0_101 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:5af2af1
          JIRA Issue HADOOP-12082
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834299/HADOOP-12082-branch-2.8-001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 0028420b96fa 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.8 / e9592f1
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10836/artifact/patchprocess/diff-checkstyle-root.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10836/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10836/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10836/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 43s Maven dependency ordering for branch +1 mvninstall 6m 30s branch-2.8 passed +1 compile 6m 13s branch-2.8 passed with JDK v1.8.0_101 +1 compile 7m 17s branch-2.8 passed with JDK v1.7.0_111 +1 checkstyle 1m 10s branch-2.8 passed +1 mvnsite 1m 39s branch-2.8 passed +1 mvneclipse 0m 41s branch-2.8 passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 10s branch-2.8 passed +1 javadoc 1m 15s branch-2.8 passed with JDK v1.8.0_101 +1 javadoc 1m 28s branch-2.8 passed with JDK v1.7.0_111 0 mvndep 0m 28s Maven dependency ordering for patch +1 mvninstall 1m 13s the patch passed +1 compile 6m 21s the patch passed with JDK v1.8.0_101 +1 javac 6m 21s the patch passed +1 compile 7m 28s the patch passed with JDK v1.7.0_111 +1 javac 7m 28s the patch passed -0 checkstyle 1m 11s root: The patch generated 2 new + 138 unchanged - 5 fixed = 140 total (was 143) +1 mvnsite 1m 43s the patch passed +1 mvneclipse 0m 51s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 40s the patch passed +1 javadoc 1m 24s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 40s the patch passed with JDK v1.7.0_111 +1 unit 0m 12s hadoop-project in the patch passed with JDK v1.7.0_111. +1 unit 4m 16s hadoop-auth in the patch passed with JDK v1.7.0_111. +1 unit 8m 44s hadoop-common in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 114m 42s Reason Tests JDK v1.8.0_101 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:5af2af1 JIRA Issue HADOOP-12082 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834299/HADOOP-12082-branch-2.8-001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 0028420b96fa 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.8 / e9592f1 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10836/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10836/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10836/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10836/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Here is a patch against branch-2 fixing the whitespace errors.

          Show
          hgadre Hrishikesh Gadre added a comment - Here is a patch against branch-2 fixing the whitespace errors.
          Hide
          hgadre Hrishikesh Gadre added a comment - - edited

          Benoy Antony Here is the updated patch against branch-2.8 which fixes the checkstyle errors.

          Show
          hgadre Hrishikesh Gadre added a comment - - edited Benoy Antony Here is the updated patch against branch-2.8 which fixes the checkstyle errors.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Benoy Antony Regarding patch for branch-2.8,

          Following checkstyle errors seem bogus to me. Also these were not reported for the patch I submitted against trunk.

          ./hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/LdapAuthenticationHandler.java:118: public void setEnableStartTls(Boolean enableStartTls) {:41: 'enableStartTls' hides a field.
          ./hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/LdapAuthenticationHandler.java:131: Boolean disableHostNameVerification) {:15: 'disableHostNameVerification' hides a field.

          Also the following whitespace errors reported are for a file which I can't find in branch-2.8

          https://builds.apache.org/job/PreCommit-HADOOP-Build/10830/artifact/patchprocess/whitespace-eol.txt

          Show
          hgadre Hrishikesh Gadre added a comment - Benoy Antony Regarding patch for branch-2.8, Following checkstyle errors seem bogus to me. Also these were not reported for the patch I submitted against trunk. ./hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/LdapAuthenticationHandler.java:118: public void setEnableStartTls(Boolean enableStartTls) {:41: 'enableStartTls' hides a field. ./hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/LdapAuthenticationHandler.java:131: Boolean disableHostNameVerification) {:15: 'disableHostNameVerification' hides a field. Also the following whitespace errors reported are for a file which I can't find in branch-2.8 https://builds.apache.org/job/PreCommit-HADOOP-Build/10830/artifact/patchprocess/whitespace-eol.txt
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 24s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 47s Maven dependency ordering for branch
          +1 mvninstall 7m 55s branch-2.8 passed
          +1 compile 7m 41s branch-2.8 passed with JDK v1.8.0_101
          +1 compile 8m 17s branch-2.8 passed with JDK v1.7.0_111
          +1 checkstyle 1m 14s branch-2.8 passed
          +1 mvnsite 1m 39s branch-2.8 passed
          +1 mvneclipse 0m 48s branch-2.8 passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 39s branch-2.8 passed
          +1 javadoc 1m 31s branch-2.8 passed with JDK v1.8.0_101
          +1 javadoc 1m 35s branch-2.8 passed with JDK v1.7.0_111
          0 mvndep 0m 30s Maven dependency ordering for patch
          +1 mvninstall 1m 23s the patch passed
          +1 compile 7m 29s the patch passed with JDK v1.8.0_101
          +1 javac 7m 29s the patch passed
          +1 compile 8m 14s the patch passed with JDK v1.7.0_111
          +1 javac 8m 14s the patch passed
          -0 checkstyle 1m 20s root: The patch generated 4 new + 138 unchanged - 5 fixed = 142 total (was 143)
          +1 mvnsite 1m 58s the patch passed
          +1 mvneclipse 0m 54s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 41s the patch passed
          +1 javadoc 1m 23s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 38s the patch passed with JDK v1.7.0_111
          +1 unit 0m 16s hadoop-project in the patch passed with JDK v1.7.0_111.
          +1 unit 4m 29s hadoop-auth in the patch passed with JDK v1.7.0_111.
          +1 unit 8m 20s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 29s The patch does not generate ASF License warnings.
          113m 21s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:5af2af1
          JIRA Issue HADOOP-12082
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834262/HADOOP-12082-branch-2.8.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 4396d7d1e957 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.8 / 34023ca
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10830/artifact/patchprocess/diff-checkstyle-root.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10830/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10830/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10830/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 24s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 47s Maven dependency ordering for branch +1 mvninstall 7m 55s branch-2.8 passed +1 compile 7m 41s branch-2.8 passed with JDK v1.8.0_101 +1 compile 8m 17s branch-2.8 passed with JDK v1.7.0_111 +1 checkstyle 1m 14s branch-2.8 passed +1 mvnsite 1m 39s branch-2.8 passed +1 mvneclipse 0m 48s branch-2.8 passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 39s branch-2.8 passed +1 javadoc 1m 31s branch-2.8 passed with JDK v1.8.0_101 +1 javadoc 1m 35s branch-2.8 passed with JDK v1.7.0_111 0 mvndep 0m 30s Maven dependency ordering for patch +1 mvninstall 1m 23s the patch passed +1 compile 7m 29s the patch passed with JDK v1.8.0_101 +1 javac 7m 29s the patch passed +1 compile 8m 14s the patch passed with JDK v1.7.0_111 +1 javac 8m 14s the patch passed -0 checkstyle 1m 20s root: The patch generated 4 new + 138 unchanged - 5 fixed = 142 total (was 143) +1 mvnsite 1m 58s the patch passed +1 mvneclipse 0m 54s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 41s the patch passed +1 javadoc 1m 23s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 38s the patch passed with JDK v1.7.0_111 +1 unit 0m 16s hadoop-project in the patch passed with JDK v1.7.0_111. +1 unit 4m 29s hadoop-auth in the patch passed with JDK v1.7.0_111. +1 unit 8m 20s hadoop-common in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 113m 21s Subsystem Report/Notes Docker Image:yetus/hadoop:5af2af1 JIRA Issue HADOOP-12082 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834262/HADOOP-12082-branch-2.8.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 4396d7d1e957 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.8 / 34023ca Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10830/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10830/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10830/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10830/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 13m 54s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 57s Maven dependency ordering for branch
          +1 mvninstall 6m 41s branch-2 passed
          +1 compile 5m 41s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 27s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 27s branch-2 passed
          +1 mvnsite 1m 31s branch-2 passed
          +1 mvneclipse 0m 43s branch-2 passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 7s branch-2 passed
          +1 javadoc 1m 11s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 25s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 28s Maven dependency ordering for patch
          +1 mvninstall 1m 10s the patch passed
          +1 compile 5m 26s the patch passed with JDK v1.8.0_101
          +1 javac 5m 26s the patch passed
          +1 compile 6m 33s the patch passed with JDK v1.7.0_111
          +1 javac 6m 33s the patch passed
          +1 checkstyle 1m 31s root: The patch generated 0 new + 151 unchanged - 6 fixed = 151 total (was 157)
          +1 mvnsite 1m 39s the patch passed
          +1 mvneclipse 0m 51s the patch passed
          -1 whitespace 0m 0s The patch has 48 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 38s the patch passed
          +1 javadoc 1m 24s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 37s the patch passed with JDK v1.7.0_111
          +1 unit 0m 14s hadoop-project in the patch passed with JDK v1.7.0_111.
          +1 unit 4m 23s hadoop-auth in the patch passed with JDK v1.7.0_111.
          +1 unit 8m 8s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          115m 10s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-12082
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834250/HADOOP-12082-branch-2.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux cada0c005c19 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / ad7d3c4
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10829/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10829/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10829/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 13m 54s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 57s Maven dependency ordering for branch +1 mvninstall 6m 41s branch-2 passed +1 compile 5m 41s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 27s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 27s branch-2 passed +1 mvnsite 1m 31s branch-2 passed +1 mvneclipse 0m 43s branch-2 passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 7s branch-2 passed +1 javadoc 1m 11s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 25s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 28s Maven dependency ordering for patch +1 mvninstall 1m 10s the patch passed +1 compile 5m 26s the patch passed with JDK v1.8.0_101 +1 javac 5m 26s the patch passed +1 compile 6m 33s the patch passed with JDK v1.7.0_111 +1 javac 6m 33s the patch passed +1 checkstyle 1m 31s root: The patch generated 0 new + 151 unchanged - 6 fixed = 151 total (was 157) +1 mvnsite 1m 39s the patch passed +1 mvneclipse 0m 51s the patch passed -1 whitespace 0m 0s The patch has 48 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 38s the patch passed +1 javadoc 1m 24s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 37s the patch passed with JDK v1.7.0_111 +1 unit 0m 14s hadoop-project in the patch passed with JDK v1.7.0_111. +1 unit 4m 23s hadoop-auth in the patch passed with JDK v1.7.0_111. +1 unit 8m 8s hadoop-common in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 115m 10s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-12082 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834250/HADOOP-12082-branch-2.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux cada0c005c19 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / ad7d3c4 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10829/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10829/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10829/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Here is the patch for branch-2.8. This is identical to branch-2 except a small change in the import statements in DelegationTokenAuthenticationFilter.java.

          Show
          hgadre Hrishikesh Gadre added a comment - Here is the patch for branch-2.8. This is identical to branch-2 except a small change in the import statements in DelegationTokenAuthenticationFilter.java.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Here is the patch for branch-2. The main difference is that we need different dependencies for ApacheDS libraries as compared to trunk (since fix for HADOOP-12911 is not available in branch-2).

          Show
          hgadre Hrishikesh Gadre added a comment - Here is the patch for branch-2. The main difference is that we need different dependencies for ApacheDS libraries as compared to trunk (since fix for HADOOP-12911 is not available in branch-2).
          Hide
          benoyantony Benoy Antony added a comment - - edited

          Committed to trunk. Thanks for the contribution Hrishikesh Gadre.
          Could you please upload the patches for branch-2 and branch-2.8 ? There is conflict on the pom. Once uploaded, I will commit them also.

          Show
          benoyantony Benoy Antony added a comment - - edited Committed to trunk. Thanks for the contribution Hrishikesh Gadre . Could you please upload the patches for branch-2 and branch-2.8 ? There is conflict on the pom. Once uploaded, I will commit them also.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10636 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10636/)
          HADOOP-12082 Support multiple authentication schemes via (benoy: rev 4bca385241c0fc8ff168c7b0f2984a7aed2c7492)

          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationHandler.java
          • (add) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/package-info.java
          • (add) hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/server/TestLdapAuthenticationHandler.java
          • (add) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandlerUtil.java
          • (edit) hadoop-project/pom.xml
          • (edit) hadoop-common-project/hadoop-auth/pom.xml
          • (add) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/MultiSchemeAuthenticationHandler.java
          • (add) hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/server/TestMultiSchemeAuthenticationHandler.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationFilter.java
          • (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/MultiSchemeDelegationTokenAuthenticationHandler.java
          • (edit) hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/TestKerberosAuthenticator.java
          • (edit) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/client/KerberosAuthenticator.java
          • (edit) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java
          • (add) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/HttpConstants.java
          • (add) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/LdapAuthenticationHandler.java
          • (add) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/CompositeAuthenticationHandler.java
          • (edit) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationFilter.java
          • (edit) hadoop-common-project/hadoop-auth/src/site/markdown/Configuration.md
          • (add) hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/server/LdapConstants.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10636 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10636/ ) HADOOP-12082 Support multiple authentication schemes via (benoy: rev 4bca385241c0fc8ff168c7b0f2984a7aed2c7492) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationHandler.java (add) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/package-info.java (add) hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/server/TestLdapAuthenticationHandler.java (add) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandlerUtil.java (edit) hadoop-project/pom.xml (edit) hadoop-common-project/hadoop-auth/pom.xml (add) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/MultiSchemeAuthenticationHandler.java (add) hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/server/TestMultiSchemeAuthenticationHandler.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationFilter.java (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/MultiSchemeDelegationTokenAuthenticationHandler.java (edit) hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/TestKerberosAuthenticator.java (edit) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/client/KerberosAuthenticator.java (edit) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java (add) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/HttpConstants.java (add) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/LdapAuthenticationHandler.java (add) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/CompositeAuthenticationHandler.java (edit) hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationFilter.java (edit) hadoop-common-project/hadoop-auth/src/site/markdown/Configuration.md (add) hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/server/LdapConstants.java
          Hide
          benoyantony Benoy Antony added a comment -

          +1
          Will commit tomorrow if there are no other comments.

          Show
          benoyantony Benoy Antony added a comment - +1 Will commit tomorrow if there are no other comments.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Benoy Antony The patch is looking good now. Can you please take a look?

          Show
          hgadre Hrishikesh Gadre added a comment - Benoy Antony The patch is looking good now. Can you please take a look?
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 19s Maven dependency ordering for branch
          +1 mvninstall 9m 30s trunk passed
          +1 compile 8m 10s trunk passed
          +1 checkstyle 1m 28s trunk passed
          +1 mvnsite 1m 24s trunk passed
          +1 mvneclipse 0m 36s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 1m 41s trunk passed
          +1 javadoc 1m 9s trunk passed
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 4s the patch passed
          +1 compile 6m 48s the patch passed
          +1 javac 6m 48s the patch passed
          +1 checkstyle 1m 36s root: The patch generated 0 new + 151 unchanged - 6 fixed = 151 total (was 157)
          +1 mvnsite 1m 47s the patch passed
          +1 mvneclipse 0m 48s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 4s the patch passed
          +1 javadoc 1m 26s the patch passed
          +1 unit 0m 13s hadoop-project in the patch passed.
          +1 unit 3m 40s hadoop-auth in the patch passed.
          +1 unit 9m 3s hadoop-common in the patch passed.
          +1 asflicense 0m 26s The patch does not generate ASF License warnings.
          77m 7s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HADOOP-12082
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833808/HADOOP-12082-006.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 1c435914cc4f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / b671ee6
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10814/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10814/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 19s Maven dependency ordering for branch +1 mvninstall 9m 30s trunk passed +1 compile 8m 10s trunk passed +1 checkstyle 1m 28s trunk passed +1 mvnsite 1m 24s trunk passed +1 mvneclipse 0m 36s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 1m 41s trunk passed +1 javadoc 1m 9s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 4s the patch passed +1 compile 6m 48s the patch passed +1 javac 6m 48s the patch passed +1 checkstyle 1m 36s root: The patch generated 0 new + 151 unchanged - 6 fixed = 151 total (was 157) +1 mvnsite 1m 47s the patch passed +1 mvneclipse 0m 48s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 4s the patch passed +1 javadoc 1m 26s the patch passed +1 unit 0m 13s hadoop-project in the patch passed. +1 unit 3m 40s hadoop-auth in the patch passed. +1 unit 9m 3s hadoop-common in the patch passed. +1 asflicense 0m 26s The patch does not generate ASF License warnings. 77m 7s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HADOOP-12082 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833808/HADOOP-12082-006.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 1c435914cc4f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b671ee6 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10814/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10814/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 6m 40s trunk passed
          +1 compile 6m 48s trunk passed
          +1 checkstyle 1m 28s trunk passed
          +1 mvnsite 1m 24s trunk passed
          +1 mvneclipse 0m 35s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 1m 41s trunk passed
          +1 javadoc 1m 9s trunk passed
          0 mvndep 1m 12s Maven dependency ordering for patch
          +1 mvninstall 1m 5s the patch passed
          +1 compile 7m 58s the patch passed
          +1 javac 7m 58s the patch passed
          -0 checkstyle 1m 38s root: The patch generated 1 new + 151 unchanged - 6 fixed = 152 total (was 157)
          +1 mvnsite 1m 47s the patch passed
          +1 mvneclipse 0m 46s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 28s the patch passed
          +1 javadoc 1m 22s the patch passed
          +1 unit 0m 13s hadoop-project in the patch passed.
          +1 unit 3m 37s hadoop-auth in the patch passed.
          +1 unit 8m 51s hadoop-common in the patch passed.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          75m 3s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HADOOP-12082
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833778/HADOOP-12082-005.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 0d280a4e8249 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ed9fcbe
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10811/artifact/patchprocess/diff-checkstyle-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10811/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10811/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 40s trunk passed +1 compile 6m 48s trunk passed +1 checkstyle 1m 28s trunk passed +1 mvnsite 1m 24s trunk passed +1 mvneclipse 0m 35s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 1m 41s trunk passed +1 javadoc 1m 9s trunk passed 0 mvndep 1m 12s Maven dependency ordering for patch +1 mvninstall 1m 5s the patch passed +1 compile 7m 58s the patch passed +1 javac 7m 58s the patch passed -0 checkstyle 1m 38s root: The patch generated 1 new + 151 unchanged - 6 fixed = 152 total (was 157) +1 mvnsite 1m 47s the patch passed +1 mvneclipse 0m 46s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 28s the patch passed +1 javadoc 1m 22s the patch passed +1 unit 0m 13s hadoop-project in the patch passed. +1 unit 3m 37s hadoop-auth in the patch passed. +1 unit 8m 51s hadoop-common in the patch passed. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 75m 3s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HADOOP-12082 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833778/HADOOP-12082-005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 0d280a4e8249 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ed9fcbe Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10811/artifact/patchprocess/diff-checkstyle-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10811/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10811/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Benoy Antony Yes. Here is the patch which fixes the checkstyle warnings.

          Show
          hgadre Hrishikesh Gadre added a comment - Benoy Antony Yes. Here is the patch which fixes the checkstyle warnings.
          Hide
          benoyantony Benoy Antony added a comment -

          Could you please fix the checkstyle warnings also ?

          Show
          benoyantony Benoy Antony added a comment - Could you please fix the checkstyle warnings also ?
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 6m 41s trunk passed
          +1 compile 6m 46s trunk passed
          +1 checkstyle 1m 27s trunk passed
          +1 mvnsite 1m 25s trunk passed
          +1 mvneclipse 0m 35s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 1m 42s trunk passed
          +1 javadoc 1m 10s trunk passed
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 1m 5s the patch passed
          +1 compile 6m 54s the patch passed
          +1 javac 6m 54s the patch passed
          -0 checkstyle 1m 32s root: The patch generated 27 new + 151 unchanged - 6 fixed = 178 total (was 157)
          +1 mvnsite 1m 35s the patch passed
          +1 mvneclipse 0m 47s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 3s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 4s the patch passed
          +1 javadoc 1m 20s the patch passed
          +1 unit 0m 15s hadoop-project in the patch passed.
          +1 unit 3m 35s hadoop-auth in the patch passed.
          +1 unit 8m 11s hadoop-common in the patch passed.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          71m 54s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HADOOP-12082
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833669/HADOOP-12082-004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 42961314b2db 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1f304b0
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10809/artifact/patchprocess/diff-checkstyle-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10809/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10809/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 41s trunk passed +1 compile 6m 46s trunk passed +1 checkstyle 1m 27s trunk passed +1 mvnsite 1m 25s trunk passed +1 mvneclipse 0m 35s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 1m 42s trunk passed +1 javadoc 1m 10s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 1m 5s the patch passed +1 compile 6m 54s the patch passed +1 javac 6m 54s the patch passed -0 checkstyle 1m 32s root: The patch generated 27 new + 151 unchanged - 6 fixed = 178 total (was 157) +1 mvnsite 1m 35s the patch passed +1 mvneclipse 0m 47s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 4s the patch passed +1 javadoc 1m 20s the patch passed +1 unit 0m 15s hadoop-project in the patch passed. +1 unit 3m 35s hadoop-auth in the patch passed. +1 unit 8m 11s hadoop-common in the patch passed. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 71m 54s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HADOOP-12082 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833669/HADOOP-12082-004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 42961314b2db 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1f304b0 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10809/artifact/patchprocess/diff-checkstyle-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10809/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10809/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 1m 35s Maven dependency ordering for branch
          +1 mvninstall 6m 44s trunk passed
          +1 compile 6m 48s trunk passed
          +1 checkstyle 1m 28s trunk passed
          +1 mvnsite 1m 26s trunk passed
          +1 mvneclipse 0m 35s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 1m 47s trunk passed
          +1 javadoc 1m 10s trunk passed
          0 mvndep 0m 18s Maven dependency ordering for patch
          +1 mvninstall 1m 5s the patch passed
          +1 compile 6m 48s the patch passed
          +1 javac 6m 48s the patch passed
          -0 checkstyle 1m 33s root: The patch generated 27 new + 151 unchanged - 6 fixed = 178 total (was 157)
          -1 mvnsite 0m 21s hadoop-auth in the patch failed.
          +1 mvneclipse 0m 47s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 7s the patch passed
          +1 javadoc 1m 20s the patch passed
          +1 unit 0m 15s hadoop-project in the patch passed.
          +1 unit 3m 35s hadoop-auth in the patch passed.
          +1 unit 8m 13s hadoop-common in the patch passed.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          73m 30s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HADOOP-12082
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833662/HADOOP-12082-003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux da202271165f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1f304b0
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10808/artifact/patchprocess/diff-checkstyle-root.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/10808/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-auth.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10808/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10808/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 1m 35s Maven dependency ordering for branch +1 mvninstall 6m 44s trunk passed +1 compile 6m 48s trunk passed +1 checkstyle 1m 28s trunk passed +1 mvnsite 1m 26s trunk passed +1 mvneclipse 0m 35s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 1m 47s trunk passed +1 javadoc 1m 10s trunk passed 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 1m 5s the patch passed +1 compile 6m 48s the patch passed +1 javac 6m 48s the patch passed -0 checkstyle 1m 33s root: The patch generated 27 new + 151 unchanged - 6 fixed = 178 total (was 157) -1 mvnsite 0m 21s hadoop-auth in the patch failed. +1 mvneclipse 0m 47s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 7s the patch passed +1 javadoc 1m 20s the patch passed +1 unit 0m 15s hadoop-project in the patch passed. +1 unit 3m 35s hadoop-auth in the patch passed. +1 unit 8m 13s hadoop-common in the patch passed. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 73m 30s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HADOOP-12082 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833662/HADOOP-12082-003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux da202271165f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1f304b0 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10808/artifact/patchprocess/diff-checkstyle-root.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/10808/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-auth.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10808/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10808/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Benoy Antony I have fixed all the reported issues. Please take a look.

          Show
          hgadre Hrishikesh Gadre added a comment - Benoy Antony I have fixed all the reported issues. Please take a look.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 7m 48s trunk passed
          +1 compile 8m 25s trunk passed
          +1 checkstyle 1m 41s trunk passed
          +1 mvnsite 1m 33s trunk passed
          +1 mvneclipse 0m 35s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 1m 56s trunk passed
          +1 javadoc 1m 16s trunk passed
          0 mvndep 1m 15s Maven dependency ordering for patch
          +1 mvninstall 1m 17s the patch passed
          +1 compile 8m 4s the patch passed
          +1 javac 8m 4s the patch passed
          -0 checkstyle 1m 42s root: The patch generated 124 new + 155 unchanged - 2 fixed = 279 total (was 157)
          -1 mvnsite 0m 22s hadoop-auth in the patch failed.
          +1 mvneclipse 0m 48s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 3s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          -1 findbugs 0m 43s hadoop-common-project/hadoop-auth generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0)
          -1 javadoc 0m 19s hadoop-auth in the patch failed.
          +1 unit 0m 14s hadoop-project in the patch passed.
          +1 unit 3m 38s hadoop-auth in the patch passed.
          +1 unit 8m 40s hadoop-common in the patch passed.
          +1 asflicense 0m 27s The patch does not generate ASF License warnings.
          79m 3s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-auth
            Found reliance on default encoding in org.apache.hadoop.security.authentication.server.LdapAuthenticationHandler.authenticate(HttpServletRequest, HttpServletResponse):in org.apache.hadoop.security.authentication.server.LdapAuthenticationHandler.authenticate(HttpServletRequest, HttpServletResponse): new String(byte[]) At LdapAuthenticationHandler.java:[line 192]
            Found reliance on default encoding in org.apache.hadoop.security.authentication.server.LdapAuthenticationHandler.authenticateUser(String, String):in org.apache.hadoop.security.authentication.server.LdapAuthenticationHandler.authenticateUser(String, String): String.getBytes() At LdapAuthenticationHandler.java:[line 215]
            org.apache.hadoop.security.authentication.server.MultiSchemeAuthenticationHandler.init(Properties) makes inefficient use of keySet iterator instead of entrySet iterator At MultiSchemeAuthenticationHandler.java:keySet iterator instead of entrySet iterator At MultiSchemeAuthenticationHandler.java:[line 113]



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HADOOP-12082
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833172/HADOOP-12082-002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux eddfe4728c5c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 332a61f
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10771/artifact/patchprocess/diff-checkstyle-root.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/10771/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-auth.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/10771/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-auth.html
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/10771/artifact/patchprocess/patch-javadoc-hadoop-common-project_hadoop-auth.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10771/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10771/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 7m 48s trunk passed +1 compile 8m 25s trunk passed +1 checkstyle 1m 41s trunk passed +1 mvnsite 1m 33s trunk passed +1 mvneclipse 0m 35s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 1m 56s trunk passed +1 javadoc 1m 16s trunk passed 0 mvndep 1m 15s Maven dependency ordering for patch +1 mvninstall 1m 17s the patch passed +1 compile 8m 4s the patch passed +1 javac 8m 4s the patch passed -0 checkstyle 1m 42s root: The patch generated 124 new + 155 unchanged - 2 fixed = 279 total (was 157) -1 mvnsite 0m 22s hadoop-auth in the patch failed. +1 mvneclipse 0m 48s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project -1 findbugs 0m 43s hadoop-common-project/hadoop-auth generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) -1 javadoc 0m 19s hadoop-auth in the patch failed. +1 unit 0m 14s hadoop-project in the patch passed. +1 unit 3m 38s hadoop-auth in the patch passed. +1 unit 8m 40s hadoop-common in the patch passed. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 79m 3s Reason Tests FindBugs module:hadoop-common-project/hadoop-auth   Found reliance on default encoding in org.apache.hadoop.security.authentication.server.LdapAuthenticationHandler.authenticate(HttpServletRequest, HttpServletResponse):in org.apache.hadoop.security.authentication.server.LdapAuthenticationHandler.authenticate(HttpServletRequest, HttpServletResponse): new String(byte[]) At LdapAuthenticationHandler.java: [line 192]   Found reliance on default encoding in org.apache.hadoop.security.authentication.server.LdapAuthenticationHandler.authenticateUser(String, String):in org.apache.hadoop.security.authentication.server.LdapAuthenticationHandler.authenticateUser(String, String): String.getBytes() At LdapAuthenticationHandler.java: [line 215]   org.apache.hadoop.security.authentication.server.MultiSchemeAuthenticationHandler.init(Properties) makes inefficient use of keySet iterator instead of entrySet iterator At MultiSchemeAuthenticationHandler.java:keySet iterator instead of entrySet iterator At MultiSchemeAuthenticationHandler.java: [line 113] Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HADOOP-12082 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833172/HADOOP-12082-002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux eddfe4728c5c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 332a61f Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10771/artifact/patchprocess/diff-checkstyle-root.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/10771/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-auth.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/10771/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-auth.html javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/10771/artifact/patchprocess/patch-javadoc-hadoop-common-project_hadoop-auth.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10771/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10771/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Benoy Antony Done !

          Show
          hgadre Hrishikesh Gadre added a comment - Benoy Antony Done !
          Hide
          benoyantony Benoy Antony added a comment -

          Looks good. Could you please "submit patch" ?

          Show
          benoyantony Benoy Antony added a comment - Looks good. Could you please "submit patch" ?
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Benoy Antony Thanks for the review! Here is an updated patch which addresses the review comment. Please take a look.

          Show
          hgadre Hrishikesh Gadre added a comment - Benoy Antony Thanks for the review! Here is an updated patch which addresses the review comment. Please take a look.
          Hide
          benoyantony Benoy Antony added a comment - - edited

          Looks good. Just a suggestion for documentation.
          I think , its better to change the filter-name in the example to "authFilter" instead of "kerberosFilter" .

          Show
          benoyantony Benoy Antony added a comment - - edited Looks good. Just a suggestion for documentation. I think , its better to change the filter-name in the example to "authFilter" instead of "kerberosFilter" .
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Kai Zhang Thanks for the feedback. The requirement is to setup a LDAP server for unit testing. After reading the docs for the kerby ldap-backend, I figured that it won't be useful. So now I have added back the ApacheDS dependencies (only for unit testing).

          Benoy Antony Can you please review the patch? I have added docs as well...

          Show
          hgadre Hrishikesh Gadre added a comment - Kai Zhang Thanks for the feedback. The requirement is to setup a LDAP server for unit testing. After reading the docs for the kerby ldap-backend, I figured that it won't be useful. So now I have added back the ApacheDS dependencies (only for unit testing). Benoy Antony Can you please review the patch? I have added docs as well...
          Hide
          drankye Kai Zheng added a comment -

          Hi Hrishikesh Gadre,

          Sorry for the inconvenience and the late reply (in PRC holiday).

          Specifically I need to add unit tests to verify the LDAP authentication functionality.

          Do these tests relate to Kerberos or not? Or basically they need an LDAP backend, instead of a KDC, right?

          Can we use the LdapBackend provided by Apache Kerby for this usecase? Or should I initialize the DirectoryService API for my unit tests?

          It depends on what these tests actually need. If they just use an LDAP server, I thought you could have some options, like the DirectoryService API. The Kerby LdapBackend is only for the Kerby KDC situation so if you don't need a KDC, then it's not good for it.

          Show
          drankye Kai Zheng added a comment - Hi Hrishikesh Gadre , Sorry for the inconvenience and the late reply (in PRC holiday). Specifically I need to add unit tests to verify the LDAP authentication functionality. Do these tests relate to Kerberos or not? Or basically they need an LDAP backend, instead of a KDC, right? Can we use the LdapBackend provided by Apache Kerby for this usecase? Or should I initialize the DirectoryService API for my unit tests? It depends on what these tests actually need. If they just use an LDAP server, I thought you could have some options, like the DirectoryService API. The Kerby LdapBackend is only for the Kerby KDC situation so if you don't need a KDC, then it's not good for it.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Kai Zhang Jiajia Li I need your help to get this patch working against the trunk.

          Specifically I need to add unit tests to verify the LDAP authentication functionality. Earlier I was using the reference of DirectoryService available in the MiniKdc to bootstrap the LDAP server. It looks like as part of HADOOP-12911, the MiniKdc was refactored to remove this reference. I am wondering what is the best way going forward for adding these tests.

          Can we use the LdapBackend provided by Apache Kerby for this usecase? Or should I initialize the DirectoryService API for my unit tests?

          Show
          hgadre Hrishikesh Gadre added a comment - Kai Zhang Jiajia Li I need your help to get this patch working against the trunk. Specifically I need to add unit tests to verify the LDAP authentication functionality. Earlier I was using the reference of DirectoryService available in the MiniKdc to bootstrap the LDAP server. It looks like as part of HADOOP-12911 , the MiniKdc was refactored to remove this reference. I am wondering what is the best way going forward for adding these tests. Can we use the LdapBackend provided by Apache Kerby for this usecase? Or should I initialize the DirectoryService API for my unit tests?
          Hide
          benoyantony Benoy Antony added a comment -

          Thanks Hrishikesh Gadre,

          The patch filename convention is is like this : HADOOP-12082-001.patch
          Could you please add the documentation regarding this feature ?
          You can start with hadoop-common-project/hadoop-auth/src/site/markdown/index.md

          Show
          benoyantony Benoy Antony added a comment - Thanks Hrishikesh Gadre , The patch filename convention is is like this : HADOOP-12082 -001.patch Could you please add the documentation regarding this feature ? You can start with hadoop-common-project/hadoop-auth/src/site/markdown/index.md
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Thanks for the review Benoy Antony

          I have addressed all the review comments. Also the latest patch follows the naming convention (I hope). Please take a look and let me know your feedback.

          Show
          hgadre Hrishikesh Gadre added a comment - Thanks for the review Benoy Antony I have addressed all the review comments. Also the latest patch follows the naming convention (I hope). Please take a look and let me know your feedback.
          Hide
          benoyantony Benoy Antony added a comment -

          Thanks for Hrishikesh Gadre for taking care of the comments.

          Here are the comments on the new patch.

          nits

          1. unnecessary change in hadoop-common-project/hadoop-minikdc/src/main/resources/minikdc.ldiff
          2. Reference to Minikdc in MiniLdap#48
          3. Does it make sense to move move the ldap related test classes under a new package , say minildap ?
          4. It will be great if you could follow the patch file naming pattern .

          recommendations

          1. In AuthenticationFIlter, Instead of doing instanceof on implementation class - MultiSchemeAuthenticationHandler , does it make sense to define an interface say, _ CompositeAuthenticationHandler_ which extends AuthenticationHandler and check against that interface ?
            This will help others also to write their implementations which could contain multiple handlers.
            This applies to DelegationTokenAuthenticationFilter and MultiSchemeDelegationTokenAuthenticationHandler also.
          2. Could you please add the documentation for MultiSchemeAuthenticationHandler especially its purpose and the how to configure it ?
          Show
          benoyantony Benoy Antony added a comment - Thanks for Hrishikesh Gadre for taking care of the comments. Here are the comments on the new patch. nits unnecessary change in hadoop-common-project/hadoop-minikdc/src/main/resources/minikdc.ldiff Reference to Minikdc in MiniLdap#48 Does it make sense to move move the ldap related test classes under a new package , say minildap ? It will be great if you could follow the patch file naming pattern . recommendations In AuthenticationFIlter, Instead of doing instanceof on implementation class - MultiSchemeAuthenticationHandler , does it make sense to define an interface say, _ CompositeAuthenticationHandler_ which extends AuthenticationHandler and check against that interface ? This will help others also to write their implementations which could contain multiple handlers. This applies to DelegationTokenAuthenticationFilter and MultiSchemeDelegationTokenAuthenticationHandler also. Could you please add the documentation for MultiSchemeAuthenticationHandler especially its purpose and the how to configure it ?
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Benoy Antony

          Thanks a lot for the feedback. Please find the updated patch attached (hadoop-ldap-auth-v6.patch).

          I have addressed all the review comments. Also I have added unit tests to verify integration between Kerberos authenticator and the new handler impl. I am not sure if we need an authenticator impl for this new handler. But if deemed necessary, it can be added later on.

          Please take a look and let me have your feedback.

          Show
          hgadre Hrishikesh Gadre added a comment - Benoy Antony Thanks a lot for the feedback. Please find the updated patch attached (hadoop-ldap-auth-v6.patch). I have addressed all the review comments. Also I have added unit tests to verify integration between Kerberos authenticator and the new handler impl. I am not sure if we need an authenticator impl for this new handler. But if deemed necessary, it can be added later on. Please take a look and let me have your feedback.
          Hide
          benoyantony Benoy Antony added a comment -

          Thanks for the new patch, HrishiKesh.

          comments

          1. In AuthenticationFilter.verifyTokenType, reommend to make the comparison consistent with the existing which does not ignore case.
          2. Create a MiniLdap clas instead of changing MiniKdc.
          3. MultiSchemeAuthenticationHandler - Are schemes Case Sensitive ? Should it be exactly "Digest" ? If not, it's better to ignore case.

          nits

          1. Unneccessary modification of AuthenticationHandler.

          Questions

          1. Is there a need to define an Authenticator for the new AuthenticationHandler ?
          2. Can KerberosAuthenticator work with MultiSchemeAuthenticationHandler ?
          Show
          benoyantony Benoy Antony added a comment - Thanks for the new patch, HrishiKesh. comments In AuthenticationFilter.verifyTokenType, reommend to make the comparison consistent with the existing which does not ignore case. Create a MiniLdap clas instead of changing MiniKdc. MultiSchemeAuthenticationHandler - Are schemes Case Sensitive ? Should it be exactly "Digest" ? If not, it's better to ignore case. nits Unneccessary modification of AuthenticationHandler. Questions Is there a need to define an Authenticator for the new AuthenticationHandler ? Can KerberosAuthenticator work with MultiSchemeAuthenticationHandler ?
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Please find the updated patch (hadoop-ldap-auth-v5.patch) and let me have your feedback.

          Show
          hgadre Hrishikesh Gadre added a comment - Please find the updated patch (hadoop-ldap-auth-v5.patch) and let me have your feedback.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Larry McCay Arun Suresh I have added Javadocs for few classes (hadoop-ldap-auth-v4.patch). Could you please take a look?

          Show
          hgadre Hrishikesh Gadre added a comment - Larry McCay Arun Suresh I have added Javadocs for few classes (hadoop-ldap-auth-v4.patch). Could you please take a look?
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Larry McCay I am sorry for inconvenience. Here is an updated patch (hadoop-ldap-auth-v3.patch).

          This also contains support for secure communication between LDAP client & server (using either LDAPS scheme OR StartTLS extension). Please take a look.

          Show
          hgadre Hrishikesh Gadre added a comment - Larry McCay I am sorry for inconvenience. Here is an updated patch (hadoop-ldap-auth-v3.patch). This also contains support for secure communication between LDAP client & server (using either LDAPS scheme OR StartTLS extension). Please take a look.
          Hide
          lmccay Larry McCay added a comment -

          For whatever reason, I am unable to successfully apply the patch.
          Only for of the files get modified and none get added.
          What are you using to apply it?

          Show
          lmccay Larry McCay added a comment - For whatever reason, I am unable to successfully apply the patch. Only for of the files get modified and none get added. What are you using to apply it?
          Hide
          lmccay Larry McCay added a comment -

          Sorry for the delay in responding - I haven't had a chance to review it in detail.
          Your description sounds dead on to me.
          I will take some time tonight to look at the implementation.

          Show
          lmccay Larry McCay added a comment - Sorry for the delay in responding - I haven't had a chance to review it in detail. Your description sounds dead on to me. I will take some time tonight to look at the implementation.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Larry McCay Any thoughts on the latest patch?

          Show
          hgadre Hrishikesh Gadre added a comment - Larry McCay Any thoughts on the latest patch?
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Larry McCay Here is an updated patch. I think this will take care of most of comments/concerns. Please take a look and let me know your thoughts.

          • MultiSchemeAuthenticationHandler is now configurable. The configuration is currently based on Java system properties (but could also be extended to use ServiceLoader pattern).

          Essentially we need to configure 'authentication.multi-scheme-auth-handler.schemes' system property to define supported authentication schemes e.g.
          authentication.multi-scheme-auth-handler.schemes=Basic,Negotiate

          For every authentication scheme, we need to configure an Auth handler implementation. e.g.
          multi-scheme-auth-handler.schemes.basic.handler=ldap
          multi-scheme-auth-handler.schemes.negotiate.handler=kerberos

          This will allow us to provide a different backend implementation for a given auth scheme (e.g. based on database etc.).

          Each auth handler will continue to have its own system properties to configure its behavior. e.g.

          authentication.ldap.providerUrl
          authentication.ldap.baseDN
          authentication.ldap.securityAuthentication

          authentication.kerberos.keytab
          authentication.kerberos.principal
          authentication.kerberos.name.rules

          I also have introduced LDAP support in MiniKDC and unit tests to validate LdapAuthenticationHandler.

          Show
          hgadre Hrishikesh Gadre added a comment - Larry McCay Here is an updated patch. I think this will take care of most of comments/concerns. Please take a look and let me know your thoughts. MultiSchemeAuthenticationHandler is now configurable. The configuration is currently based on Java system properties (but could also be extended to use ServiceLoader pattern). Essentially we need to configure 'authentication.multi-scheme-auth-handler.schemes' system property to define supported authentication schemes e.g. authentication.multi-scheme-auth-handler.schemes=Basic,Negotiate For every authentication scheme, we need to configure an Auth handler implementation. e.g. multi-scheme-auth-handler.schemes.basic.handler=ldap multi-scheme-auth-handler.schemes.negotiate.handler=kerberos This will allow us to provide a different backend implementation for a given auth scheme (e.g. based on database etc.). Each auth handler will continue to have its own system properties to configure its behavior. e.g. authentication.ldap.providerUrl authentication.ldap.baseDN authentication.ldap.securityAuthentication authentication.kerberos.keytab authentication.kerberos.principal authentication.kerberos.name.rules I also have introduced LDAP support in MiniKDC and unit tests to validate LdapAuthenticationHandler.
          Hide
          lmccay Larry McCay added a comment -

          I've applied the patch, built it and given it a quick review.
          The implementation seems fine for very specific usecase of HTTP Basic auth against an LDAP server.

          IMO, the coupling of credential scraping: pulling the credentials from the in coming request and a particular backend implementation like LDAP is unfortunate and should be reconsidered and refactored in a follow up patch. As I have described earlier, we should be able to scrape the same credentials from the request and use them against: LDAP, RDMS or some proprietary auth server. I think that this refactoring would require separating the current handler for Ldap into two things:

          1. scheme handler
          2. authentication handler

          basic.authentication.handler=ldap
          

          Using the above configuration, we could load all implementations of the UsernamePasswordAuthenticationHandler interface with the Java ServiceLoader and interrogate each one for its name. When you find the one that matches the configured value then it gets used to authenticate the username and password against its specific backend.

          This would allow us to use the same UsernamePasswordAuthenticationHandler for other ways to get username and password and to use other implementations for different backends.

          The kerberos handler doesn't need this decoupling as far as I can tell.

          Unfortunately, I don't think that I am going to be able to leverage this work for introducing WebSSO flows. Browsers only have the four well known schemes to work with and will behave according to those schemes upon getting the list of WWW-Authorization headers. I would love to use this so that we could avoid the AltKerberosAuthenticationHandler but don't see how.

          Show
          lmccay Larry McCay added a comment - I've applied the patch, built it and given it a quick review. The implementation seems fine for very specific usecase of HTTP Basic auth against an LDAP server. IMO, the coupling of credential scraping: pulling the credentials from the in coming request and a particular backend implementation like LDAP is unfortunate and should be reconsidered and refactored in a follow up patch. As I have described earlier, we should be able to scrape the same credentials from the request and use them against: LDAP, RDMS or some proprietary auth server. I think that this refactoring would require separating the current handler for Ldap into two things: 1. scheme handler 2. authentication handler basic.authentication.handler=ldap Using the above configuration, we could load all implementations of the UsernamePasswordAuthenticationHandler interface with the Java ServiceLoader and interrogate each one for its name. When you find the one that matches the configured value then it gets used to authenticate the username and password against its specific backend. This would allow us to use the same UsernamePasswordAuthenticationHandler for other ways to get username and password and to use other implementations for different backends. The kerberos handler doesn't need this decoupling as far as I can tell. Unfortunately, I don't think that I am going to be able to leverage this work for introducing WebSSO flows. Browsers only have the four well known schemes to work with and will behave according to those schemes upon getting the list of WWW-Authorization headers. I would love to use this so that we could avoid the AltKerberosAuthenticationHandler but don't see how.
          Hide
          lmccay Larry McCay added a comment -

          Hi Hrishikesh Gadre - Thank you for reaching out - I look forward to reviewing it!
          I will try and carve out some time this weekend to take a look at the patch in detail.

          The service loader pluggability that I mentioned before would be a Java ServiceLoader mechanism to load a set of classes that implement a given interface. If you provide a getName() as part of that interface then you can iterate over the set to pull out the specific configured handler.

          What I am trying to do is provide a decoupling of the challenge type and the actual backend authentication type.

          So, if we could indicate that HTTP Basic auth was accepted through the challenge, we could have the backend authenticate against the preferred user store: LDAP, RDBMS, etc. Rather than tying HTTP Basic auth the LDAP alone.

          The easiest way forward - I think - is to provide an implementation with discrete handlers for the challenges. We can then refactor that to support multiple backends.
          I would get your concrete usecases met with an implementation that we can move forward and maintain backward compatibility for without slowing progress.
          Adding pluggability later shouldn't be too hard and having tests for the initial functionality would help that work.

          Show
          lmccay Larry McCay added a comment - Hi Hrishikesh Gadre - Thank you for reaching out - I look forward to reviewing it! I will try and carve out some time this weekend to take a look at the patch in detail. The service loader pluggability that I mentioned before would be a Java ServiceLoader mechanism to load a set of classes that implement a given interface. If you provide a getName() as part of that interface then you can iterate over the set to pull out the specific configured handler. What I am trying to do is provide a decoupling of the challenge type and the actual backend authentication type. So, if we could indicate that HTTP Basic auth was accepted through the challenge, we could have the backend authenticate against the preferred user store: LDAP, RDBMS, etc. Rather than tying HTTP Basic auth the LDAP alone. The easiest way forward - I think - is to provide an implementation with discrete handlers for the challenges. We can then refactor that to support multiple backends. I would get your concrete usecases met with an implementation that we can move forward and maintain backward compatibility for without slowing progress. Adding pluggability later shouldn't be too hard and having tests for the initial functionality would help that work.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Larry McCay Please review the latest patch (hadoop-ldap.patch file). I have completed the basic implementation of LDAP integration in this patch. I think we can improve this patch in couple of aspects

          • We can fold the functionality of MultiSchemeAuthenticationHandler in the AuthenticationFilter itself. With this change, AuthenticationFilter would allow users to configure multiple handlers (e.g. kerberos + ldap) or a single handler (e.g. kerberos) in a uniform way.
          • Alternatively we can allow AuthenticationHandler to define/implement multiple authentication modes (instead of single mode).

          Please note that this patch does not contain unit tests etc. I just want to ensure that I am designing this appropriately. Once we agree on the design, I will work on completeness. Also you mentioned previously using service loader pattern. Could you please elaborate?

          Show
          hgadre Hrishikesh Gadre added a comment - Larry McCay Please review the latest patch (hadoop-ldap.patch file). I have completed the basic implementation of LDAP integration in this patch. I think we can improve this patch in couple of aspects We can fold the functionality of MultiSchemeAuthenticationHandler in the AuthenticationFilter itself. With this change, AuthenticationFilter would allow users to configure multiple handlers (e.g. kerberos + ldap) or a single handler (e.g. kerberos) in a uniform way. Alternatively we can allow AuthenticationHandler to define/implement multiple authentication modes (instead of single mode). Please note that this patch does not contain unit tests etc. I just want to ensure that I am designing this appropriately. Once we agree on the design, I will work on completeness. Also you mentioned previously using service loader pattern. Could you please elaborate?
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Larry McCay I have been traveling for last few weeks. Expect a patch in next few days.

          Show
          hgadre Hrishikesh Gadre added a comment - Larry McCay I have been traveling for last few weeks. Expect a patch in next few days.
          Hide
          roji Shay Rojansky added a comment -

          Btw, if you're looking at reworking authentication it may be a good idea to look at SASL.

          Show
          roji Shay Rojansky added a comment - Btw, if you're looking at reworking authentication it may be a good idea to look at SASL.
          Hide
          lmccay Larry McCay added a comment -

          Hi Hrishikesh Gadre - curious about the status of this work.
          I think that it adds value and would like to see it move forward.

          Show
          lmccay Larry McCay added a comment - Hi Hrishikesh Gadre - curious about the status of this work. I think that it adds value and would like to see it move forward.
          Hide
          lmccay Larry McCay added a comment -

          GSSAPI is supposedly a generic security API. That is just noise though. We
          won't need anything other than kerberos for negotiate.

          Show
          lmccay Larry McCay added a comment - GSSAPI is supposedly a generic security API. That is just noise though. We won't need anything other than kerberos for negotiate.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Larry McCay

          >>Technically, GSS based Negotiate doesn't have to be limited to SPNEGO either.

          Interesting. As per my understanding (based on the articles below), that is not the case. Can you please take a look and comment?

          SPNEGO-based Kerberos and NTLM HTTP Authentication
          Understanding HTTP Authentication

          Show
          hgadre Hrishikesh Gadre added a comment - Larry McCay >>Technically, GSS based Negotiate doesn't have to be limited to SPNEGO either. Interesting. As per my understanding (based on the articles below), that is not the case. Can you please take a look and comment? SPNEGO-based Kerberos and NTLM HTTP Authentication Understanding HTTP Authentication
          Hide
          lmccay Larry McCay added a comment -

          To be clear, I don't think that those considerations should block progress here - they could be discussed and handled in a follow up jira.

          Show
          lmccay Larry McCay added a comment - To be clear, I don't think that those considerations should block progress here - they could be discussed and handled in a follow up jira.
          Hide
          lmccay Larry McCay added a comment -

          One thing that I am curious about - we are describing a Basic challenge as representing LDAP - however, UIs generally want to present a form-based login page rather than use the browser Basic login dialog. How do we add this level of fidelity.

          Coming back to the server side - there really isn't anything about Basic that requires it to be LDAP - of course this is probably most common.

          I have been mulling around the thought of having the auth mechanisms be pluggable using a combination of ServiceLoader and names. We would add (probably just marker) interfaces to represent Negotiate, Basic, etc. Use ServiceLoader to get all the Negotiate implementations and interrogate each one for a getName() that matches the configured name for the Negotiate impl.

          This would allow us to overload Basic with LDAP, JDBC, etc. Likewise, if we were to consider a Bearer token impl, we could overload Bearer with SAML, JWT, etc - though we may be able to derive the type of the Bearer token through the request. For instance, OAuth JWT+Bearer grant type would be set to something like "grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer".

          Technically, GSS based Negotiate doesn't have to be limited to SPNEGO either. Doubtful that we would need to overload that any time soon though.

          Show
          lmccay Larry McCay added a comment - One thing that I am curious about - we are describing a Basic challenge as representing LDAP - however, UIs generally want to present a form-based login page rather than use the browser Basic login dialog. How do we add this level of fidelity. Coming back to the server side - there really isn't anything about Basic that requires it to be LDAP - of course this is probably most common. I have been mulling around the thought of having the auth mechanisms be pluggable using a combination of ServiceLoader and names. We would add (probably just marker) interfaces to represent Negotiate, Basic, etc. Use ServiceLoader to get all the Negotiate implementations and interrogate each one for a getName() that matches the configured name for the Negotiate impl. This would allow us to overload Basic with LDAP, JDBC, etc. Likewise, if we were to consider a Bearer token impl, we could overload Bearer with SAML, JWT, etc - though we may be able to derive the type of the Bearer token through the request. For instance, OAuth JWT+Bearer grant type would be set to something like "grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer". Technically, GSS based Negotiate doesn't have to be limited to SPNEGO either. Doubtful that we would need to overload that any time soon though.
          Hide
          roji Shay Rojansky added a comment -

          +1 on this.

          An additional use case is accessing the Hadoop web UIs from outside the corporate infrastructure, e.g. from home over a VPN. The home computer isn't in the Kerberos realm so can't log in. With the current AltKerberosAuthenticationHandler scheme it's impossible for internal browsers to use SPNEGO/Kerberos and for external browsers to fall back to LDAP.

          Show
          roji Shay Rojansky added a comment - +1 on this. An additional use case is accessing the Hadoop web UIs from outside the corporate infrastructure, e.g. from home over a VPN. The home computer isn't in the Kerberos realm so can't log in. With the current AltKerberosAuthenticationHandler scheme it's impossible for internal browsers to use SPNEGO/Kerberos and for external browsers to fall back to LDAP.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Larry McCay

          >>AltKerberosAuthenticationProvider seems to require that kerberos be enabled for HTTP - does this implementation also require kerberos? While it being enabled does make sense - there are folks that want to provide authenticated access to the UIs for deployments without SPNEGO turned on too.

          I don't think so. The AuthenticationFilter implementation is generic enough to allow specifying custom handlers. So it should be straightforward to configure a Auth handler implementing a scheme different than Kerberos. (Note for our use-case we do want to support LDAP and Kerberos simultaneously).

          Show
          hgadre Hrishikesh Gadre added a comment - Larry McCay >>AltKerberosAuthenticationProvider seems to require that kerberos be enabled for HTTP - does this implementation also require kerberos? While it being enabled does make sense - there are folks that want to provide authenticated access to the UIs for deployments without SPNEGO turned on too. I don't think so. The AuthenticationFilter implementation is generic enough to allow specifying custom handlers. So it should be straightforward to configure a Auth handler implementing a scheme different than Kerberos. (Note for our use-case we do want to support LDAP and Kerberos simultaneously).
          Hide
          lmccay Larry McCay added a comment -

          I was aware of that but didn't think it would be picked up by browsers or other clients that were aware of this multiple scheme protocol.

          That would be ideal.

          Adding another handler would be easy enough - I think that we should make it more dynamic/configurable though.
          There are multiple types of Bearer tokens for instance: JWT, SAML, OAuth, etc.

          I'd be happy to help out with additional handlers to plugin if that is of interest.

          Show
          lmccay Larry McCay added a comment - I was aware of that but didn't think it would be picked up by browsers or other clients that were aware of this multiple scheme protocol. That would be ideal. Adding another handler would be easy enough - I think that we should make it more dynamic/configurable though. There are multiple types of Bearer tokens for instance: JWT, SAML, OAuth, etc. I'd be happy to help out with additional handlers to plugin if that is of interest.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Larry McCay

          >>It's a shame that there doesn't seem to be a way to present a Bearer token authentication scheme.

          Not really. HTTP specs support Bearer token authentication. Please take a look at
          http://www.iana.org/assignments/http-authschemes/http-authschemes.xhtml

          I think we would need to add another AuthenticationHandler implementing Bearer tokens and add it to MultiSchemeAuthenticationHandler (Please see the attached file multi-scheme-auth-support-poc.patch).

          Show
          hgadre Hrishikesh Gadre added a comment - Larry McCay >>It's a shame that there doesn't seem to be a way to present a Bearer token authentication scheme. Not really. HTTP specs support Bearer token authentication. Please take a look at http://www.iana.org/assignments/http-authschemes/http-authschemes.xhtml I think we would need to add another AuthenticationHandler implementing Bearer tokens and add it to MultiSchemeAuthenticationHandler (Please see the attached file multi-scheme-auth-support-poc.patch).
          Hide
          lmccay Larry McCay added a comment -

          I like this approach!

          One benefit that I can see over AltKerberosAuthenticationHandler is that I've encountered "non-browser" user agents that were calling the JMX servlet and they weren't capable of NEGOTIATE or the custom handler - like a redirect to a login page. This would at least allow them to use Basic. I don't think that is actually a possibility for what I saw but could maybe be added.

          It's a shame that there doesn't seem to be a way to present a Bearer token authentication scheme.

          I wonder if we could add something like Bearer+JWT to the server side and at then clients that are aware of it can take advantage....this needs to be thought through more but feels like something that could add value.

          Question: AltKerberosAuthenticationProvider seems to require that kerberos be enabled for HTTP - does this implementation also require kerberos? While it being enabled does make sense - there are folks that want to provide authenticated access to the UIs for deployments without SPNEGO turned on too.

          Show
          lmccay Larry McCay added a comment - I like this approach! One benefit that I can see over AltKerberosAuthenticationHandler is that I've encountered "non-browser" user agents that were calling the JMX servlet and they weren't capable of NEGOTIATE or the custom handler - like a redirect to a login page. This would at least allow them to use Basic. I don't think that is actually a possibility for what I saw but could maybe be added. It's a shame that there doesn't seem to be a way to present a Bearer token authentication scheme. I wonder if we could add something like Bearer+JWT to the server side and at then clients that are aware of it can take advantage....this needs to be thought through more but feels like something that could add value. Question: AltKerberosAuthenticationProvider seems to require that kerberos be enabled for HTTP - does this implementation also require kerberos? While it being enabled does make sense - there are folks that want to provide authenticated access to the UIs for deployments without SPNEGO turned on too.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Arun Suresh

          >>Considering the fact that browsers automatically always pick the "strongest" scheme, is there anyway to allow that certain clients/browsers over-ride this and pick LDAP over Kerb ?

          It looks like browsers are able to detect the fact that

          • server supports multiple authentication schemes (viz. Basic and Negotiate)
          • the kerberos credentials are missing on the client side

          Hence the browser pops-up a dialog box for an user to enter username/password. Upon entering correct credentials, the website opens as expected. If wrong credentials are supplied, the server throws 403 error (as expected). I have tested this on three different browsers - Safari, Chrome & Firefox.

          As per my understanding - the "strongest security scheme" is chosen based on available information. e.g. if a browser already has access to kerberos credentials, it will not ask for username/password since kerberos auth scheme is "more" secure than basic auth.

          Show
          hgadre Hrishikesh Gadre added a comment - Arun Suresh >>Considering the fact that browsers automatically always pick the "strongest" scheme, is there anyway to allow that certain clients/browsers over-ride this and pick LDAP over Kerb ? It looks like browsers are able to detect the fact that server supports multiple authentication schemes (viz. Basic and Negotiate) the kerberos credentials are missing on the client side Hence the browser pops-up a dialog box for an user to enter username/password. Upon entering correct credentials, the website opens as expected. If wrong credentials are supplied, the server throws 403 error (as expected). I have tested this on three different browsers - Safari, Chrome & Firefox. As per my understanding - the "strongest security scheme" is chosen based on available information. e.g. if a browser already has access to kerberos credentials, it will not ask for username/password since kerberos auth scheme is "more" secure than basic auth.
          Hide
          asuresh Arun Suresh added a comment -

          Hrishikesh Gadre, thanks for opening this.

          One specific usecase of the AltKerberosHandler was to allow clients (browsers) not within the security infrastructure to talk to hadoop. Considering the fact that browsers automatically always pick the "strongest" scheme, is there anyway to allow that certain clients/browsers over-ride this and pick LDAP over Kerb ?

          Show
          asuresh Arun Suresh added a comment - Hrishikesh Gadre , thanks for opening this. One specific usecase of the AltKerberosHandler was to allow clients (browsers) not within the security infrastructure to talk to hadoop. Considering the fact that browsers automatically always pick the "strongest" scheme, is there anyway to allow that certain clients/browsers over-ride this and pick LDAP over Kerb ?
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Here is a skeleton code implementing the multi-scheme authentication. Currently it is supporting two authentication schemes,

          • Basic authentication using hardcoded username/password
          • Negotiate scheme using Kerberos

          We would be expanding on the LDAP support to replace the hardcoded username/password. I have tested this patch of hadoop 2.6 codebase and it is working as expected. This patch will provide more insights for this proposal.

          Show
          hgadre Hrishikesh Gadre added a comment - Here is a skeleton code implementing the multi-scheme authentication. Currently it is supporting two authentication schemes, Basic authentication using hardcoded username/password Negotiate scheme using Kerberos We would be expanding on the LDAP support to replace the hardcoded username/password. I have tested this patch of hadoop 2.6 codebase and it is working as expected. This patch will provide more insights for this proposal.

            People

            • Assignee:
              hgadre Hrishikesh Gadre
              Reporter:
              hgadre Hrishikesh Gadre
            • Votes:
              1 Vote for this issue
              Watchers:
              26 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development