Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13707

If kerberos is enabled while HTTP SPNEGO is not configured, some links cannot be accessed

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:

      Description

      In HttpServer2#hasAdministratorAccess, it uses `hadoop.security.authorization` to detect whether HTTP is authenticated.
      It's not correct, because enabling Kerberos and HTTP SPNEGO are two steps. If Kerberos is enabled while HTTP SPNEGO is not, some links cannot be accessed, such as "/logs", and it will return error message as below:

      HTTP ERROR 403
      Problem accessing /logs/. Reason:
      User dr.who is unauthorized to access this page.

      We should make sure HttpServletRequest#getAuthType is not null before we invoke HttpServer2#hasAdministratorAccess.

      getAuthType means to get the authorization scheme of this request

      1. HADOOP-13707.001.patch
        2 kB
        Yuanbo Liu
      2. HADOOP-13707.002.patch
        7 kB
        Yuanbo Liu
      3. HADOOP-13707.003.patch
        11 kB
        Yuanbo Liu
      4. HADOOP-13707.004.patch
        11 kB
        Yuanbo Liu
      5. HADOOP-13707-branch-2.8.patch
        11 kB
        Yuanbo Liu
      6. HADOOP-13707-branch-2.patch
        12 kB
        Yuanbo Liu
      7. HADOOP-13707-branch-2-addendum.patch
        0.8 kB
        Brahma Reddy Battula

        Issue Links

          Activity

          Hide
          yuanbo Yuanbo Liu added a comment -

          I've prepared a patch without any test case. I'd like to get community's thoughts before I complete it.
          Any response will be appreciated, thanks in advance.

          Show
          yuanbo Yuanbo Liu added a comment - I've prepared a patch without any test case. I'd like to get community's thoughts before I complete it. Any response will be appreciated, thanks in advance.
          Hide
          aw Allen Wittenauer added a comment - - edited

          /logs was specifically blocked way back when due to the sensitive nature of the content. Non-admin users shouldn't be looking at it at all and admin users have access from the shell.

          It's probably also worth pointing out that these logs are typically huge and viewing them in a browser is a pretty terrible experience.

          Show
          aw Allen Wittenauer added a comment - - edited /logs was specifically blocked way back when due to the sensitive nature of the content. Non-admin users shouldn't be looking at it at all and admin users have access from the shell. It's probably also worth pointing out that these logs are typically huge and viewing them in a browser is a pretty terrible experience.
          Hide
          yuanbo Yuanbo Liu added a comment - - edited

          Allen Wittenauer Thanks for your response.
          Non-admin users shouldn't be looking at it in security environment. But if HTTP SPNEGO is not enabled, that is to say, in non-security environment for http sever, users cannot be authenticated and passed to NameNode, and "/logs" should be accessed by all users.

          It's probably also worth pointing out that these logs are typically huge...

          Agree with you. I think the biggest feature of "/logs" is to provide urls to download logs. Browsing logs online shouldn't be encouraged.

          Show
          yuanbo Yuanbo Liu added a comment - - edited Allen Wittenauer Thanks for your response. Non-admin users shouldn't be looking at it in security environment. But if HTTP SPNEGO is not enabled, that is to say, in non-security environment for http sever, users cannot be authenticated and passed to NameNode, and "/logs" should be accessed by all users. It's probably also worth pointing out that these logs are typically huge... Agree with you. I think the biggest feature of "/logs" is to provide urls to download logs. Browsing logs online shouldn't be encouraged.
          Hide
          yuanbo Yuanbo Liu added a comment -

          I've changed my description a bit, because it seems not logic to change code in HttpServer2#hasAdministratorAccess, we should stop servlet invokes it if authType is null.

          Show
          yuanbo Yuanbo Liu added a comment - I've changed my description a bit, because it seems not logic to change code in HttpServer2#hasAdministratorAccess , we should stop servlet invokes it if authType is null.
          Hide
          yuanbo Yuanbo Liu added a comment -

          If HTTP SPNEGO is not enabled, user can not be authenticated, and "dr.who" is used as a default user.
          HttpServer2#hasAdministratorAccess is a authorization method to verify whether user has admin access. Since user(dr.who) is not authenticated, http sever is in non-security environment, there is no need to call hasAdministratorAccess to do authorization check.
          HttpServletRequest#getAuthType returns null in non-security environment (without AuthenticationFilter), so we can take advantage of it and use it to determine whether hasAdministratorAccess should be called.

          Show
          yuanbo Yuanbo Liu added a comment - If HTTP SPNEGO is not enabled, user can not be authenticated, and "dr.who" is used as a default user. HttpServer2#hasAdministratorAccess is a authorization method to verify whether user has admin access. Since user(dr.who) is not authenticated, http sever is in non-security environment, there is no need to call hasAdministratorAccess to do authorization check. HttpServletRequest#getAuthType returns null in non-security environment (without AuthenticationFilter), so we can take advantage of it and use it to determine whether hasAdministratorAccess should be called.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 14s trunk passed
          +1 compile 8m 16s trunk passed
          +1 checkstyle 0m 28s trunk passed
          +1 mvnsite 1m 5s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 27s trunk passed
          +1 javadoc 0m 45s trunk passed
          +1 mvninstall 0m 42s the patch passed
          +1 compile 7m 51s the patch passed
          +1 javac 7m 51s the patch passed
          +1 checkstyle 0m 28s the patch passed
          +1 mvnsite 1m 2s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 37s the patch passed
          +1 javadoc 0m 43s the patch passed
          +1 unit 8m 32s hadoop-common in the patch passed.
          +1 asflicense 0m 23s The patch does not generate ASF License warnings.
          43m 45s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HADOOP-13707
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832835/HADOOP-13707.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 58db3f1c961e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6476934
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10739/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10739/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 14s trunk passed +1 compile 8m 16s trunk passed +1 checkstyle 0m 28s trunk passed +1 mvnsite 1m 5s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 27s trunk passed +1 javadoc 0m 45s trunk passed +1 mvninstall 0m 42s the patch passed +1 compile 7m 51s the patch passed +1 javac 7m 51s the patch passed +1 checkstyle 0m 28s the patch passed +1 mvnsite 1m 2s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 37s the patch passed +1 javadoc 0m 43s the patch passed +1 unit 8m 32s hadoop-common in the patch passed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 43m 45s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HADOOP-13707 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832835/HADOOP-13707.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 58db3f1c961e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6476934 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10739/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10739/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          yuanbo Yuanbo Liu added a comment -

          After discussing with Weiwei Yang, I realize that sometimes users may just want to pass user name to NameNode without authentication, but want to keep admin authorization check. So I restrict conditions of non-security environment in my v003 patch. Only static user without authentication will be considered as non-security environment.

          Show
          yuanbo Yuanbo Liu added a comment - After discussing with Weiwei Yang , I realize that sometimes users may just want to pass user name to NameNode without authentication, but want to keep admin authorization check. So I restrict conditions of non-security environment in my v003 patch. Only static user without authentication will be considered as non-security environment.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Yuanbo Liu thanks for working in a patch, however I am not sure if this the right approach. Like what Allen said, logs are not supposed to be seen by non-admin if the cluster is Kerberized. I feel like a correct approach is to add a SPENGO filter for /logs so that it is accessible for Kerberos users just like /jmx and /logLevel. Does that make sense?

          Show
          jojochuang Wei-Chiu Chuang added a comment - Yuanbo Liu thanks for working in a patch, however I am not sure if this the right approach. Like what Allen said, logs are not supposed to be seen by non-admin if the cluster is Kerberized. I feel like a correct approach is to add a SPENGO filter for /logs so that it is accessible for Kerberos users just like /jmx and /logLevel. Does that make sense?
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 25s trunk passed
          +1 compile 8m 23s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 1m 4s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 32s trunk passed
          +1 javadoc 0m 46s trunk passed
          +1 mvninstall 0m 43s the patch passed
          +1 compile 8m 11s the patch passed
          +1 javac 8m 11s the patch passed
          -0 checkstyle 0m 27s hadoop-common-project/hadoop-common: The patch generated 1 new + 135 unchanged - 0 fixed = 136 total (was 135)
          +1 mvnsite 1m 2s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 47s the patch passed
          +1 javadoc 0m 44s the patch passed
          +1 unit 8m 8s hadoop-common in the patch passed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          44m 13s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HADOOP-13707
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832867/HADOOP-13707.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 867e043ecce8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6476934
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10742/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10742/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10742/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 25s trunk passed +1 compile 8m 23s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 4s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 32s trunk passed +1 javadoc 0m 46s trunk passed +1 mvninstall 0m 43s the patch passed +1 compile 8m 11s the patch passed +1 javac 8m 11s the patch passed -0 checkstyle 0m 27s hadoop-common-project/hadoop-common: The patch generated 1 new + 135 unchanged - 0 fixed = 136 total (was 135) +1 mvnsite 1m 2s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 44s the patch passed +1 unit 8m 8s hadoop-common in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 44m 13s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HADOOP-13707 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832867/HADOOP-13707.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 867e043ecce8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6476934 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10742/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10742/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10742/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Wei-Chiu Chuang Thanks for your comments.

          I feel like a correct approach is to add a SPENGO filter...

          Yes you're right, actually I'm ready to add a SPENGO filter with delegation feature in HADOOP-13119. But as I said, enabling Kerberos and SPENGO are two steps. If users enable Kerberos without SPENGO, that means the http sever of the cluster is in non-security environment. In this situation, static user's authorization shouldn't be checked.
          In the very first installation of Hadoop, http sever is also in non-security environment without any authorization check. So I think the behavior here should be consistent and "dr.who" issue should be avoid.
          Thanks again for your comments, looking forward to your response.

          Show
          yuanbo Yuanbo Liu added a comment - Wei-Chiu Chuang Thanks for your comments. I feel like a correct approach is to add a SPENGO filter... Yes you're right, actually I'm ready to add a SPENGO filter with delegation feature in HADOOP-13119 . But as I said, enabling Kerberos and SPENGO are two steps. If users enable Kerberos without SPENGO, that means the http sever of the cluster is in non-security environment. In this situation, static user's authorization shouldn't be checked. In the very first installation of Hadoop, http sever is also in non-security environment without any authorization check. So I think the behavior here should be consistent and "dr.who" issue should be avoid. Thanks again for your comments, looking forward to your response.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Adding SPENGO filter belongs to enabling SPENGO step.

          Show
          yuanbo Yuanbo Liu added a comment - Adding SPENGO filter belongs to enabling SPENGO step.
          Hide
          yuanbo Yuanbo Liu added a comment -

          uploaded v4 patch to address checkstyle issue

          Show
          yuanbo Yuanbo Liu added a comment - uploaded v4 patch to address checkstyle issue
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 9m 5s trunk passed
          +1 compile 8m 22s trunk passed
          +1 checkstyle 0m 25s trunk passed
          +1 mvnsite 0m 59s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 21s trunk passed
          +1 javadoc 0m 43s trunk passed
          +1 mvninstall 0m 38s the patch passed
          +1 compile 6m 49s the patch passed
          +1 javac 6m 49s the patch passed
          +1 checkstyle 0m 25s the patch passed
          +1 mvnsite 0m 54s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 26s the patch passed
          +1 javadoc 0m 41s the patch passed
          +1 unit 7m 47s hadoop-common in the patch passed.
          +1 asflicense 0m 23s The patch does not generate ASF License warnings.
          42m 10s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HADOOP-13707
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833010/HADOOP-13707.004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux a8353e013568 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 12d739a
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10754/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10754/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 5s trunk passed +1 compile 8m 22s trunk passed +1 checkstyle 0m 25s trunk passed +1 mvnsite 0m 59s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 21s trunk passed +1 javadoc 0m 43s trunk passed +1 mvninstall 0m 38s the patch passed +1 compile 6m 49s the patch passed +1 javac 6m 49s the patch passed +1 checkstyle 0m 25s the patch passed +1 mvnsite 0m 54s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 26s the patch passed +1 javadoc 0m 41s the patch passed +1 unit 7m 47s hadoop-common in the patch passed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 42m 10s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HADOOP-13707 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833010/HADOOP-13707.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a8353e013568 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 12d739a Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10754/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10754/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          eyang Eric Yang added a comment -

          This is a intermediate step required to provide ability to expand security options for securing web interface. Look forward to HADOOP-13119. I just committed this. Thank you, Yuanbo.

          Show
          eyang Eric Yang added a comment - This is a intermediate step required to provide ability to expand security options for securing web interface. Look forward to HADOOP-13119 . I just committed this. Thank you, Yuanbo.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Eric Yang it's broke branch-2 compilation, as it's following import is not there...
          import javax.servlet.ServletContext...will attach addendum patch..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Eric Yang it's broke branch-2 compilation, as it's following import is not there... import javax.servlet.ServletContext ...will attach addendum patch..
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Uploaded the addendum patch for branch-2..Please review, we can commit asap Orlese we may need to revert.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Uploaded the addendum patch for branch-2..Please review, we can commit asap Orlese we may need to revert.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Eric Yang Thanks for your commit
          Brahma Reddy Battula Thanks for your review
          I've prepared branch-2, branch-2.8 patches for this issue. please see the attachments and review them. Thanks in advance!

          Show
          yuanbo Yuanbo Liu added a comment - Eric Yang Thanks for your commit Brahma Reddy Battula Thanks for your review I've prepared branch-2, branch-2.8 patches for this issue. please see the attachments and review them. Thanks in advance!
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Reopening issue, Reverted for branch-2 and branch-2 as it's broken compilation...Bytheway thanks Yuanbo Liu for updating patches for branch-2 and branch-2.8.Lets jenkins run against this.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Reopening issue, Reverted for branch-2 and branch-2 as it's broken compilation...Bytheway thanks Yuanbo Liu for updating patches for branch-2 and branch-2.8.Lets jenkins run against this.
          Hide
          yuanbo Yuanbo Liu added a comment - - edited

          Brahma Reddy Battula I have no idea about how to re-establish Jenkins job. I was using "Resume Progress" -> "Submit patch", but it didn't work. It would be better if the dashboard contains something like "Rerun Jenkins" button.

          Show
          yuanbo Yuanbo Liu added a comment - - edited Brahma Reddy Battula I have no idea about how to re-establish Jenkins job. I was using "Resume Progress" -> "Submit patch", but it didn't work. It would be better if the dashboard contains something like "Rerun Jenkins" button.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          I was using "Resume Progress" -> "Submit patch", but it didn't work

          you've to re-upload the patch to run jenkins..

          It would be better if the dashboard contains something like "Rerun Jenkins" button.

          there is rebuild button and build with parameters.Only login users(with apache id) can trigger.

          I Triggered the jenkins.

          Show
          brahmareddy Brahma Reddy Battula added a comment - I was using "Resume Progress" -> "Submit patch", but it didn't work you've to re-upload the patch to run jenkins.. It would be better if the dashboard contains something like "Rerun Jenkins" button. there is rebuild button and build with parameters.Only login users(with apache id) can trigger. I Triggered the jenkins.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 37s branch-2.8 passed
          +1 compile 5m 54s branch-2.8 passed with JDK v1.8.0_101
          +1 compile 6m 49s branch-2.8 passed with JDK v1.7.0_111
          +1 checkstyle 0m 24s branch-2.8 passed
          +1 mvnsite 0m 58s branch-2.8 passed
          +1 mvneclipse 0m 16s branch-2.8 passed
          +1 findbugs 1m 38s branch-2.8 passed
          +1 javadoc 0m 45s branch-2.8 passed with JDK v1.8.0_101
          +1 javadoc 0m 57s branch-2.8 passed with JDK v1.7.0_111
          +1 mvninstall 0m 40s the patch passed
          +1 compile 5m 45s the patch passed with JDK v1.8.0_101
          +1 javac 5m 45s the patch passed
          +1 compile 6m 52s the patch passed with JDK v1.7.0_111
          +1 javac 6m 52s the patch passed
          +1 checkstyle 0m 23s the patch passed
          +1 mvnsite 0m 59s the patch passed
          +1 mvneclipse 0m 16s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 findbugs 1m 52s the patch passed
          +1 javadoc 0m 45s the patch passed with JDK v1.8.0_101
          +1 javadoc 0m 56s the patch passed with JDK v1.7.0_111
          +1 unit 8m 19s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          61m 55s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:5af2af1
          JIRA Issue HADOOP-13707
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833592/HADOOP-13707-branch-2.8.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 49a6066a2212 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.8 / 1c47389
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10807/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10807/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10807/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 37s branch-2.8 passed +1 compile 5m 54s branch-2.8 passed with JDK v1.8.0_101 +1 compile 6m 49s branch-2.8 passed with JDK v1.7.0_111 +1 checkstyle 0m 24s branch-2.8 passed +1 mvnsite 0m 58s branch-2.8 passed +1 mvneclipse 0m 16s branch-2.8 passed +1 findbugs 1m 38s branch-2.8 passed +1 javadoc 0m 45s branch-2.8 passed with JDK v1.8.0_101 +1 javadoc 0m 57s branch-2.8 passed with JDK v1.7.0_111 +1 mvninstall 0m 40s the patch passed +1 compile 5m 45s the patch passed with JDK v1.8.0_101 +1 javac 5m 45s the patch passed +1 compile 6m 52s the patch passed with JDK v1.7.0_111 +1 javac 6m 52s the patch passed +1 checkstyle 0m 23s the patch passed +1 mvnsite 0m 59s the patch passed +1 mvneclipse 0m 16s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 findbugs 1m 52s the patch passed +1 javadoc 0m 45s the patch passed with JDK v1.8.0_101 +1 javadoc 0m 56s the patch passed with JDK v1.7.0_111 +1 unit 8m 19s hadoop-common in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 61m 55s Subsystem Report/Notes Docker Image:yetus/hadoop:5af2af1 JIRA Issue HADOOP-13707 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833592/HADOOP-13707-branch-2.8.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 49a6066a2212 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.8 / 1c47389 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10807/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10807/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10807/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          eyang Eric Yang added a comment -

          Sorry for breaking branch-2. This has been fixed. Thanks Brahma for trigger the build. The white space issue isn't related to this patch. Thanks Yuanbo for follow up.

          Show
          eyang Eric Yang added a comment - Sorry for breaking branch-2. This has been fixed. Thanks Brahma for trigger the build. The white space issue isn't related to this patch. Thanks Yuanbo for follow up.
          Hide
          cheersyang Weiwei Yang added a comment -

          Hi Eric Yang

          Looks like branch-2 is still broken, the LogLevel class is still missing import causing the compilation failure. Please check. Thank you.

          Show
          cheersyang Weiwei Yang added a comment - Hi Eric Yang Looks like branch-2 is still broken, the LogLevel class is still missing import causing the compilation failure. Please check. Thank you.
          Hide
          eyang Eric Yang added a comment -

          WeiWei, thanks for the verification. I just found out git didn't commit LogLevel, and committed accordingly. It should work now. Thanks

          Show
          eyang Eric Yang added a comment - WeiWei, thanks for the verification. I just found out git didn't commit LogLevel, and committed accordingly. It should work now. Thanks
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Yes, again "import javax.servlet.ServletContext" is missed which brokes branch-2 compliation and MetricsServlet changes are not present. Actullay Yuanbo Liu patches are correct but same is not get committed...Eric Yang there might be problem while commiting to branch-2 and branch-2.8 from your side.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Yes, again "import javax.servlet.ServletContext" is missed which brokes branch-2 compliation and MetricsServlet changes are not present. Actullay Yuanbo Liu patches are correct but same is not get committed... Eric Yang there might be problem while commiting to branch-2 and branch-2.8 from your side.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          checking this locally.

          -Eric's fix for branch-2 has got it happy; Brahma's reversion of the patch from 2.8 has that happy too.

          I'm going to tweak the tagged release version of this JIRA to 2.9 until a patch that builds goes into 2.8. I'm expecting it to work this time, but do a quick preflight build before pushing up, if you can. At least with weekend patches things can get fixed before our US colleagues get to their keyboards on a monday morning.

          Show
          stevel@apache.org Steve Loughran added a comment - checking this locally. -Eric's fix for branch-2 has got it happy; Brahma's reversion of the patch from 2.8 has that happy too. I'm going to tweak the tagged release version of this JIRA to 2.9 until a patch that builds goes into 2.8. I'm expecting it to work this time, but do a quick preflight build before pushing up, if you can. At least with weekend patches things can get fixed before our US colleagues get to their keyboards on a monday morning.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Eric Yang As I mentioned above MetricsServlet.java changes are missed in branch-2 Commit ,Please have a look at latest patch uploaded by Yuanbo Liu for branch-2 and branch-2.8..

          May be you need revert from branch-2 and apply the latest patch for branch-2 OR prepare addedum patch for only MetricsServlet.java changes.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Eric Yang As I mentioned above MetricsServlet.java changes are missed in branch-2 Commit ,Please have a look at latest patch uploaded by Yuanbo Liu for branch-2 and branch-2.8.. May be you need revert from branch-2 and apply the latest patch for branch-2 OR prepare addedum patch for only MetricsServlet.java changes.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Eric Yang Really sorry for not pointing out trunk patch and branch-2.8/branch-2 patch are slightly different because of MetricsServlet.java. My patches in the attachment contain some changes about MetricsServlet.java. Hope my mistake won't bother you too much!

          Brahma Reddy Battula Thanks a lot for your reminder!

          Show
          yuanbo Yuanbo Liu added a comment - Eric Yang Really sorry for not pointing out trunk patch and branch-2.8/branch-2 patch are slightly different because of MetricsServlet.java . My patches in the attachment contain some changes about MetricsServlet.java . Hope my mistake won't bother you too much! Brahma Reddy Battula Thanks a lot for your reminder!
          Hide
          eyang Eric Yang added a comment -

          Thanks Brahma and Steve for catching the mistakes. I have amended the patch with MetricsServlet changes, and also applied branch-2.8 patch. Preflight passed this time. Thank you all.

          Show
          eyang Eric Yang added a comment - Thanks Brahma and Steve for catching the mistakes. I have amended the patch with MetricsServlet changes, and also applied branch-2.8 patch. Preflight passed this time. Thank you all.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Eric Yang this is not committed to trunk. As of now, I am removing the fix version.

          You committed to master branch which is stale.

          you can check the master branch commit history in following link, Last commit is on Aug 7, 2015.

          https://github.com/apache/hadoop/commits/master..

          I think, we should delete the master branch.. Let me discuss about removal of master.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Eric Yang this is not committed to trunk. As of now, I am removing the fix version. You committed to master branch which is stale. you can check the master branch commit history in following link, Last commit is on Aug 7, 2015. https://github.com/apache/hadoop/commits/master .. I think, we should delete the master branch.. Let me discuss about removal of master.
          Hide
          brahmareddy Brahma Reddy Battula added a comment - - edited

          Pushed to trunk..

          Steve Loughran can we delete master branch..? shalI we discuss in mailing-list..?
          am I wrong here..? thanks

          Show
          brahmareddy Brahma Reddy Battula added a comment - - edited Pushed to trunk.. Steve Loughran can we delete master branch..? shalI we discuss in mailing-list..? am I wrong here..? thanks
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10789 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10789/)
          HADOOP-13707. If kerberos is enabled while HTTP SPNEGO is not (brahma: rev dbb133ccfc00e20622a5dbf7a6e1126fb63d7487)

          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/AdminAuthorizedServlet.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/log/LogLevel.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/jmx/JMXJsonServlet.java
          • (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServer.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/ConfServlet.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10789 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10789/ ) HADOOP-13707 . If kerberos is enabled while HTTP SPNEGO is not (brahma: rev dbb133ccfc00e20622a5dbf7a6e1126fb63d7487) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/AdminAuthorizedServlet.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/log/LogLevel.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/jmx/JMXJsonServlet.java (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServer.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/ConfServlet.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java

            People

            • Assignee:
              yuanbo Yuanbo Liu
              Reporter:
              yuanbo Yuanbo Liu
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development