Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15280

TestKMS.testWebHDFSProxyUserKerb and TestKMS.testWebHDFSProxyUserSimple fail in trunk

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 3.1.0
    • None
    • None
    • Reviewed

    Description

      I'm seeing these messages on OS X and on Linux.

      [ERROR] Failures:
      [ERROR] TestKMS.testWebHDFSProxyUserKerb:2526->doWebHDFSProxyUserTest:2625->runServer:158->runServer:176 org.apache.hadoop.security.authentication.client.AuthenticationException: Error while authenticating with endpoint: http://localhost:56112/kms/v1/keys?doAs=foo1
      [ERROR] TestKMS.testWebHDFSProxyUserSimple:2531->doWebHDFSProxyUserTest:2625->runServer:158->runServer:176 org.apache.hadoop.security.authentication.client.AuthenticationException: Error while authenticating with endpoint: http://localhost:56206/kms/v1/keys?doAs=foo1 
      

      as well as a recent PreCommit-HADOOP-Build job.

      Attachments

        1. HADOOP-15280.01.patch
          1 kB
          Bharat Viswanadham
        2. HADOOP-15280.00.patch
          0.9 kB
          Bharat Viswanadham

        Issue Links

          Activity

            xiaochen Xiao Chen added a comment - - edited

            Did a quick git bisect and it appears to be HADOOP-12897

            xiaochen Xiao Chen added a comment - - edited Did a quick git bisect and it appears to be  HADOOP-12897

            xiaochen This change is due to different exception message being thrown.

            Fixed test case to address this issue.

            bharat Bharat Viswanadham added a comment - xiaochen This change is due to different exception message being thrown. Fixed test case to address this issue.
            genericqa genericqa added a comment -
            +1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 17s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                  trunk Compile Tests
            +1 mvninstall 17m 29s trunk passed
            +1 compile 14m 11s trunk passed
            +1 checkstyle 0m 22s trunk passed
            +1 mvnsite 0m 28s trunk passed
            +1 shadedclient 10m 55s branch has no errors when building and testing our client artifacts.
            +1 findbugs 0m 33s trunk passed
            +1 javadoc 0m 24s trunk passed
                  Patch Compile Tests
            +1 mvninstall 0m 21s the patch passed
            +1 compile 12m 25s the patch passed
            +1 javac 12m 25s the patch passed
            +1 checkstyle 0m 22s hadoop-common-project/hadoop-kms: The patch generated 0 new + 97 unchanged - 1 fixed = 97 total (was 98)
            +1 mvnsite 0m 29s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedclient 10m 17s patch has no errors when building and testing our client artifacts.
            +1 findbugs 0m 42s the patch passed
            +1 javadoc 0m 22s the patch passed
                  Other Tests
            +1 unit 4m 10s hadoop-kms in the patch passed.
            +1 asflicense 0m 37s The patch does not generate ASF License warnings.
            75m 0s



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f
            JIRA Issue HADOOP-15280
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12913516/HADOOP-15280.00.patch
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux 2dd9e48e5433 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision trunk / 583f459
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_151
            findbugs v3.1.0-RC1
            Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/14280/testReport/
            Max. process+thread count 315 (vs. ulimit of 10000)
            modules C: hadoop-common-project/hadoop-kms U: hadoop-common-project/hadoop-kms
            Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/14280/console
            Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            genericqa genericqa added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests +1 mvninstall 17m 29s trunk passed +1 compile 14m 11s trunk passed +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 28s trunk passed +1 shadedclient 10m 55s branch has no errors when building and testing our client artifacts. +1 findbugs 0m 33s trunk passed +1 javadoc 0m 24s trunk passed       Patch Compile Tests +1 mvninstall 0m 21s the patch passed +1 compile 12m 25s the patch passed +1 javac 12m 25s the patch passed +1 checkstyle 0m 22s hadoop-common-project/hadoop-kms: The patch generated 0 new + 97 unchanged - 1 fixed = 97 total (was 98) +1 mvnsite 0m 29s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 10m 17s patch has no errors when building and testing our client artifacts. +1 findbugs 0m 42s the patch passed +1 javadoc 0m 22s the patch passed       Other Tests +1 unit 4m 10s hadoop-kms in the patch passed. +1 asflicense 0m 37s The patch does not generate ASF License warnings. 75m 0s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f JIRA Issue HADOOP-15280 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12913516/HADOOP-15280.00.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 2dd9e48e5433 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / 583f459 maven version: Apache Maven 3.3.9 Default Java 1.8.0_151 findbugs v3.1.0-RC1 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/14280/testReport/ Max. process+thread count 315 (vs. ulimit of 10000) modules C: hadoop-common-project/hadoop-kms U: hadoop-common-project/hadoop-kms Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/14280/console Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            xiaochen Xiao Chen added a comment -

            Thanks for working on this issue bharatviswa.

            I agree we should change the test. But instead of changing it to validate the wrapper message, we should still validate the original message ("Forbidden") instead of the wrapper message. Otherwise it defeats the purpose of the assertion.

            Maybe we can add a utility method into GenericTestUtils so this can be reused.

            xiaochen Xiao Chen added a comment - Thanks for working on this issue bharatviswa . I agree we should change the test. But instead of changing it to validate the wrapper message, we should still validate the original message ("Forbidden") instead of the wrapper message. Otherwise it defeats the purpose of the assertion. Maybe we can add a utility method into GenericTestUtils so this can be reused.
            bharat Bharat Viswanadham added a comment - - edited

            xiaochen

            As we are wrapping the exception with a new message, checking the original cause exception will get original message. I have not added any utility methods to GenericTestUtils, let me know if you want to do in a different approach, other than proposed in the patch.
            Attached v02 patch.

            bharat Bharat Viswanadham added a comment - - edited xiaochen As we are wrapping the exception with a new message, checking the original cause exception will get original message. I have not added any utility methods to GenericTestUtils, let me know if you want to do in a different approach, other than proposed in the patch. Attached v02 patch.
            genericqa genericqa added a comment -
            +1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 16s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                  trunk Compile Tests
            +1 mvninstall 17m 33s trunk passed
            +1 compile 14m 5s trunk passed
            +1 checkstyle 0m 26s trunk passed
            +1 mvnsite 0m 30s trunk passed
            +1 shadedclient 11m 3s branch has no errors when building and testing our client artifacts.
            +1 findbugs 0m 33s trunk passed
            +1 javadoc 0m 23s trunk passed
                  Patch Compile Tests
            +1 mvninstall 0m 20s the patch passed
            +1 compile 12m 43s the patch passed
            +1 javac 12m 43s the patch passed
            +1 checkstyle 0m 22s hadoop-common-project/hadoop-kms: The patch generated 0 new + 97 unchanged - 1 fixed = 97 total (was 98)
            +1 mvnsite 0m 29s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedclient 9m 59s patch has no errors when building and testing our client artifacts.
            +1 findbugs 0m 42s the patch passed
            +1 javadoc 0m 23s the patch passed
                  Other Tests
            +1 unit 3m 59s hadoop-kms in the patch passed.
            +1 asflicense 0m 33s The patch does not generate ASF License warnings.
            74m 51s



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f
            JIRA Issue HADOOP-15280
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12913627/HADOOP-15280.01.patch
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux 360098f02dee 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision trunk / 7ef4d94
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_151
            findbugs v3.1.0-RC1
            Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/14284/testReport/
            Max. process+thread count 319 (vs. ulimit of 10000)
            modules C: hadoop-common-project/hadoop-kms U: hadoop-common-project/hadoop-kms
            Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/14284/console
            Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            genericqa genericqa added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests +1 mvninstall 17m 33s trunk passed +1 compile 14m 5s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 30s trunk passed +1 shadedclient 11m 3s branch has no errors when building and testing our client artifacts. +1 findbugs 0m 33s trunk passed +1 javadoc 0m 23s trunk passed       Patch Compile Tests +1 mvninstall 0m 20s the patch passed +1 compile 12m 43s the patch passed +1 javac 12m 43s the patch passed +1 checkstyle 0m 22s hadoop-common-project/hadoop-kms: The patch generated 0 new + 97 unchanged - 1 fixed = 97 total (was 98) +1 mvnsite 0m 29s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 9m 59s patch has no errors when building and testing our client artifacts. +1 findbugs 0m 42s the patch passed +1 javadoc 0m 23s the patch passed       Other Tests +1 unit 3m 59s hadoop-kms in the patch passed. +1 asflicense 0m 33s The patch does not generate ASF License warnings. 74m 51s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f JIRA Issue HADOOP-15280 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12913627/HADOOP-15280.01.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 360098f02dee 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / 7ef4d94 maven version: Apache Maven 3.3.9 Default Java 1.8.0_151 findbugs v3.1.0-RC1 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/14284/testReport/ Max. process+thread count 319 (vs. ulimit of 10000) modules C: hadoop-common-project/hadoop-kms U: hadoop-common-project/hadoop-kms Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/14284/console Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            xiaochen Xiao Chen added a comment -

            I was more thinking of walking into the cause of the exception and check cause there in the util. But don't feel strongly. Let's fix trunk tests for now. +1

            xiaochen Xiao Chen added a comment - I was more thinking of walking into the cause of the exception and check cause there in the util. But don't feel strongly. Let's fix trunk tests for now. +1
            xiaochen Xiao Chen added a comment -

            Committed to trunk.

            Thanks rchiang for filing the Jira and bharatviswa for the fix.

            xiaochen Xiao Chen added a comment - Committed to trunk. Thanks rchiang for filing the Jira and bharatviswa for the fix.
            hudson Hudson added a comment -

            SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13798 (See https://builds.apache.org/job/Hadoop-trunk-Commit/13798/)
            HADOOP-15280. TestKMS.testWebHDFSProxyUserKerb and (xiao: rev a906a226458a0b4c4b2df61d9bcf375a1d194925)

            • (edit) hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
            hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13798 (See https://builds.apache.org/job/Hadoop-trunk-Commit/13798/ ) HADOOP-15280 . TestKMS.testWebHDFSProxyUserKerb and (xiao: rev a906a226458a0b4c4b2df61d9bcf375a1d194925) (edit) hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
            arp Arpit Agarwal added a comment -

            Thanks for taking care of this bharatviswa and xiaochen.

            arp Arpit Agarwal added a comment - Thanks for taking care of this bharatviswa and xiaochen .

            People

              bharat Bharat Viswanadham
              rchiang Ray Chiang
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: