Details

    • Type: Test
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.6.0
    • Component/s: security, test
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      We should test EZs with KMS.

      1. COMBO.patch
        7 kB
        Alejandro Abdelnur
      2. HDFS-7006.patch
        15 kB
        Alejandro Abdelnur
      3. HDFS-7006.patch
        5 kB
        Alejandro Abdelnur
      4. HDFS-7006.patch
        5 kB
        Alejandro Abdelnur

        Issue Links

          Activity

          Hide
          tucu00 Alejandro Abdelnur added a comment -

          rebased patch.

          Show
          tucu00 Alejandro Abdelnur added a comment - rebased patch.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12667194/HDFS-7006.patch
          against trunk revision df8c84c.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          -1 eclipse:eclipse. The patch failed to build with eclipse:eclipse.

          -1 findbugs. The patch appears to cause Findbugs (version 2.0.3) to fail.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.
          -1 core tests. Failed to build the native portion of hadoop-common prior to running the unit tests in hadoop-hdfs-project/hadoop-hdfs

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7951//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7951//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12667194/HDFS-7006.patch against trunk revision df8c84c. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. -1 eclipse:eclipse . The patch failed to build with eclipse:eclipse. -1 findbugs . The patch appears to cause Findbugs (version 2.0.3) to fail. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . Failed to build the native portion of hadoop-common prior to running the unit tests in hadoop-hdfs-project/hadoop-hdfs +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7951//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7951//console This message is automatically generated.
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          uploading COMBO of HDFS-7006 & HADOOP-11075 to see if this fixes test-patch run.

          Show
          tucu00 Alejandro Abdelnur added a comment - uploading COMBO of HDFS-7006 & HADOOP-11075 to see if this fixes test-patch run.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12667527/COMBO.patch
          against trunk revision 28d99db.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.crypto.key.kms.server.TestKMS
          org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS
          org.apache.hadoop.fs.TestHdfsNativeCodeLoader
          org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCacheRevocation

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7977//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7977//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12667527/COMBO.patch against trunk revision 28d99db. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.crypto.key.kms.server.TestKMS org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS org.apache.hadoop.fs.TestHdfsNativeCodeLoader org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCacheRevocation +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7977//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7977//console This message is automatically generated.
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          testcases pass when run from hadoop source root, they don't when run from hdfs source root (using kms stuff from JARs).

          Jetty WebAppContext has different behavior when the webapp is working out or an expanded directory or out a JAR.

          Looking on how to make MiniKMS to work correctly in both scenarios.

          Show
          tucu00 Alejandro Abdelnur added a comment - testcases pass when run from hadoop source root, they don't when run from hdfs source root (using kms stuff from JARs). Jetty WebAppContext has different behavior when the webapp is working out or an expanded directory or out a JAR. Looking on how to make MiniKMS to work correctly in both scenarios.
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          After a few hours of debugging (courtesy of Jetty handling webapps completely differently when in directory, JAR and WARs) I've sorted this one out.

          I've merged HADOOP-11075 in this patch, it required some tweaks.

          I had to do add a special handling for the webapp in the MiniKDC (because of Jetty's webapp handling).

          I've tested MiniKMS works when consumed from a directory in the classpath and when consumed from Maven local repo as JAR.

          Also, I've verified all necessary JARs are published to the maven repo.

          Show
          tucu00 Alejandro Abdelnur added a comment - After a few hours of debugging (courtesy of Jetty handling webapps completely differently when in directory, JAR and WARs) I've sorted this one out. I've merged HADOOP-11075 in this patch, it required some tweaks. I had to do add a special handling for the webapp in the MiniKDC (because of Jetty's webapp handling). I've tested MiniKMS works when consumed from a directory in the classpath and when consumed from Maven local repo as JAR. Also, I've verified all necessary JARs are published to the maven repo.
          Hide
          andrew.wang Andrew Wang added a comment -

          +1 nice one tucu

          Show
          andrew.wang Andrew Wang added a comment - +1 nice one tucu
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12669143/HDFS-7006.patch
          against trunk revision 8e5d671.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCacheRevocation

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8038//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8038//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12669143/HDFS-7006.patch against trunk revision 8e5d671. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCacheRevocation +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8038//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8038//console This message is automatically generated.
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          test failure unrelated.

          Show
          tucu00 Alejandro Abdelnur added a comment - test failure unrelated.
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          committed to trunk and branch-2.

          Show
          tucu00 Alejandro Abdelnur added a comment - committed to trunk and branch-2.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #683 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/683/)
          HDFS-7006. Test encryption zones with KMS. (Anthony Young-Garner and tucu) (tucu: rev 3e85f5b605b9ccee54aba7b4a683f81734571d60)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZonesWithKMS.java
          • hadoop-common-project/hadoop-kms/src/test/resources/mini-kms-acls-default.xml
          • hadoop-project/pom.xml
          • hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/MiniKMS.java
          • hadoop-common-project/hadoop-kms/pom.xml
          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #683 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/683/ ) HDFS-7006 . Test encryption zones with KMS. (Anthony Young-Garner and tucu) (tucu: rev 3e85f5b605b9ccee54aba7b4a683f81734571d60) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZonesWithKMS.java hadoop-common-project/hadoop-kms/src/test/resources/mini-kms-acls-default.xml hadoop-project/pom.xml hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/MiniKMS.java hadoop-common-project/hadoop-kms/pom.xml hadoop-hdfs-project/hadoop-hdfs/pom.xml hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #1899 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1899/)
          HDFS-7006. Test encryption zones with KMS. (Anthony Young-Garner and tucu) (tucu: rev 3e85f5b605b9ccee54aba7b4a683f81734571d60)

          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • hadoop-common-project/hadoop-kms/pom.xml
          • hadoop-common-project/hadoop-kms/src/test/resources/mini-kms-acls-default.xml
          • hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/MiniKMS.java
          • hadoop-project/pom.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZonesWithKMS.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1899 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1899/ ) HDFS-7006 . Test encryption zones with KMS. (Anthony Young-Garner and tucu) (tucu: rev 3e85f5b605b9ccee54aba7b4a683f81734571d60) hadoop-hdfs-project/hadoop-hdfs/pom.xml hadoop-common-project/hadoop-kms/pom.xml hadoop-common-project/hadoop-kms/src/test/resources/mini-kms-acls-default.xml hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/MiniKMS.java hadoop-project/pom.xml hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZonesWithKMS.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #1874 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1874/)
          HDFS-7006. Test encryption zones with KMS. (Anthony Young-Garner and tucu) (tucu: rev 3e85f5b605b9ccee54aba7b4a683f81734571d60)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-project/pom.xml
          • hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/MiniKMS.java
          • hadoop-common-project/hadoop-kms/pom.xml
          • hadoop-common-project/hadoop-kms/src/test/resources/mini-kms-acls-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZonesWithKMS.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java
          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1874 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1874/ ) HDFS-7006 . Test encryption zones with KMS. (Anthony Young-Garner and tucu) (tucu: rev 3e85f5b605b9ccee54aba7b4a683f81734571d60) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-project/pom.xml hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/MiniKMS.java hadoop-common-project/hadoop-kms/pom.xml hadoop-common-project/hadoop-kms/src/test/resources/mini-kms-acls-default.xml hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZonesWithKMS.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java hadoop-hdfs-project/hadoop-hdfs/pom.xml

            People

            • Assignee:
              tucu00 Alejandro Abdelnur
              Reporter:
              tucu00 Alejandro Abdelnur
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development