Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0-alpha-1, 2.3.0
    • None
    • None
    • Reviewed

    Description

      I've tried to make a patch for HBASE-21879, most of work seems to be fine, but the trouble is:
      HFileBlock#validateChecksum can only accept ByteBuffer as its input, while after the HBASE-21916, we will use an ourself-defined ByteBuff (which can be SingleByteBuff or MultiByteBuff).

      Now, need to create our own ByteBuff checksum validation method, should not be so hard but an separate issue will be more clearer.

      Attachments

        1. HBASE-21917.addendum.HBASE-21879.v7.patch
          2 kB
          Zheng Hu
        2. HBASE-21917.HBASE-21879.v5.patch
          18 kB
          Zheng Hu
        3. HBASE-21917.HBASE-21879.v6.patch
          19 kB
          Zheng Hu
        4. HBASE-21917.v1.patch
          10 kB
          Zheng Hu
        5. HBASE-21917.v2.patch
          10 kB
          Zheng Hu
        6. HBASE-21917.v3.patch
          11 kB
          Zheng Hu
        7. HBASE-21917.v4.patch
          11 kB
          Zheng Hu

        Issue Links

          Activity

            openinx Zheng Hu added a comment -

            Oh, our default bytesPerChecksum is 16KB. while there's no ByteBuffer update(..) inteface for java.util.zip.Checksum, which means we still need to allocate an 16KB on-heap byte[] to validate the checksum. FYI stack, anoop.hbase, ram_krish

              public static int getBytesPerChecksum(Configuration conf) {
                return conf.getInt(HConstants.BYTES_PER_CHECKSUM,
                                   HFile.DEFAULT_BYTES_PER_CHECKSUM);
              }
            
            openinx Zheng Hu added a comment - Oh, our default bytesPerChecksum is 16KB. while there's no ByteBuffer update(..) inteface for java.util.zip.Checksum, which means we still need to allocate an 16KB on-heap byte[] to validate the checksum. FYI stack , anoop.hbase , ram_krish public static int getBytesPerChecksum(Configuration conf) { return conf.getInt(HConstants.BYTES_PER_CHECKSUM, HFile.DEFAULT_BYTES_PER_CHECKSUM); }
            openinx Zheng Hu added a comment -

            Maybe we can new a very small heap size such as 256, and update(..) many times until we update all the 16KB bytes.

            openinx Zheng Hu added a comment - Maybe we can new a very small heap size such as 256, and update(..) many times until we update all the 16KB bytes.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 10s Docker mode activated.
                  Prechecks
            +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
            +1 @author 0m 0s The patch does not contain any @author tags.
            -0 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                  master Compile Tests
            +1 mvninstall 4m 46s master passed
            +1 compile 2m 2s master passed
            +1 checkstyle 1m 15s master passed
            +1 shadedjars 4m 31s branch has no errors when building our shaded downstream artifacts.
            +1 findbugs 2m 22s master passed
            +1 javadoc 0m 29s master passed
                  Patch Compile Tests
            +1 mvninstall 4m 39s the patch passed
            +1 compile 2m 2s the patch passed
            -1 javac 2m 2s hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188)
            -1 checkstyle 1m 15s hbase-server: The patch generated 1 new + 27 unchanged - 0 fixed = 28 total (was 27)
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedjars 4m 32s patch has no errors when building our shaded downstream artifacts.
            +1 hadoopcheck 9m 44s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0.
            +1 findbugs 2m 35s the patch passed
            +1 javadoc 0m 30s the patch passed
                  Other Tests
            +1 unit 137m 18s hbase-server in the patch passed.
            +1 asflicense 0m 22s The patch does not generate ASF License warnings.
            179m 0s



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b
            JIRA Issue HBASE-21917
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12959097/HBASE-21917.v2.patch
            Optional Tests dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
            uname Linux 9d53effccd6c 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 GNU/Linux
            Build tool maven
            Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
            git revision master / 7be71c0f55
            maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
            Default Java 1.8.0_181
            findbugs v3.1.0-RC3
            javac https://builds.apache.org/job/PreCommit-HBASE-Build/16020/artifact/patchprocess/diff-compile-javac-hbase-server.txt
            checkstyle https://builds.apache.org/job/PreCommit-HBASE-Build/16020/artifact/patchprocess/diff-checkstyle-hbase-server.txt
            Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/16020/testReport/
            Max. process+thread count 4858 (vs. ulimit of 10000)
            modules C: hbase-server U: hbase-server
            Console output https://builds.apache.org/job/PreCommit-HBASE-Build/16020/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated.       Prechecks +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. -0 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       master Compile Tests +1 mvninstall 4m 46s master passed +1 compile 2m 2s master passed +1 checkstyle 1m 15s master passed +1 shadedjars 4m 31s branch has no errors when building our shaded downstream artifacts. +1 findbugs 2m 22s master passed +1 javadoc 0m 29s master passed       Patch Compile Tests +1 mvninstall 4m 39s the patch passed +1 compile 2m 2s the patch passed -1 javac 2m 2s hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188) -1 checkstyle 1m 15s hbase-server: The patch generated 1 new + 27 unchanged - 0 fixed = 28 total (was 27) +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedjars 4m 32s patch has no errors when building our shaded downstream artifacts. +1 hadoopcheck 9m 44s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. +1 findbugs 2m 35s the patch passed +1 javadoc 0m 30s the patch passed       Other Tests +1 unit 137m 18s hbase-server in the patch passed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 179m 0s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b JIRA Issue HBASE-21917 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12959097/HBASE-21917.v2.patch Optional Tests dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile uname Linux 9d53effccd6c 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh git revision master / 7be71c0f55 maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) Default Java 1.8.0_181 findbugs v3.1.0-RC3 javac https://builds.apache.org/job/PreCommit-HBASE-Build/16020/artifact/patchprocess/diff-compile-javac-hbase-server.txt checkstyle https://builds.apache.org/job/PreCommit-HBASE-Build/16020/artifact/patchprocess/diff-checkstyle-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/16020/testReport/ Max. process+thread count 4858 (vs. ulimit of 10000) modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/16020/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 22s Docker mode activated.
                  Prechecks
            +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
            +1 @author 0m 0s The patch does not contain any @author tags.
            -0 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                  master Compile Tests
            +1 mvninstall 4m 5s master passed
            +1 compile 1m 52s master passed
            +1 checkstyle 1m 5s master passed
            +1 shadedjars 4m 11s branch has no errors when building our shaded downstream artifacts.
            +1 findbugs 2m 25s master passed
            +1 javadoc 0m 29s master passed
                  Patch Compile Tests
            +1 mvninstall 4m 12s the patch passed
            +1 compile 1m 51s the patch passed
            -1 javac 1m 51s hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188)
            -1 checkstyle 1m 6s hbase-server: The patch generated 1 new + 27 unchanged - 0 fixed = 28 total (was 27)
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedjars 4m 10s patch has no errors when building our shaded downstream artifacts.
            +1 hadoopcheck 8m 39s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0.
            +1 findbugs 2m 29s the patch passed
            +1 javadoc 0m 30s the patch passed
                  Other Tests
            -1 unit 259m 22s hbase-server in the patch failed.
            +1 asflicense 0m 26s The patch does not generate ASF License warnings.
            297m 46s



            Reason Tests
            Failed junit tests hadoop.hbase.master.procedure.TestServerCrashProcedure
              hadoop.hbase.client.TestFromClientSide3
              hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory
              hadoop.hbase.master.procedure.TestServerCrashProcedureWithReplicas
              hadoop.hbase.client.TestAdmin1
              hadoop.hbase.client.TestFromClientSide
              hadoop.hbase.client.TestFromClientSideWithCoprocessor



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b
            JIRA Issue HBASE-21917
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12959073/HBASE-21917.v1.patch
            Optional Tests dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
            uname Linux 26fa911b3c2d 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
            Build tool maven
            Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
            git revision master / 40e1d9174e
            maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
            Default Java 1.8.0_181
            findbugs v3.1.0-RC3
            javac https://builds.apache.org/job/PreCommit-HBASE-Build/16016/artifact/patchprocess/diff-compile-javac-hbase-server.txt
            checkstyle https://builds.apache.org/job/PreCommit-HBASE-Build/16016/artifact/patchprocess/diff-checkstyle-hbase-server.txt
            unit https://builds.apache.org/job/PreCommit-HBASE-Build/16016/artifact/patchprocess/patch-unit-hbase-server.txt
            Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/16016/testReport/
            Max. process+thread count 5315 (vs. ulimit of 10000)
            modules C: hbase-server U: hbase-server
            Console output https://builds.apache.org/job/PreCommit-HBASE-Build/16016/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated.       Prechecks +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. -0 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       master Compile Tests +1 mvninstall 4m 5s master passed +1 compile 1m 52s master passed +1 checkstyle 1m 5s master passed +1 shadedjars 4m 11s branch has no errors when building our shaded downstream artifacts. +1 findbugs 2m 25s master passed +1 javadoc 0m 29s master passed       Patch Compile Tests +1 mvninstall 4m 12s the patch passed +1 compile 1m 51s the patch passed -1 javac 1m 51s hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188) -1 checkstyle 1m 6s hbase-server: The patch generated 1 new + 27 unchanged - 0 fixed = 28 total (was 27) +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedjars 4m 10s patch has no errors when building our shaded downstream artifacts. +1 hadoopcheck 8m 39s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. +1 findbugs 2m 29s the patch passed +1 javadoc 0m 30s the patch passed       Other Tests -1 unit 259m 22s hbase-server in the patch failed. +1 asflicense 0m 26s The patch does not generate ASF License warnings. 297m 46s Reason Tests Failed junit tests hadoop.hbase.master.procedure.TestServerCrashProcedure   hadoop.hbase.client.TestFromClientSide3   hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory   hadoop.hbase.master.procedure.TestServerCrashProcedureWithReplicas   hadoop.hbase.client.TestAdmin1   hadoop.hbase.client.TestFromClientSide   hadoop.hbase.client.TestFromClientSideWithCoprocessor Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b JIRA Issue HBASE-21917 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12959073/HBASE-21917.v1.patch Optional Tests dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile uname Linux 26fa911b3c2d 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 40e1d9174e maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) Default Java 1.8.0_181 findbugs v3.1.0-RC3 javac https://builds.apache.org/job/PreCommit-HBASE-Build/16016/artifact/patchprocess/diff-compile-javac-hbase-server.txt checkstyle https://builds.apache.org/job/PreCommit-HBASE-Build/16016/artifact/patchprocess/diff-checkstyle-hbase-server.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/16016/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/16016/testReport/ Max. process+thread count 5315 (vs. ulimit of 10000) modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/16016/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 13s Docker mode activated.
                  Prechecks
            +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
            +1 @author 0m 0s The patch does not contain any @author tags.
            -0 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                  master Compile Tests
            +1 mvninstall 4m 24s master passed
            +1 compile 2m 3s master passed
            +1 checkstyle 1m 4s master passed
            +1 shadedjars 4m 26s branch has no errors when building our shaded downstream artifacts.
            +1 findbugs 2m 22s master passed
            +1 javadoc 0m 31s master passed
                  Patch Compile Tests
            +1 mvninstall 4m 24s the patch passed
            +1 compile 2m 5s the patch passed
            -1 javac 2m 5s hbase-server generated 2 new + 186 unchanged - 2 fixed = 188 total (was 188)
            -1 checkstyle 1m 9s hbase-server: The patch generated 1 new + 27 unchanged - 0 fixed = 28 total (was 27)
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedjars 4m 13s patch has no errors when building our shaded downstream artifacts.
            +1 hadoopcheck 8m 50s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0.
            +1 findbugs 2m 30s the patch passed
            +1 javadoc 0m 27s the patch passed
                  Other Tests
            -1 unit 132m 15s hbase-server in the patch failed.
            +1 asflicense 0m 28s The patch does not generate ASF License warnings.
            171m 51s



            Reason Tests
            Failed junit tests hadoop.hbase.master.TestServerCrashProcedureCarryingMetaStuck



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b
            JIRA Issue HBASE-21917
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12959112/HBASE-21917.v3.patch
            Optional Tests dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
            uname Linux c93e02471fdc 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
            Build tool maven
            Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
            git revision master / 6176471957
            maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
            Default Java 1.8.0_181
            findbugs v3.1.0-RC3
            javac https://builds.apache.org/job/PreCommit-HBASE-Build/16025/artifact/patchprocess/diff-compile-javac-hbase-server.txt
            checkstyle https://builds.apache.org/job/PreCommit-HBASE-Build/16025/artifact/patchprocess/diff-checkstyle-hbase-server.txt
            unit https://builds.apache.org/job/PreCommit-HBASE-Build/16025/artifact/patchprocess/patch-unit-hbase-server.txt
            Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/16025/testReport/
            Max. process+thread count 5013 (vs. ulimit of 10000)
            modules C: hbase-server U: hbase-server
            Console output https://builds.apache.org/job/PreCommit-HBASE-Build/16025/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated.       Prechecks +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. -0 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       master Compile Tests +1 mvninstall 4m 24s master passed +1 compile 2m 3s master passed +1 checkstyle 1m 4s master passed +1 shadedjars 4m 26s branch has no errors when building our shaded downstream artifacts. +1 findbugs 2m 22s master passed +1 javadoc 0m 31s master passed       Patch Compile Tests +1 mvninstall 4m 24s the patch passed +1 compile 2m 5s the patch passed -1 javac 2m 5s hbase-server generated 2 new + 186 unchanged - 2 fixed = 188 total (was 188) -1 checkstyle 1m 9s hbase-server: The patch generated 1 new + 27 unchanged - 0 fixed = 28 total (was 27) +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedjars 4m 13s patch has no errors when building our shaded downstream artifacts. +1 hadoopcheck 8m 50s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. +1 findbugs 2m 30s the patch passed +1 javadoc 0m 27s the patch passed       Other Tests -1 unit 132m 15s hbase-server in the patch failed. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 171m 51s Reason Tests Failed junit tests hadoop.hbase.master.TestServerCrashProcedureCarryingMetaStuck Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b JIRA Issue HBASE-21917 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12959112/HBASE-21917.v3.patch Optional Tests dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile uname Linux c93e02471fdc 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh git revision master / 6176471957 maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) Default Java 1.8.0_181 findbugs v3.1.0-RC3 javac https://builds.apache.org/job/PreCommit-HBASE-Build/16025/artifact/patchprocess/diff-compile-javac-hbase-server.txt checkstyle https://builds.apache.org/job/PreCommit-HBASE-Build/16025/artifact/patchprocess/diff-checkstyle-hbase-server.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/16025/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/16025/testReport/ Max. process+thread count 5013 (vs. ulimit of 10000) modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/16025/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 13s Docker mode activated.
                  Prechecks
            +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
            +1 @author 0m 0s The patch does not contain any @author tags.
            -0 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                  master Compile Tests
            +1 mvninstall 5m 15s master passed
            +1 compile 2m 22s master passed
            +1 checkstyle 1m 30s master passed
            +1 shadedjars 5m 25s branch has no errors when building our shaded downstream artifacts.
            +1 findbugs 2m 52s master passed
            +1 javadoc 0m 38s master passed
                  Patch Compile Tests
            +1 mvninstall 5m 18s the patch passed
            +1 compile 2m 25s the patch passed
            -1 javac 2m 25s hbase-server generated 2 new + 186 unchanged - 2 fixed = 188 total (was 188)
            -1 checkstyle 1m 22s hbase-server: The patch generated 2 new + 27 unchanged - 0 fixed = 29 total (was 27)
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedjars 5m 9s patch has no errors when building our shaded downstream artifacts.
            +1 hadoopcheck 11m 4s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0.
            +1 findbugs 3m 3s the patch passed
            +1 javadoc 0m 38s the patch passed
                  Other Tests
            -1 unit 144m 6s hbase-server in the patch failed.
            +1 asflicense 0m 25s The patch does not generate ASF License warnings.
            192m 20s



            Reason Tests
            Failed junit tests hadoop.hbase.master.TestServerCrashProcedureCarryingMetaStuck



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b
            JIRA Issue HBASE-21917
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12959111/HBASE-21917.v3.patch
            Optional Tests dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
            uname Linux e0be646dd6c9 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 GNU/Linux
            Build tool maven
            Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
            git revision master / 6176471957
            maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
            Default Java 1.8.0_181
            findbugs v3.1.0-RC3
            javac https://builds.apache.org/job/PreCommit-HBASE-Build/16024/artifact/patchprocess/diff-compile-javac-hbase-server.txt
            checkstyle https://builds.apache.org/job/PreCommit-HBASE-Build/16024/artifact/patchprocess/diff-checkstyle-hbase-server.txt
            unit https://builds.apache.org/job/PreCommit-HBASE-Build/16024/artifact/patchprocess/patch-unit-hbase-server.txt
            Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/16024/testReport/
            Max. process+thread count 5247 (vs. ulimit of 10000)
            modules C: hbase-server U: hbase-server
            Console output https://builds.apache.org/job/PreCommit-HBASE-Build/16024/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated.       Prechecks +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. -0 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       master Compile Tests +1 mvninstall 5m 15s master passed +1 compile 2m 22s master passed +1 checkstyle 1m 30s master passed +1 shadedjars 5m 25s branch has no errors when building our shaded downstream artifacts. +1 findbugs 2m 52s master passed +1 javadoc 0m 38s master passed       Patch Compile Tests +1 mvninstall 5m 18s the patch passed +1 compile 2m 25s the patch passed -1 javac 2m 25s hbase-server generated 2 new + 186 unchanged - 2 fixed = 188 total (was 188) -1 checkstyle 1m 22s hbase-server: The patch generated 2 new + 27 unchanged - 0 fixed = 29 total (was 27) +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedjars 5m 9s patch has no errors when building our shaded downstream artifacts. +1 hadoopcheck 11m 4s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. +1 findbugs 3m 3s the patch passed +1 javadoc 0m 38s the patch passed       Other Tests -1 unit 144m 6s hbase-server in the patch failed. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 192m 20s Reason Tests Failed junit tests hadoop.hbase.master.TestServerCrashProcedureCarryingMetaStuck Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b JIRA Issue HBASE-21917 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12959111/HBASE-21917.v3.patch Optional Tests dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile uname Linux e0be646dd6c9 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 6176471957 maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) Default Java 1.8.0_181 findbugs v3.1.0-RC3 javac https://builds.apache.org/job/PreCommit-HBASE-Build/16024/artifact/patchprocess/diff-compile-javac-hbase-server.txt checkstyle https://builds.apache.org/job/PreCommit-HBASE-Build/16024/artifact/patchprocess/diff-checkstyle-hbase-server.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/16024/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/16024/testReport/ Max. process+thread count 5247 (vs. ulimit of 10000) modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/16024/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            stack Michael Stack added a comment -

            Thanks for hacking on this openinx. IIRC, we want native checksum checking to cut in. Is that happening here? If it is, small buffers means we cross the native divide many times – each of which costs – rather than a few times?

            stack Michael Stack added a comment - Thanks for hacking on this openinx . IIRC, we want native checksum checking to cut in. Is that happening here? If it is, small buffers means we cross the native divide many times – each of which costs – rather than a few times?
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 13s Docker mode activated.
                  Prechecks
            +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
            +1 @author 0m 0s The patch does not contain any @author tags.
            -0 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                  master Compile Tests
            +1 mvninstall 5m 35s master passed
            +1 compile 2m 26s master passed
            +1 checkstyle 1m 26s master passed
            +1 shadedjars 5m 12s branch has no errors when building our shaded downstream artifacts.
            +1 findbugs 2m 58s master passed
            +1 javadoc 0m 39s master passed
                  Patch Compile Tests
            +1 mvninstall 5m 15s the patch passed
            +1 compile 2m 23s the patch passed
            -1 javac 2m 23s hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188)
            +1 checkstyle 1m 26s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedjars 4m 59s patch has no errors when building our shaded downstream artifacts.
            +1 hadoopcheck 10m 40s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0.
            +1 findbugs 2m 58s the patch passed
            +1 javadoc 0m 37s the patch passed
                  Other Tests
            -1 unit 133m 9s hbase-server in the patch failed.
            +1 asflicense 0m 22s The patch does not generate ASF License warnings.
            180m 52s



            Reason Tests
            Failed junit tests hadoop.hbase.master.TestServerCrashProcedureCarryingMetaStuck



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b
            JIRA Issue HBASE-21917
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12959134/HBASE-21917.v4.patch
            Optional Tests dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
            uname Linux 7fba71c33bdf 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 GNU/Linux
            Build tool maven
            Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
            git revision master / 593745e8ac
            maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
            Default Java 1.8.0_181
            findbugs v3.1.0-RC3
            javac https://builds.apache.org/job/PreCommit-HBASE-Build/16027/artifact/patchprocess/diff-compile-javac-hbase-server.txt
            unit https://builds.apache.org/job/PreCommit-HBASE-Build/16027/artifact/patchprocess/patch-unit-hbase-server.txt
            Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/16027/testReport/
            Max. process+thread count 5114 (vs. ulimit of 10000)
            modules C: hbase-server U: hbase-server
            Console output https://builds.apache.org/job/PreCommit-HBASE-Build/16027/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated.       Prechecks +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. -0 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       master Compile Tests +1 mvninstall 5m 35s master passed +1 compile 2m 26s master passed +1 checkstyle 1m 26s master passed +1 shadedjars 5m 12s branch has no errors when building our shaded downstream artifacts. +1 findbugs 2m 58s master passed +1 javadoc 0m 39s master passed       Patch Compile Tests +1 mvninstall 5m 15s the patch passed +1 compile 2m 23s the patch passed -1 javac 2m 23s hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188) +1 checkstyle 1m 26s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedjars 4m 59s patch has no errors when building our shaded downstream artifacts. +1 hadoopcheck 10m 40s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. +1 findbugs 2m 58s the patch passed +1 javadoc 0m 37s the patch passed       Other Tests -1 unit 133m 9s hbase-server in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 180m 52s Reason Tests Failed junit tests hadoop.hbase.master.TestServerCrashProcedureCarryingMetaStuck Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b JIRA Issue HBASE-21917 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12959134/HBASE-21917.v4.patch Optional Tests dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile uname Linux 7fba71c33bdf 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh git revision master / 593745e8ac maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) Default Java 1.8.0_181 findbugs v3.1.0-RC3 javac https://builds.apache.org/job/PreCommit-HBASE-Build/16027/artifact/patchprocess/diff-compile-javac-hbase-server.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/16027/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/16027/testReport/ Max. process+thread count 5114 (vs. ulimit of 10000) modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/16027/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            openinx Zheng Hu added a comment -

            IIRC, we want native checksum checking to cut in. Is that happening here? If it is, small buffers means we cross the native divide many times – each of which costs – rather than a few times?

            Thanks stack, yeah, you are right. before this patch , the hadoop's verifyChunkedSums will try to use native checksum checking if NativeCrc32 is available. but the bad news is: NativeCrc32 is not a public class in hadoop-common, our HBase has no access. so i can only uniform those logics into a general checksum method as the patch described. BTW, the hadoop-common's checksum looks like this:

              public void verifyChunkedSums(ByteBuffer data, ByteBuffer checksums,
                  String fileName, long basePos)
              throws ChecksumException {
                if (type.size == 0) return;
                
                if (data.hasArray() && checksums.hasArray()) {
                  verifyChunkedSums(
                      data.array(), data.arrayOffset() + data.position(), data.remaining(),
                      checksums.array(), checksums.arrayOffset() + checksums.position(),
                      fileName, basePos); // -------> private method, no access
                  return;
                }
                if (NativeCrc32.isAvailable()) {
                  NativeCrc32.verifyChunkedSums(bytesPerChecksum, type.id, checksums, data,
                      fileName, basePos); ------->  NativeCrc32 is not public, no access. 
                  return;
                }
               
                // ...general  ByteBuffer checksum verify logic...
            
            openinx Zheng Hu added a comment - IIRC, we want native checksum checking to cut in. Is that happening here? If it is, small buffers means we cross the native divide many times – each of which costs – rather than a few times? Thanks stack , yeah, you are right. before this patch , the hadoop's verifyChunkedSums will try to use native checksum checking if NativeCrc32 is available. but the bad news is: NativeCrc32 is not a public class in hadoop-common, our HBase has no access. so i can only uniform those logics into a general checksum method as the patch described. BTW, the hadoop-common's checksum looks like this: public void verifyChunkedSums(ByteBuffer data, ByteBuffer checksums, String fileName, long basePos) throws ChecksumException { if (type.size == 0) return ; if (data.hasArray() && checksums.hasArray()) { verifyChunkedSums( data.array(), data.arrayOffset() + data.position(), data.remaining(), checksums.array(), checksums.arrayOffset() + checksums.position(), fileName, basePos); // -------> private method, no access return ; } if (NativeCrc32.isAvailable()) { NativeCrc32.verifyChunkedSums(bytesPerChecksum, type.id, checksums, data, fileName, basePos); -------> NativeCrc32 is not public , no access. return ; } // ...general ByteBuffer checksum verify logic...
            stack Michael Stack added a comment - - edited

            Can we bring it local?

            (IIRC when I looked long time ago, it isn't really possible. If we could bring it local we could use it in meantime while worked on changing access downstream – thanks openinx)

            stack Michael Stack added a comment - - edited Can we bring it local? (IIRC when I looked long time ago, it isn't really possible. If we could bring it local we could use it in meantime while worked on changing access downstream – thanks openinx )
            openinx Zheng Hu added a comment -

            If we could bring it local we could use it in meantime while worked on changing access downstream

            Got your mind, thanks stack. but I'm worrying that after we bring those native checksum methods into hbase, those methods can still only accept ByteBuffer or byte[] ( data&checksum ) arguments, so we may need to copy few bytes from ByteBuff (NOT nio.ByteBuffer) each time to update the checksum and the heavy native cost happen(as you said before). Another way will be: verify 16KB data each time, no native call cost but the young gc issue may still exists.
            So I prefer to keep the current general small byte[] checksum updating in patch.v4.

            openinx Zheng Hu added a comment - If we could bring it local we could use it in meantime while worked on changing access downstream Got your mind, thanks stack . but I'm worrying that after we bring those native checksum methods into hbase, those methods can still only accept ByteBuffer or byte[] ( data&checksum ) arguments, so we may need to copy few bytes from ByteBuff (NOT nio.ByteBuffer) each time to update the checksum and the heavy native cost happen(as you said before). Another way will be: verify 16KB data each time, no native call cost but the young gc issue may still exists. So I prefer to keep the current general small byte[] checksum updating in patch.v4.
            anoop.hbase Anoop Sam John added a comment -

            so we may need to copy few bytes from ByteBuff (NOT nio.ByteBuffer) each time to update the checksum

            Well one thing which can help here. See ByteBuff#asSubByteBuffer(int offset, int length, ObjectIntPair<ByteBuffer> pair).
            This API will convert a ByteBuff into an NIO BB. When it is an MBB, and we need one NIO BB from it from offset to len, it can so happen that this region of bytes in a single BB. If so this API will not do any copy but directly give that individual BB. Only when the region crosses 2 BB, we will end up in copying. In BC we have 4 MB sized BBs and what we need as part of HFile block is 64 KB, the chances that this across BBs block may not be that big!
            Will see the patch now

            anoop.hbase Anoop Sam John added a comment - so we may need to copy few bytes from ByteBuff (NOT nio.ByteBuffer) each time to update the checksum Well one thing which can help here. See ByteBuff#asSubByteBuffer(int offset, int length, ObjectIntPair<ByteBuffer> pair). This API will convert a ByteBuff into an NIO BB. When it is an MBB, and we need one NIO BB from it from offset to len, it can so happen that this region of bytes in a single BB. If so this API will not do any copy but directly give that individual BB. Only when the region crosses 2 BB, we will end up in copying. In BC we have 4 MB sized BBs and what we need as part of HFile block is 64 KB, the chances that this across BBs block may not be that big! Will see the patch now
            anoop.hbase Anoop Sam John added a comment -
            public void verifyChunkedSums(ByteBuffer data, ByteBuffer checksums,
                  String fileName, long basePos)
              throws ChecksumException {
                if (type.size == 0) return;
                
                if (data.hasArray() && checksums.hasArray()) {
                  verifyChunkedSums(
                      data.array(), data.arrayOffset() + data.position(), data.remaining(),
                      checksums.array(), checksums.arrayOffset() + checksums.position(),
                      fileName, basePos); // -------> private method, no access
                  return;
                }
                if (NativeCrc32.isAvailable()) {
                  NativeCrc32.verifyChunkedSums(bytesPerChecksum, type.id, checksums, data,
                      fileName, basePos); ------->  NativeCrc32 is not public, no access. 
                  return;
                }
            

            Based on this code in Hadoop, if we can pass ByteBuffer to this Hadoop API and that BB is DBB , we will not go into 1st if block.l But if native CRC is available (if so, we wanted to make use of it), Hadoop can call it using the BB that we pass. Native API calls means it would work with DBB with out any copy. So if we used ByteBuff#asSubByteBuffer API, and mostly it will not incur any copy (as explained above) we can make use of native API with out any extra on heap copy and GC over head no?

            anoop.hbase Anoop Sam John added a comment - public void verifyChunkedSums(ByteBuffer data, ByteBuffer checksums, String fileName, long basePos) throws ChecksumException { if (type.size == 0) return ; if (data.hasArray() && checksums.hasArray()) { verifyChunkedSums( data.array(), data.arrayOffset() + data.position(), data.remaining(), checksums.array(), checksums.arrayOffset() + checksums.position(), fileName, basePos); // -------> private method, no access return ; } if (NativeCrc32.isAvailable()) { NativeCrc32.verifyChunkedSums(bytesPerChecksum, type.id, checksums, data, fileName, basePos); -------> NativeCrc32 is not public , no access. return ; } Based on this code in Hadoop, if we can pass ByteBuffer to this Hadoop API and that BB is DBB , we will not go into 1st if block.l But if native CRC is available (if so, we wanted to make use of it), Hadoop can call it using the BB that we pass. Native API calls means it would work with DBB with out any copy. So if we used ByteBuff#asSubByteBuffer API, and mostly it will not incur any copy (as explained above) we can make use of native API with out any extra on heap copy and GC over head no?
            hudson Hudson added a comment -

            Results for branch HBASE-21917
            build #1 on builds.a.o: -1 overall


            details (if available):

            -1 general checks
            – For more information see general report

            -1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            -1 jdk8 hadoop3 checks
            – For more information see jdk8 (hadoop3) report

            -1 source release artifact
            – See build output for details.

            -1 client integration test
            – Something went wrong with this stage, check relevant console output.

            hudson Hudson added a comment - Results for branch HBASE-21917 build #1 on builds.a.o : -1 overall details (if available): -1 general checks – For more information see general report -1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report -1 jdk8 hadoop3 checks – For more information see jdk8 (hadoop3) report -1 source release artifact – See build output for details. -1 client integration test – Something went wrong with this stage, check relevant console output .
            hudson Hudson added a comment -

            Results for branch HBASE-21917
            build #2 on builds.a.o: -1 overall


            details (if available):

            -1 general checks
            – For more information see general report

            -1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            -1 jdk8 hadoop3 checks
            – For more information see jdk8 (hadoop3) report

            -1 source release artifact
            – See build output for details.

            -1 client integration test
            – Something went wrong with this stage, check relevant console output.

            hudson Hudson added a comment - Results for branch HBASE-21917 build #2 on builds.a.o : -1 overall details (if available): -1 general checks – For more information see general report -1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report -1 jdk8 hadoop3 checks – For more information see jdk8 (hadoop3) report -1 source release artifact – See build output for details. -1 client integration test – Something went wrong with this stage, check relevant console output .
            hudson Hudson added a comment -

            Results for branch HBASE-21917
            build #3 on builds.a.o: -1 overall


            details (if available):

            -1 general checks
            – For more information see general report

            -1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            -1 jdk8 hadoop3 checks
            – For more information see jdk8 (hadoop3) report

            -1 source release artifact
            – See build output for details.

            -1 client integration test
            – Something went wrong with this stage, check relevant console output.

            hudson Hudson added a comment - Results for branch HBASE-21917 build #3 on builds.a.o : -1 overall details (if available): -1 general checks – For more information see general report -1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report -1 jdk8 hadoop3 checks – For more information see jdk8 (hadoop3) report -1 source release artifact – See build output for details. -1 client integration test – Something went wrong with this stage, check relevant console output .
            hudson Hudson added a comment -

            Results for branch HBASE-21917
            build #4 on builds.a.o: -1 overall


            details (if available):

            -1 general checks
            – For more information see general report

            -1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            -1 jdk8 hadoop3 checks
            – For more information see jdk8 (hadoop3) report

            -1 source release artifact
            – See build output for details.

            -1 client integration test
            – Something went wrong with this stage, check relevant console output.

            hudson Hudson added a comment - Results for branch HBASE-21917 build #4 on builds.a.o : -1 overall details (if available): -1 general checks – For more information see general report -1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report -1 jdk8 hadoop3 checks – For more information see jdk8 (hadoop3) report -1 source release artifact – See build output for details. -1 client integration test – Something went wrong with this stage, check relevant console output .
            hudson Hudson added a comment -

            Results for branch HBASE-21917
            build #5 on builds.a.o: -1 overall


            details (if available):

            -1 general checks
            – For more information see general report

            -1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            -1 jdk8 hadoop3 checks
            – For more information see jdk8 (hadoop3) report

            -1 source release artifact
            – See build output for details.

            -1 client integration test
            – Something went wrong with this stage, check relevant console output.

            hudson Hudson added a comment - Results for branch HBASE-21917 build #5 on builds.a.o : -1 overall details (if available): -1 general checks – For more information see general report -1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report -1 jdk8 hadoop3 checks – For more information see jdk8 (hadoop3) report -1 source release artifact – See build output for details. -1 client integration test – Something went wrong with this stage, check relevant console output .
            hudson Hudson added a comment -

            Results for branch HBASE-21917
            build #6 on builds.a.o: -1 overall


            details (if available):

            -1 general checks
            – For more information see general report

            -1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            -1 jdk8 hadoop3 checks
            – For more information see jdk8 (hadoop3) report

            -1 source release artifact
            – See build output for details.

            -1 client integration test
            – Something went wrong with this stage, check relevant console output.

            hudson Hudson added a comment - Results for branch HBASE-21917 build #6 on builds.a.o : -1 overall details (if available): -1 general checks – For more information see general report -1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report -1 jdk8 hadoop3 checks – For more information see jdk8 (hadoop3) report -1 source release artifact – See build output for details. -1 client integration test – Something went wrong with this stage, check relevant console output .
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 26s Docker mode activated.
                  Prechecks
            +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                  HBASE-21879 Compile Tests
            +1 mvninstall 5m 54s HBASE-21879 passed
            +1 compile 2m 46s HBASE-21879 passed
            +1 checkstyle 1m 26s HBASE-21879 passed
            +1 shadedjars 5m 38s branch has no errors when building our shaded downstream artifacts.
            0 findbugs 3m 5s hbase-server in HBASE-21879 has 1 extant Findbugs warnings.
            +1 javadoc 0m 40s HBASE-21879 passed
                  Patch Compile Tests
            +1 mvninstall 4m 54s the patch passed
            +1 compile 2m 32s the patch passed
            -1 javac 2m 32s hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188)
            +1 checkstyle 1m 27s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedjars 5m 39s patch has no errors when building our shaded downstream artifacts.
            +1 hadoopcheck 10m 51s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0.
            +1 findbugs 3m 11s the patch passed
            +1 javadoc 0m 38s the patch passed
                  Other Tests
            -1 unit 291m 44s hbase-server in the patch failed.
            +1 asflicense 0m 27s The patch does not generate ASF License warnings.
            341m 50s



            Reason Tests
            Failed junit tests hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated
              hadoop.hbase.client.TestFromClientSide3
              hadoop.hbase.client.TestAdmin1
              hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas
              hadoop.hbase.client.TestFromClientSideWithCoprocessor
              hadoop.hbase.client.TestFromClientSide
              hadoop.hbase.client.TestHbck
              hadoop.hbase.master.procedure.TestSCPWithReplicas



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b
            JIRA Issue HBASE-21917
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12961287/HBASE-21917.HBASE-21879.v5.patch
            Optional Tests dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
            uname Linux 6f4e883e9f6b 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
            Build tool maven
            Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
            git revision HBASE-21879 / 3c80e417f6
            maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
            Default Java 1.8.0_181
            findbugs v3.1.11
            javac https://builds.apache.org/job/PreCommit-HBASE-Build/16259/artifact/patchprocess/diff-compile-javac-hbase-server.txt
            unit https://builds.apache.org/job/PreCommit-HBASE-Build/16259/artifact/patchprocess/patch-unit-hbase-server.txt
            Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/16259/testReport/
            Max. process+thread count 5389 (vs. ulimit of 10000)
            modules C: hbase-server U: hbase-server
            Console output https://builds.apache.org/job/PreCommit-HBASE-Build/16259/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 26s Docker mode activated.       Prechecks +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       HBASE-21879 Compile Tests +1 mvninstall 5m 54s HBASE-21879 passed +1 compile 2m 46s HBASE-21879 passed +1 checkstyle 1m 26s HBASE-21879 passed +1 shadedjars 5m 38s branch has no errors when building our shaded downstream artifacts. 0 findbugs 3m 5s hbase-server in HBASE-21879 has 1 extant Findbugs warnings. +1 javadoc 0m 40s HBASE-21879 passed       Patch Compile Tests +1 mvninstall 4m 54s the patch passed +1 compile 2m 32s the patch passed -1 javac 2m 32s hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188) +1 checkstyle 1m 27s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedjars 5m 39s patch has no errors when building our shaded downstream artifacts. +1 hadoopcheck 10m 51s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. +1 findbugs 3m 11s the patch passed +1 javadoc 0m 38s the patch passed       Other Tests -1 unit 291m 44s hbase-server in the patch failed. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 341m 50s Reason Tests Failed junit tests hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated   hadoop.hbase.client.TestFromClientSide3   hadoop.hbase.client.TestAdmin1   hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas   hadoop.hbase.client.TestFromClientSideWithCoprocessor   hadoop.hbase.client.TestFromClientSide   hadoop.hbase.client.TestHbck   hadoop.hbase.master.procedure.TestSCPWithReplicas Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b JIRA Issue HBASE-21917 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12961287/HBASE-21917.HBASE-21879.v5.patch Optional Tests dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile uname Linux 6f4e883e9f6b 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision HBASE-21879 / 3c80e417f6 maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) Default Java 1.8.0_181 findbugs v3.1.11 javac https://builds.apache.org/job/PreCommit-HBASE-Build/16259/artifact/patchprocess/diff-compile-javac-hbase-server.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/16259/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/16259/testReport/ Max. process+thread count 5389 (vs. ulimit of 10000) modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/16259/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 17s Docker mode activated.
                  Prechecks
            +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                  HBASE-21879 Compile Tests
            +1 mvninstall 4m 43s HBASE-21879 passed
            +1 compile 2m 6s HBASE-21879 passed
            +1 checkstyle 1m 18s HBASE-21879 passed
            +1 shadedjars 4m 35s branch has no errors when building our shaded downstream artifacts.
            0 findbugs 2m 35s hbase-server in HBASE-21879 has 1 extant Findbugs warnings.
            +1 javadoc 0m 32s HBASE-21879 passed
                  Patch Compile Tests
            +1 mvninstall 4m 6s the patch passed
            +1 compile 2m 7s the patch passed
            -1 javac 2m 7s hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188)
            +1 checkstyle 1m 16s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedjars 4m 36s patch has no errors when building our shaded downstream artifacts.
            +1 hadoopcheck 8m 53s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0.
            +1 findbugs 2m 40s the patch passed
            +1 javadoc 0m 31s the patch passed
                  Other Tests
            -1 unit 143m 18s hbase-server in the patch failed.
            +1 asflicense 0m 22s The patch does not generate ASF License warnings.
            184m 26s



            Reason Tests
            Failed junit tests hadoop.hbase.client.TestAsyncTableGetMultiThreaded



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b
            JIRA Issue HBASE-21917
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12961299/HBASE-21917.HBASE-21879.v6.patch
            Optional Tests dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
            uname Linux 703cb8f6d65c 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 GNU/Linux
            Build tool maven
            Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
            git revision HBASE-21879 / 3c80e417f6
            maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
            Default Java 1.8.0_181
            findbugs v3.1.11
            javac https://builds.apache.org/job/PreCommit-HBASE-Build/16261/artifact/patchprocess/diff-compile-javac-hbase-server.txt
            unit https://builds.apache.org/job/PreCommit-HBASE-Build/16261/artifact/patchprocess/patch-unit-hbase-server.txt
            Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/16261/testReport/
            Max. process+thread count 5123 (vs. ulimit of 10000)
            modules C: hbase-server U: hbase-server
            Console output https://builds.apache.org/job/PreCommit-HBASE-Build/16261/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated.       Prechecks +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       HBASE-21879 Compile Tests +1 mvninstall 4m 43s HBASE-21879 passed +1 compile 2m 6s HBASE-21879 passed +1 checkstyle 1m 18s HBASE-21879 passed +1 shadedjars 4m 35s branch has no errors when building our shaded downstream artifacts. 0 findbugs 2m 35s hbase-server in HBASE-21879 has 1 extant Findbugs warnings. +1 javadoc 0m 32s HBASE-21879 passed       Patch Compile Tests +1 mvninstall 4m 6s the patch passed +1 compile 2m 7s the patch passed -1 javac 2m 7s hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188) +1 checkstyle 1m 16s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedjars 4m 36s patch has no errors when building our shaded downstream artifacts. +1 hadoopcheck 8m 53s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. +1 findbugs 2m 40s the patch passed +1 javadoc 0m 31s the patch passed       Other Tests -1 unit 143m 18s hbase-server in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 184m 26s Reason Tests Failed junit tests hadoop.hbase.client.TestAsyncTableGetMultiThreaded Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b JIRA Issue HBASE-21917 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12961299/HBASE-21917.HBASE-21879.v6.patch Optional Tests dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile uname Linux 703cb8f6d65c 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision HBASE-21879 / 3c80e417f6 maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) Default Java 1.8.0_181 findbugs v3.1.11 javac https://builds.apache.org/job/PreCommit-HBASE-Build/16261/artifact/patchprocess/diff-compile-javac-hbase-server.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/16261/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/16261/testReport/ Max. process+thread count 5123 (vs. ulimit of 10000) modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/16261/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            openinx Zheng Hu added a comment -

            The failed TestAsyncTableGetMultiThreaded works fine under my host, and not related to this patch. Pushed to branch HBASE-21879, thanks Apache9 for reviewing.

            openinx Zheng Hu added a comment - The failed TestAsyncTableGetMultiThreaded works fine under my host, and not related to this patch. Pushed to branch HBASE-21879 , thanks Apache9 for reviewing.
            anoop.hbase Anoop Sam John added a comment -

            +1 for addendum

            anoop.hbase Anoop Sam John added a comment - +1 for addendum
            openinx Zheng Hu added a comment -

            Pushed the addendum to branch HBASE-21879, Thanks anoop.hbase's comments.

            openinx Zheng Hu added a comment - Pushed the addendum to branch HBASE-21879 , Thanks anoop.hbase 's comments.
            hudson Hudson added a comment -

            Results for branch master
            build #1168 on builds.a.o: -1 overall


            details (if available):

            -1 general checks
            – For more information see general report

            -1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            -1 jdk8 hadoop3 checks
            – For more information see jdk8 (hadoop3) report

            +1 source release artifact
            – See build output for details.

            +1 client integration test

            hudson Hudson added a comment - Results for branch master build #1168 on builds.a.o : -1 overall details (if available): -1 general checks – For more information see general report -1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report -1 jdk8 hadoop3 checks – For more information see jdk8 (hadoop3) report +1 source release artifact – See build output for details. +1 client integration test
            hudson Hudson added a comment -

            Results for branch branch-2
            build #2029 on builds.a.o: -1 overall


            details (if available):

            -1 general checks
            – For more information see general report

            -1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            -1 jdk8 hadoop3 checks
            – For more information see jdk8 (hadoop3) report

            +1 source release artifact
            – See build output for details.

            +1 client integration test

            hudson Hudson added a comment - Results for branch branch-2 build #2029 on builds.a.o : -1 overall details (if available): -1 general checks – For more information see general report -1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report -1 jdk8 hadoop3 checks – For more information see jdk8 (hadoop3) report +1 source release artifact – See build output for details. +1 client integration test

            People

              openinx Zheng Hu
              openinx Zheng Hu
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: