Details
-
Sub-task
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
Reviewed
Description
I've tried to make a patch for HBASE-21879, most of work seems to be fine, but the trouble is:
HFileBlock#validateChecksum can only accept ByteBuffer as its input, while after the HBASE-21916, we will use an ourself-defined ByteBuff (which can be SingleByteBuff or MultiByteBuff).
Now, need to create our own ByteBuff checksum validation method, should not be so hard but an separate issue will be more clearer.
Attachments
Attachments
- HBASE-21917.addendum.HBASE-21879.v7.patch
- 2 kB
- Zheng Hu
- HBASE-21917.HBASE-21879.v5.patch
- 18 kB
- Zheng Hu
- HBASE-21917.HBASE-21879.v6.patch
- 19 kB
- Zheng Hu
- HBASE-21917.v1.patch
- 10 kB
- Zheng Hu
- HBASE-21917.v2.patch
- 10 kB
- Zheng Hu
- HBASE-21917.v3.patch
- 11 kB
- Zheng Hu
- HBASE-21917.v4.patch
- 11 kB
- Zheng Hu
Issue Links
- links to
Activity
Maybe we can new a very small heap size such as 256, and update(..) many times until we update all the 16KB bytes.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 10s | Docker mode activated. |
Prechecks | |||
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
-0 | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
master Compile Tests | |||
+1 | mvninstall | 4m 46s | master passed |
+1 | compile | 2m 2s | master passed |
+1 | checkstyle | 1m 15s | master passed |
+1 | shadedjars | 4m 31s | branch has no errors when building our shaded downstream artifacts. |
+1 | findbugs | 2m 22s | master passed |
+1 | javadoc | 0m 29s | master passed |
Patch Compile Tests | |||
+1 | mvninstall | 4m 39s | the patch passed |
+1 | compile | 2m 2s | the patch passed |
-1 | javac | 2m 2s | hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188) |
-1 | checkstyle | 1m 15s | hbase-server: The patch generated 1 new + 27 unchanged - 0 fixed = 28 total (was 27) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 4m 32s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 9m 44s | Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. |
+1 | findbugs | 2m 35s | the patch passed |
+1 | javadoc | 0m 30s | the patch passed |
Other Tests | |||
+1 | unit | 137m 18s | hbase-server in the patch passed. |
+1 | asflicense | 0m 22s | The patch does not generate ASF License warnings. |
179m 0s |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12959097/HBASE-21917.v2.patch |
Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux 9d53effccd6c 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh |
git revision | master / 7be71c0f55 |
maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
Default Java | 1.8.0_181 |
findbugs | v3.1.0-RC3 |
javac | https://builds.apache.org/job/PreCommit-HBASE-Build/16020/artifact/patchprocess/diff-compile-javac-hbase-server.txt |
checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/16020/artifact/patchprocess/diff-checkstyle-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/16020/testReport/ |
Max. process+thread count | 4858 (vs. ulimit of 10000) |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/16020/console |
Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 22s | Docker mode activated. |
Prechecks | |||
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
-0 | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
master Compile Tests | |||
+1 | mvninstall | 4m 5s | master passed |
+1 | compile | 1m 52s | master passed |
+1 | checkstyle | 1m 5s | master passed |
+1 | shadedjars | 4m 11s | branch has no errors when building our shaded downstream artifacts. |
+1 | findbugs | 2m 25s | master passed |
+1 | javadoc | 0m 29s | master passed |
Patch Compile Tests | |||
+1 | mvninstall | 4m 12s | the patch passed |
+1 | compile | 1m 51s | the patch passed |
-1 | javac | 1m 51s | hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188) |
-1 | checkstyle | 1m 6s | hbase-server: The patch generated 1 new + 27 unchanged - 0 fixed = 28 total (was 27) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 4m 10s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 8m 39s | Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. |
+1 | findbugs | 2m 29s | the patch passed |
+1 | javadoc | 0m 30s | the patch passed |
Other Tests | |||
-1 | unit | 259m 22s | hbase-server in the patch failed. |
+1 | asflicense | 0m 26s | The patch does not generate ASF License warnings. |
297m 46s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.master.procedure.TestServerCrashProcedure |
hadoop.hbase.client.TestFromClientSide3 | |
hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory | |
hadoop.hbase.master.procedure.TestServerCrashProcedureWithReplicas | |
hadoop.hbase.client.TestAdmin1 | |
hadoop.hbase.client.TestFromClientSide | |
hadoop.hbase.client.TestFromClientSideWithCoprocessor |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12959073/HBASE-21917.v1.patch |
Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux 26fa911b3c2d 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / 40e1d9174e |
maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
Default Java | 1.8.0_181 |
findbugs | v3.1.0-RC3 |
javac | https://builds.apache.org/job/PreCommit-HBASE-Build/16016/artifact/patchprocess/diff-compile-javac-hbase-server.txt |
checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/16016/artifact/patchprocess/diff-checkstyle-hbase-server.txt |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/16016/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/16016/testReport/ |
Max. process+thread count | 5315 (vs. ulimit of 10000) |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/16016/console |
Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 13s | Docker mode activated. |
Prechecks | |||
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
-0 | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
master Compile Tests | |||
+1 | mvninstall | 4m 24s | master passed |
+1 | compile | 2m 3s | master passed |
+1 | checkstyle | 1m 4s | master passed |
+1 | shadedjars | 4m 26s | branch has no errors when building our shaded downstream artifacts. |
+1 | findbugs | 2m 22s | master passed |
+1 | javadoc | 0m 31s | master passed |
Patch Compile Tests | |||
+1 | mvninstall | 4m 24s | the patch passed |
+1 | compile | 2m 5s | the patch passed |
-1 | javac | 2m 5s | hbase-server generated 2 new + 186 unchanged - 2 fixed = 188 total (was 188) |
-1 | checkstyle | 1m 9s | hbase-server: The patch generated 1 new + 27 unchanged - 0 fixed = 28 total (was 27) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 4m 13s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 8m 50s | Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. |
+1 | findbugs | 2m 30s | the patch passed |
+1 | javadoc | 0m 27s | the patch passed |
Other Tests | |||
-1 | unit | 132m 15s | hbase-server in the patch failed. |
+1 | asflicense | 0m 28s | The patch does not generate ASF License warnings. |
171m 51s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.master.TestServerCrashProcedureCarryingMetaStuck |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12959112/HBASE-21917.v3.patch |
Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux c93e02471fdc 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh |
git revision | master / 6176471957 |
maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
Default Java | 1.8.0_181 |
findbugs | v3.1.0-RC3 |
javac | https://builds.apache.org/job/PreCommit-HBASE-Build/16025/artifact/patchprocess/diff-compile-javac-hbase-server.txt |
checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/16025/artifact/patchprocess/diff-checkstyle-hbase-server.txt |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/16025/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/16025/testReport/ |
Max. process+thread count | 5013 (vs. ulimit of 10000) |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/16025/console |
Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 13s | Docker mode activated. |
Prechecks | |||
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
-0 | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
master Compile Tests | |||
+1 | mvninstall | 5m 15s | master passed |
+1 | compile | 2m 22s | master passed |
+1 | checkstyle | 1m 30s | master passed |
+1 | shadedjars | 5m 25s | branch has no errors when building our shaded downstream artifacts. |
+1 | findbugs | 2m 52s | master passed |
+1 | javadoc | 0m 38s | master passed |
Patch Compile Tests | |||
+1 | mvninstall | 5m 18s | the patch passed |
+1 | compile | 2m 25s | the patch passed |
-1 | javac | 2m 25s | hbase-server generated 2 new + 186 unchanged - 2 fixed = 188 total (was 188) |
-1 | checkstyle | 1m 22s | hbase-server: The patch generated 2 new + 27 unchanged - 0 fixed = 29 total (was 27) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 5m 9s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 11m 4s | Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. |
+1 | findbugs | 3m 3s | the patch passed |
+1 | javadoc | 0m 38s | the patch passed |
Other Tests | |||
-1 | unit | 144m 6s | hbase-server in the patch failed. |
+1 | asflicense | 0m 25s | The patch does not generate ASF License warnings. |
192m 20s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.master.TestServerCrashProcedureCarryingMetaStuck |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12959111/HBASE-21917.v3.patch |
Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux e0be646dd6c9 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / 6176471957 |
maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
Default Java | 1.8.0_181 |
findbugs | v3.1.0-RC3 |
javac | https://builds.apache.org/job/PreCommit-HBASE-Build/16024/artifact/patchprocess/diff-compile-javac-hbase-server.txt |
checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/16024/artifact/patchprocess/diff-checkstyle-hbase-server.txt |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/16024/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/16024/testReport/ |
Max. process+thread count | 5247 (vs. ulimit of 10000) |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/16024/console |
Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
Thanks for hacking on this openinx. IIRC, we want native checksum checking to cut in. Is that happening here? If it is, small buffers means we cross the native divide many times – each of which costs – rather than a few times?
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 13s | Docker mode activated. |
Prechecks | |||
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
-0 | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
master Compile Tests | |||
+1 | mvninstall | 5m 35s | master passed |
+1 | compile | 2m 26s | master passed |
+1 | checkstyle | 1m 26s | master passed |
+1 | shadedjars | 5m 12s | branch has no errors when building our shaded downstream artifacts. |
+1 | findbugs | 2m 58s | master passed |
+1 | javadoc | 0m 39s | master passed |
Patch Compile Tests | |||
+1 | mvninstall | 5m 15s | the patch passed |
+1 | compile | 2m 23s | the patch passed |
-1 | javac | 2m 23s | hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188) |
+1 | checkstyle | 1m 26s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 4m 59s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 10m 40s | Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. |
+1 | findbugs | 2m 58s | the patch passed |
+1 | javadoc | 0m 37s | the patch passed |
Other Tests | |||
-1 | unit | 133m 9s | hbase-server in the patch failed. |
+1 | asflicense | 0m 22s | The patch does not generate ASF License warnings. |
180m 52s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.master.TestServerCrashProcedureCarryingMetaStuck |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12959134/HBASE-21917.v4.patch |
Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux 7fba71c33bdf 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh |
git revision | master / 593745e8ac |
maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
Default Java | 1.8.0_181 |
findbugs | v3.1.0-RC3 |
javac | https://builds.apache.org/job/PreCommit-HBASE-Build/16027/artifact/patchprocess/diff-compile-javac-hbase-server.txt |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/16027/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/16027/testReport/ |
Max. process+thread count | 5114 (vs. ulimit of 10000) |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/16027/console |
Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
IIRC, we want native checksum checking to cut in. Is that happening here? If it is, small buffers means we cross the native divide many times – each of which costs – rather than a few times?
Thanks stack, yeah, you are right. before this patch , the hadoop's verifyChunkedSums will try to use native checksum checking if NativeCrc32 is available. but the bad news is: NativeCrc32 is not a public class in hadoop-common, our HBase has no access. so i can only uniform those logics into a general checksum method as the patch described. BTW, the hadoop-common's checksum looks like this:
public void verifyChunkedSums(ByteBuffer data, ByteBuffer checksums, String fileName, long basePos) throws ChecksumException { if (type.size == 0) return; if (data.hasArray() && checksums.hasArray()) { verifyChunkedSums( data.array(), data.arrayOffset() + data.position(), data.remaining(), checksums.array(), checksums.arrayOffset() + checksums.position(), fileName, basePos); // -------> private method, no access return; } if (NativeCrc32.isAvailable()) { NativeCrc32.verifyChunkedSums(bytesPerChecksum, type.id, checksums, data, fileName, basePos); -------> NativeCrc32 is not public, no access. return; } // ...general ByteBuffer checksum verify logic...
Can we bring it local?
(IIRC when I looked long time ago, it isn't really possible. If we could bring it local we could use it in meantime while worked on changing access downstream – thanks openinx)
If we could bring it local we could use it in meantime while worked on changing access downstream
Got your mind, thanks stack. but I'm worrying that after we bring those native checksum methods into hbase, those methods can still only accept ByteBuffer or byte[] ( data&checksum ) arguments, so we may need to copy few bytes from ByteBuff (NOT nio.ByteBuffer) each time to update the checksum and the heavy native cost happen(as you said before). Another way will be: verify 16KB data each time, no native call cost but the young gc issue may still exists.
So I prefer to keep the current general small byte[] checksum updating in patch.v4.
so we may need to copy few bytes from ByteBuff (NOT nio.ByteBuffer) each time to update the checksum
Well one thing which can help here. See ByteBuff#asSubByteBuffer(int offset, int length, ObjectIntPair<ByteBuffer> pair).
This API will convert a ByteBuff into an NIO BB. When it is an MBB, and we need one NIO BB from it from offset to len, it can so happen that this region of bytes in a single BB. If so this API will not do any copy but directly give that individual BB. Only when the region crosses 2 BB, we will end up in copying. In BC we have 4 MB sized BBs and what we need as part of HFile block is 64 KB, the chances that this across BBs block may not be that big!
Will see the patch now
public void verifyChunkedSums(ByteBuffer data, ByteBuffer checksums, String fileName, long basePos) throws ChecksumException { if (type.size == 0) return; if (data.hasArray() && checksums.hasArray()) { verifyChunkedSums( data.array(), data.arrayOffset() + data.position(), data.remaining(), checksums.array(), checksums.arrayOffset() + checksums.position(), fileName, basePos); // -------> private method, no access return; } if (NativeCrc32.isAvailable()) { NativeCrc32.verifyChunkedSums(bytesPerChecksum, type.id, checksums, data, fileName, basePos); -------> NativeCrc32 is not public, no access. return; }
Based on this code in Hadoop, if we can pass ByteBuffer to this Hadoop API and that BB is DBB , we will not go into 1st if block.l But if native CRC is available (if so, we wanted to make use of it), Hadoop can call it using the BB that we pass. Native API calls means it would work with DBB with out any copy. So if we used ByteBuff#asSubByteBuffer API, and mostly it will not incur any copy (as explained above) we can make use of native API with out any extra on heap copy and GC over head no?
Results for branch HBASE-21917
build #1 on builds.a.o: -1 overall
details (if available):
-1 general checks
– For more information see general report
-1 jdk8 hadoop2 checks
– For more information see jdk8 (hadoop2) report
-1 jdk8 hadoop3 checks
– For more information see jdk8 (hadoop3) report
-1 source release artifact
– See build output for details.
-1 client integration test
– Something went wrong with this stage, check relevant console output.
Results for branch HBASE-21917
build #2 on builds.a.o: -1 overall
details (if available):
-1 general checks
– For more information see general report
-1 jdk8 hadoop2 checks
– For more information see jdk8 (hadoop2) report
-1 jdk8 hadoop3 checks
– For more information see jdk8 (hadoop3) report
-1 source release artifact
– See build output for details.
-1 client integration test
– Something went wrong with this stage, check relevant console output.
Results for branch HBASE-21917
build #3 on builds.a.o: -1 overall
details (if available):
-1 general checks
– For more information see general report
-1 jdk8 hadoop2 checks
– For more information see jdk8 (hadoop2) report
-1 jdk8 hadoop3 checks
– For more information see jdk8 (hadoop3) report
-1 source release artifact
– See build output for details.
-1 client integration test
– Something went wrong with this stage, check relevant console output.
Results for branch HBASE-21917
build #4 on builds.a.o: -1 overall
details (if available):
-1 general checks
– For more information see general report
-1 jdk8 hadoop2 checks
– For more information see jdk8 (hadoop2) report
-1 jdk8 hadoop3 checks
– For more information see jdk8 (hadoop3) report
-1 source release artifact
– See build output for details.
-1 client integration test
– Something went wrong with this stage, check relevant console output.
Results for branch HBASE-21917
build #5 on builds.a.o: -1 overall
details (if available):
-1 general checks
– For more information see general report
-1 jdk8 hadoop2 checks
– For more information see jdk8 (hadoop2) report
-1 jdk8 hadoop3 checks
– For more information see jdk8 (hadoop3) report
-1 source release artifact
– See build output for details.
-1 client integration test
– Something went wrong with this stage, check relevant console output.
Results for branch HBASE-21917
build #6 on builds.a.o: -1 overall
details (if available):
-1 general checks
– For more information see general report
-1 jdk8 hadoop2 checks
– For more information see jdk8 (hadoop2) report
-1 jdk8 hadoop3 checks
– For more information see jdk8 (hadoop3) report
-1 source release artifact
– See build output for details.
-1 client integration test
– Something went wrong with this stage, check relevant console output.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 26s | Docker mode activated. |
Prechecks | |||
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
|
|||
+1 | mvninstall | 5m 54s | |
+1 | compile | 2m 46s | |
+1 | checkstyle | 1m 26s | |
+1 | shadedjars | 5m 38s | branch has no errors when building our shaded downstream artifacts. |
0 | findbugs | 3m 5s | hbase-server in |
+1 | javadoc | 0m 40s | |
Patch Compile Tests | |||
+1 | mvninstall | 4m 54s | the patch passed |
+1 | compile | 2m 32s | the patch passed |
-1 | javac | 2m 32s | hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188) |
+1 | checkstyle | 1m 27s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 5m 39s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 10m 51s | Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. |
+1 | findbugs | 3m 11s | the patch passed |
+1 | javadoc | 0m 38s | the patch passed |
Other Tests | |||
-1 | unit | 291m 44s | hbase-server in the patch failed. |
+1 | asflicense | 0m 27s | The patch does not generate ASF License warnings. |
341m 50s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated |
hadoop.hbase.client.TestFromClientSide3 | |
hadoop.hbase.client.TestAdmin1 | |
hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | |
hadoop.hbase.client.TestFromClientSideWithCoprocessor | |
hadoop.hbase.client.TestFromClientSide | |
hadoop.hbase.client.TestHbck | |
hadoop.hbase.master.procedure.TestSCPWithReplicas |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12961287/HBASE-21917.HBASE-21879.v5.patch |
Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux 6f4e883e9f6b 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | |
maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
Default Java | 1.8.0_181 |
findbugs | v3.1.11 |
javac | https://builds.apache.org/job/PreCommit-HBASE-Build/16259/artifact/patchprocess/diff-compile-javac-hbase-server.txt |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/16259/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/16259/testReport/ |
Max. process+thread count | 5389 (vs. ulimit of 10000) |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/16259/console |
Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 17s | Docker mode activated. |
Prechecks | |||
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
|
|||
+1 | mvninstall | 4m 43s | |
+1 | compile | 2m 6s | |
+1 | checkstyle | 1m 18s | |
+1 | shadedjars | 4m 35s | branch has no errors when building our shaded downstream artifacts. |
0 | findbugs | 2m 35s | hbase-server in |
+1 | javadoc | 0m 32s | |
Patch Compile Tests | |||
+1 | mvninstall | 4m 6s | the patch passed |
+1 | compile | 2m 7s | the patch passed |
-1 | javac | 2m 7s | hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 total (was 188) |
+1 | checkstyle | 1m 16s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 4m 36s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 8m 53s | Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. |
+1 | findbugs | 2m 40s | the patch passed |
+1 | javadoc | 0m 31s | the patch passed |
Other Tests | |||
-1 | unit | 143m 18s | hbase-server in the patch failed. |
+1 | asflicense | 0m 22s | The patch does not generate ASF License warnings. |
184m 26s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.client.TestAsyncTableGetMultiThreaded |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12961299/HBASE-21917.HBASE-21879.v6.patch |
Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux 703cb8f6d65c 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | |
maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
Default Java | 1.8.0_181 |
findbugs | v3.1.11 |
javac | https://builds.apache.org/job/PreCommit-HBASE-Build/16261/artifact/patchprocess/diff-compile-javac-hbase-server.txt |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/16261/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/16261/testReport/ |
Max. process+thread count | 5123 (vs. ulimit of 10000) |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/16261/console |
Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
The failed TestAsyncTableGetMultiThreaded works fine under my host, and not related to this patch. Pushed to branch HBASE-21879, thanks Apache9 for reviewing.
Pushed the addendum to branch HBASE-21879, Thanks anoop.hbase's comments.
Results for branch master
build #1168 on builds.a.o: -1 overall
details (if available):
-1 general checks
– For more information see general report
-1 jdk8 hadoop2 checks
– For more information see jdk8 (hadoop2) report
-1 jdk8 hadoop3 checks
– For more information see jdk8 (hadoop3) report
+1 source release artifact
– See build output for details.
+1 client integration test
Results for branch branch-2
build #2029 on builds.a.o: -1 overall
details (if available):
-1 general checks
– For more information see general report
-1 jdk8 hadoop2 checks
– For more information see jdk8 (hadoop2) report
-1 jdk8 hadoop3 checks
– For more information see jdk8 (hadoop3) report
+1 source release artifact
– See build output for details.
+1 client integration test
Oh, our default bytesPerChecksum is 16KB. while there's no ByteBuffer update(..) inteface for java.util.zip.Checksum, which means we still need to allocate an 16KB on-heap byte[] to validate the checksum. FYI stack, anoop.hbase, ram_krish