Index: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
===================================================================
--- hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java (revision 1511511)
+++ hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java (working copy)
@@ -492,6 +492,16 @@
// is enabled, then we automatically switch off hdfs checksum verification.
this.useHBaseChecksum = conf.getBoolean(HConstants.HBASE_CHECKSUM_VERIFICATION, false);
+ // check that the user has not set the "dfs.client.read.shortcircuit.skip.checksum" property.
+ boolean shortCircuitSkipChecksum = conf.getBoolean(
+ "dfs.client.read.shortcircuit.skip.checksum", false);
+ if (shortCircuitSkipChecksum) {
+ LOG.warn("Configuration \"dfs.client.read.shortcircuit.skip.checksum\" should not " +
+ "be set to true." + (this.useHBaseChecksum ? " HBase checksum doesn't require " +
+ "it, see https://issues.apache.org/jira/browse/HBASE-6868." : ""));
+ assert !shortCircuitSkipChecksum; //this will fail if assertions are on
+ }
+
// Config'ed params
this.numRetries = this.conf.getInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER,
HConstants.DEFAULT_HBASE_CLIENT_RETRIES_NUMBER);
Index: src/main/docbkx/performance.xml
===================================================================
--- src/main/docbkx/performance.xml (revision 1511511)
+++ src/main/docbkx/performance.xml (working copy)
@@ -202,7 +202,11 @@
hbase.regionserver.checksum.verifyHave HBase write the checksum into the datablock and save
- having to do the checksum seek whenever you read. See the
+ having to do the checksum seek whenever you read.
+
+ See ,
+ and
+ For more information see the
release note on HBASE-5074 support checksums in HBase block cache.
@@ -670,7 +674,10 @@
For optimal performance when short-circuit reads are enabled, it is recommended that HDFS checksums are disabled.
To maintain data integrity with HDFS checksums disabled, HBase can be configured to write its own checksums into
- its datablocks and verify against these. See .
+ its datablocks and verify against these. See . When both
+ local short-circuit reads and hbase level checksums are enabled, you SHOULD NOT disable configuration parameter
+ "dfs.client.read.shortcircuit.skip.checksum", which will cause skipping checksum on non-hfile reads. HBase already
+ manages that setting under the covers.
The DataNodes need to be restarted in order to pick up the new
Index: hbase-common/src/main/resources/hbase-default.xml
===================================================================
--- hbase-common/src/main/resources/hbase-default.xml (revision 1511511)
+++ hbase-common/src/main/resources/hbase-default.xml (working copy)
@@ -931,4 +931,32 @@
datanode, performing block recovery to timeout on a dead datanode; usually
dfs.socket.timeout. See the end of HBASE-8389 for more.
+
+ hbase.regionserver.checksum.verify
+ true
+
+ If set to true, HBase will read data and then verify checksums for
+ hfile blocks. Checksum verification inside HDFS will be switched off.
+ If the hbase-checksum verification fails, then it will switch back to
+ using HDFS checksums.
+
+
+
+ hbase.hstore.bytes.per.checksum
+ 16384
+
+ Number of bytes in a newly created checksum chunk for HBase-level
+ checksums in hfile blocks.
+
+
+
+ hbase.hstore.checksum.algorithm
+ CRC32
+
+ Name of an algorithm that is used to compute checksums. Possible values
+ are NULL, CRC32, CRC32C.
+
+
+
+