Index: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java =================================================================== --- hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java (revision 1511511) +++ hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java (working copy) @@ -492,6 +492,16 @@ // is enabled, then we automatically switch off hdfs checksum verification. this.useHBaseChecksum = conf.getBoolean(HConstants.HBASE_CHECKSUM_VERIFICATION, false); + // check that the user has not set the "dfs.client.read.shortcircuit.skip.checksum" property. + boolean shortCircuitSkipChecksum = conf.getBoolean( + "dfs.client.read.shortcircuit.skip.checksum", false); + if (shortCircuitSkipChecksum) { + LOG.warn("Configuration \"dfs.client.read.shortcircuit.skip.checksum\" should not " + + "be set to true." + (this.useHBaseChecksum ? " HBase checksum doesn't require " + + "it, see https://issues.apache.org/jira/browse/HBASE-6868." : "")); + assert !shortCircuitSkipChecksum; //this will fail if assertions are on + } + // Config'ed params this.numRetries = this.conf.getInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER, HConstants.DEFAULT_HBASE_CLIENT_RETRIES_NUMBER); Index: src/main/docbkx/performance.xml =================================================================== --- src/main/docbkx/performance.xml (revision 1511511) +++ src/main/docbkx/performance.xml (working copy) @@ -202,7 +202,11 @@
<varname>hbase.regionserver.checksum.verify</varname> Have HBase write the checksum into the datablock and save - having to do the checksum seek whenever you read. See the + having to do the checksum seek whenever you read. + + See , + and + For more information see the release note on HBASE-5074 support checksums in HBase block cache.
@@ -670,7 +674,10 @@ For optimal performance when short-circuit reads are enabled, it is recommended that HDFS checksums are disabled. To maintain data integrity with HDFS checksums disabled, HBase can be configured to write its own checksums into - its datablocks and verify against these. See . + its datablocks and verify against these. See . When both + local short-circuit reads and hbase level checksums are enabled, you SHOULD NOT disable configuration parameter + "dfs.client.read.shortcircuit.skip.checksum", which will cause skipping checksum on non-hfile reads. HBase already + manages that setting under the covers. The DataNodes need to be restarted in order to pick up the new Index: hbase-common/src/main/resources/hbase-default.xml =================================================================== --- hbase-common/src/main/resources/hbase-default.xml (revision 1511511) +++ hbase-common/src/main/resources/hbase-default.xml (working copy) @@ -931,4 +931,32 @@ datanode, performing block recovery to timeout on a dead datanode; usually dfs.socket.timeout. See the end of HBASE-8389 for more. + + hbase.regionserver.checksum.verify + true + + If set to true, HBase will read data and then verify checksums for + hfile blocks. Checksum verification inside HDFS will be switched off. + If the hbase-checksum verification fails, then it will switch back to + using HDFS checksums. + + + + hbase.hstore.bytes.per.checksum + 16384 + + Number of bytes in a newly created checksum chunk for HBase-level + checksums in hfile blocks. + + + + hbase.hstore.checksum.algorithm + CRC32 + + Name of an algorithm that is used to compute checksums. Possible values + are NULL, CRC32, CRC32C. + + + +