Index: hbase-common/src/main/resources/hbase-default.xml =================================================================== --- hbase-common/src/main/resources/hbase-default.xml (revision 1534795) +++ hbase-common/src/main/resources/hbase-default.xml (working copy) @@ -983,6 +983,22 @@ are NULL, CRC32, CRC32C. + + dfs.client.read.shortcircuit.buffer.size + 131072 + This is an HDFS DFSClient configuration. It comes into play if you have + local shortcircuit reads enabled. Having it here, even thought it an hbase file, it will + be picked up by DFSClients running in HBase servers. This configuration is how large of a + direct byte buffer to allocate per HDFS block read (HDFS block as opposed to a HBase block). + In HBase, we keep all files open all the time so reading we can have a bunch of + allocated direct buffers open at any one time. The default buffer size is 1MB which + is larger than we can really use given HBase reads 64k at a time (by default). If + an HBase RegionServer has a few hundred regions each of which is carrying many storefiles, + the 1MB allocations soon add up such that you may run into a + 'java.lang.OutOfMemoryError: Direct buffer memory' error. 128k should be sufficient + with some padding running with defaults. See HBASE-8143 for background. + + hbase.status.published