Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
2.0.0
-
None
-
None
-
Reviewed
Description
During perf verification of HBASE-14463, found offheap BucketCache not working with assertion enabled in hbase-env.sh:
export HBASE_OPTS="-ea -XX:+HeapDumpOnOutOfMemoryError -XX:+UseConcMarkSweepGC"
And the error when running PE tool is like:
15/10/21 16:06:34 INFO client.AsyncProcess: #10, table=TestTable, attempt=10/21 failed=20ops, last exception: java.io.IOException: java.io.IOException
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2181)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.AssertionError
at org.apache.hadoop.hbase.OffheapKeyValue.<init>(OffheapKeyValue.java:52)
at org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl$ShareableMemoryOffheapKeyValue.<init>(HFileReaderImpl.java:1003)
at org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.getCell(HFileReaderImpl.java:949)
at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:201)
at org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:323)
at org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:279)
at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:825)
at org.apache.hadoop.hbase.regionserver.StoreScanner.seekAsDirection(StoreScanner.java:813)
at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:641)
at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:153)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5649)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:5795)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5568)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:5544)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:5530)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2044)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:663)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2156)
ram_krish and anoop.hbase, mind to take a look?