Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
When compact a mob table, RS JVM may crash or failed to do compaction as the following logs:
2022-03-11T16:18:44,089 ERROR [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(675): Compaction failed Request=regionName=t1,,1646986716811.964618e679a2434aa7d27018baef8154., storeName=A, fileCount=2, fileSize=2.0 M (1010.2 K, 1010.2 K), priority=1, time=1646986723135java.io.IOException: Mob compaction failed for region: 964618e679a2434aa7d27018baef8154 at org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.performCompaction(DefaultMobStoreCompactor.java:574) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:365) ~[classes/:?] at org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.compact(DefaultMobStoreCompactor.java:225) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:125) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1141) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2442) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:656) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:702) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_292] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_292] at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_292]Caused by: java.io.IOException: Added a key not lexically larger than previous. Current cell = org.apache.hadoop.hbase.PrivateCellUtil$ValueAndTagRewriteByteBufferExtendedCell@565d5bac, prevCell = user00000000000000000000/A:filed01/1646986721047/Put/vlen=0/mvcc=0 at org.apache.hadoop.hbase.util.BloomContext.sanityCheck(BloomContext.java:63) ~[classes/:?] at org.apache.hadoop.hbase.util.BloomContext.writeBloom(BloomContext.java:54) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.StoreFileWriter.appendGeneralBloomfilter(StoreFileWriter.java:296) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.StoreFileWriter.append(StoreFileWriter.java:315) ~[classes/:?] at org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.performCompaction(DefaultMobStoreCompactor.java:464) ~[classes/:?] ... 10 more
It is the same problem as HBASE-25929, because DefaultMobStoreCompactor overwrite performCompaction method of DefaultCompactor.
Attachments
Issue Links
- links to