[HBASE-18030] Per Cell TTL tags may get duplicated with increments/Append causing tags length overflow

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.0.0, 0.98.9, 2.0.0
• Fix Version/s: 1.4.0, 1.3.2, 2.0.0, 1.2.7
• Component/s: regionserver
• Hadoop Flags: Reviewed
• Assignee: Anoop Sam John
• Reporter: Abhishek Kumar
• Created: 11/May/17 04:31
• Updated: 01/Aug/18 06:24
• Resolved: 06/Jun/17 06:59

Description

2017-04-29 14:24:14,135 ERROR [B.fifo.QRpcServer.handler=49,queue=1,port=16020] ipc.RpcServer: Unexpected throwable object java.lang.IllegalStateException: Invalid currTagsLen -32712. Block offset: 3707853, block length: 72841, position: 0 (without header). at org.apache.hadoop.hbase.io.hfile.HFileReaderV3$ScannerV3.checkTagsLen(HFileReaderV3.java:226)

I am not not using any hbase tags feature.
The Increment operation from the application side is triggering this error. The same is happening when scanner is run on this table. It feels that one or more particular HFile block is corrupt (with negative tagLength).

hbase(main):007:0> scan 'table-name',

{LIMIT=>1,STARTROW=>'ad:event_count:a'}

Returning the result

hbase(main):008:0> scan 'table-name',

{LIMIT=>1,STARTROW=>'ad:event_count:b'}

ROW COLUMN+CELL
ERROR: java.io.IOException: java.lang.IllegalStateException: Invalid currTagsLen -32701. Block offset: 272031, block length: 72441, position: 32487 (without header).
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleException(HRegion.java:5607)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5579)
at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2627)
at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2613)
at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2595)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2282)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32295)