Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-13520

NullPointerException in TagRewriteCell



    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.0.0
    • 1.1.0, 1.0.2, 2.0.0
    • None
    • None
    • Reviewed


      Found via running IntegrationTestIngestWithVisibilityLabels with Kerberos enabled.

      2015-04-20 18:54:36,712 ERROR [B.defaultRpcServer.handler=17,queue=2,port=16020] ipc.RpcServer: Unexpected throwable object
              at org.apache.hadoop.hbase.TagRewriteCell.getTagsLength(TagRewriteCell.java:157)
              at org.apache.hadoop.hbase.TagRewriteCell.heapSize(TagRewriteCell.java:186)
              at org.apache.hadoop.hbase.CellUtil.estimatedHeapSizeOf(CellUtil.java:568)
              at org.apache.hadoop.hbase.regionserver.DefaultMemStore.heapSizeChange(DefaultMemStore.java:1024)
              at org.apache.hadoop.hbase.regionserver.DefaultMemStore.internalAdd(DefaultMemStore.java:259)
              at org.apache.hadoop.hbase.regionserver.DefaultMemStore.upsert(DefaultMemStore.java:567)
              at org.apache.hadoop.hbase.regionserver.DefaultMemStore.upsert(DefaultMemStore.java:541)
              at org.apache.hadoop.hbase.regionserver.HStore.upsert(HStore.java:2154)
              at org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:7127)
              at org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:504)
              at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2020)
              at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31967)
              at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2106)
              at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
              at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
              at org.apache.hadoop.hbase.ipc.RpcExecutor$2.run(RpcExecutor.java:107)
              at java.lang.Thread.run(Thread.java:745)

      HBASE-11870 tried to be tricky when only the tags of a Cell need to be altered in the write-pipeline by creating a TagRewriteCell which avoided copying all components of the original Cell. In an attempt to help free the tags on the old cell that we wouldn't be referencing anymore, TagRewriteCell nulls out the original byte[] tags.

      This causes a problem in that the implementation of heapSize() as it getTagsLength() on the original Cell instead of the on this. Because the tags on the passed in Cell (which was also a TagRewriteCell) were null'ed out in the constructor, this results in a NPE by the byte array is null.

      I believe this isn't observed in normal, unsecure deployments because there is only one RegionObserver/Coprocessor loaded that gets invoked via postMutationBeforeWAL. When there is only one RegionObserver, the TagRewriteCell isn't passed another TagRewriteCell, but instead a cell from the wire/protobuf. This means that the optimization isn't performed. When we have two (or more) observers that a TagRewriteCell passes through (and a new TagRewriteCell is created and the old TagRewriteCell's tags array is nulled), this enables the described-above NPE.


        1. HBASE-13520.patch
          4 kB
          Josh Elser
        2. HBASE-13520-v1.patch
          4 kB
          Josh Elser

        Issue Links



              elserj Josh Elser
              elserj Josh Elser
              0 Vote for this issue
              4 Start watching this issue