Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-17020

keylen in midkey() dont computed correctly

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4, 2.0.0
    • Fix Version/s: 1.4.0, 1.2.5, 0.98.24, 1.1.8, 2.0.0
    • Component/s: HFile
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      in CellBasedKeyBlockIndexReader.midkey():

                ByteBuff b = midLeafBlock.getBufferWithoutHeader();
                int numDataBlocks = b.getIntAfterPosition(0);
                int keyRelOffset = b.getIntAfterPosition(Bytes.SIZEOF_INT * (midKeyEntry + 1));
                int keyLen = b.getIntAfterPosition(Bytes.SIZEOF_INT * (midKeyEntry + 2)) - keyRelOffset;
      

      the local varible keyLen get this should be total length of: SECONDARY_INDEX_ENTRY_OVERHEAD + firstKey.length;

      the code is:

          void add(byte[] firstKey, long blockOffset, int onDiskDataSize,
              long curTotalNumSubEntries) {
            // Record the offset for the secondary index
            secondaryIndexOffsetMarks.add(curTotalNonRootEntrySize);
            curTotalNonRootEntrySize += SECONDARY_INDEX_ENTRY_OVERHEAD
                + firstKey.length;
      

      when the midkey last entry of a leaf-level index block, this may throw:

      2016-10-01 12:27:55,186 ERROR [MemStoreFlusher.0] regionserver.MemStoreFlusher: Cache flusher failed for entry [flush region pora_6_item_feature,0061:,1473838922457.12617bc4ebbfd171018bf96ac9bdd2a7.]
      java.lang.ArrayIndexOutOfBoundsException
      at org.apache.hadoop.hbase.util.ByteBufferUtils.copyFromBufferToArray(ByteBufferUtils.java:936)
      at org.apache.hadoop.hbase.nio.SingleByteBuff.toBytes(SingleByteBuff.java:303)
      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$CellBasedKeyBlockIndexReader.midkey(HFileBlockIndex.java:419)
      at org.apache.hadoop.hbase.io.hfile.HFileReaderImpl.midkey(HFileReaderImpl.java:1519)
      at org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1520)
      at org.apache.hadoop.hbase.regionserver.StoreFile.getFileSplitPoint(StoreFile.java:706)
      at org.apache.hadoop.hbase.regionserver.DefaultStoreFileManager.getSplitPoint(DefaultStoreFileManager.java:126)
      at org.apache.hadoop.hbase.regionserver.HStore.getSplitPoint(HStore.java:1983)
      at org.apache.hadoop.hbase.regionserver.ConstantFamilySizeRegionSplitPolicy.getSplitPoint(ConstantFamilySizeRegionSplitPolicy.java:77)
      at org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:7756)
      at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:513)
      at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:471)
      at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$800(MemStoreFlusher.java:75)
      at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:259)
      at java.lang.Thread.run(Thread.java:756)

        Attachments

        1. HBASE-17020.branch-0.98.patch
          6 kB
          Yu Li
        2. HBASE-17020.branch-0.98.patch
          6 kB
          Yu Li
        3. HBASE-17020.branch-1.1.patch
          6 kB
          Yu Li
        4. HBASE-17020-branch-0.98.patch
          6 kB
          Yu Sun
        5. HBASE-17020-v1.patch
          1 kB
          Yu Sun
        6. HBASE-17020-v2.patch
          6 kB
          Yu Li
        7. HBASE-17020-v2.patch
          6 kB
          Yu Sun
        8. HBASE-17020-v3-branch1.1.patch
          6 kB
          Yu Sun

          Issue Links

            Activity

              People

              • Assignee:
                haoran Yu Sun
                Reporter:
                haoran Yu Sun
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: