Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-26142

NullPointerException when set 'hbase.hregion.memstore.mslab.indexchunksize.percent' to zero



    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 3.0.0-alpha-1, 2.4.0
    • 2.5.0, 3.0.0-alpha-2, 2.4.6, 2.3.7
    • None
    • None
    • Reviewed


      The default value of hbase.hregion.memstore.mslab.indexchunksize.percent introduced by HBASE-24892 is 0.1, but when we use DefaultMemStore by default , which has no IndexChunk and ChunkCreator.indexChunksPool is useless(IndexChunk is only used by CompactingMemStore), so we set hbase.hregion.memstore.mslab.indexchunksize.percent to 0 to save memory space,
      But after running a while, the RegionServer throws NullPointerException and abort:

         Caused by: java.lang.NullPointerException
              at org.apache.hadoop.hbase.regionserver.ChunkCreator$MemStoreChunkPool.access$900(ChunkCreator.java:310)
              at org.apache.hadoop.hbase.regionserver.ChunkCreator.putbackChunks(ChunkCreator.java:608)
              at org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.recycleChunks(MemStoreLABImpl.java:297)
              at org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.close(MemStoreLABImpl.java:268)
              at org.apache.hadoop.hbase.regionserver.Segment.close(Segment.java:149)
              at org.apache.hadoop.hbase.regionserver.AbstractMemStore.clearSnapshot(AbstractMemStore.java:251)
              at org.apache.hadoop.hbase.regionserver.HStore.updateStorefiles(HStore.java:1244)
              at org.apache.hadoop.hbase.regionserver.HStore.access$700(HStore.java:137)
              at org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:2461)
              at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2963)

      The problem is caused by line 608 in ChunkCreator.putbackChunks : Chunk.isIndexChunk incorrectly returns true for DataChunk and
      unexpectedly invokes indexChunksPool.putbackChunks, while indexChunksPool is null:

      594 synchronized void putbackChunks(Set<Integer> chunks) {
      595    // if there is no pool just try to clear the chunkIdMap in case there is something
      596    if (dataChunksPool == null && indexChunksPool == null) {
      597      this.removeChunks(chunks);
      598      return;
      599    }
      601   // if there is a pool, go over all chunk IDs that came back, the chunks may be from pool or not
      602    for (int chunkID : chunks) {
      603     // translate chunk ID to chunk, if chunk initially wasn't in pool
      604      // this translation will (most likely) return null
      605      Chunk chunk = ChunkCreator.this.getChunk(chunkID);
      606     if (chunk != null) {
      607        if (chunk.isFromPool() && chunk.isIndexChunk()) {
      608          indexChunksPool.putbackChunks(chunk);

      For DataChunk , Chunk.isIndexChunk return true because Chunk.isIndexChunk determines the type of chunk based on Chunk.size

       boolean isIndexChunk() {
          return size == ChunkCreator.getInstance().getChunkSize(ChunkCreator.ChunkType.INDEX_CHUNK);

      and ChunkCreator.getChunkSize incorrectly return DataChunk size when ChunkCreator.indexChunksPool is null:

          int getChunkSize(ChunkType chunkType) {
          switch (chunkType) {
            case INDEX_CHUNK:
              if (indexChunksPool != null) {
                return indexChunksPool.getChunkSize();
            case DATA_CHUNK:
              if (dataChunksPool != null) {
                return dataChunksPool.getChunkSize();
              } else { // When pools are empty
                return chunkSize;
              throw new IllegalArgumentException(
                      "chunkType must either be INDEX_CHUNK or DATA_CHUNK");

      In my opinion, in addition to erroneous implementation of ChunkCreator.getChunkSize, we would better not determine the type of Chunk based on Chunk.size, because
      hbase.hregion.memstore.mslab.indexchunksize.percent is set by user and the size of IndexChunk and DataChunk could be the same.Tagged a ChunkType to Chunk is a better choice.


        Issue Links



              comnetwork chenglei
              comnetwork chenglei
              0 Vote for this issue
              5 Start watching this issue