Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-15555 Memory Management
  3. HBASE-16440

MemstoreChunkPool might cross its maxCount of chunks to pool

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0
    • None
    • None
    • Reviewed

    Description

      void putbackChunks(BlockingQueue<Chunk> chunks) {
          int maxNumToPutback = this.maxCount - reclaimedChunks.size();
          if (maxNumToPutback <= 0) {
            return;
          }
          chunks.drainTo(reclaimedChunks, maxNumToPutback);
          // clear reference of any non-reclaimable chunks
          if (chunks.size() > 0) {
            if (LOG.isTraceEnabled()) {
              LOG.trace("Left " + chunks.size() + " unreclaimable chunks, removing them from queue");
            }
            chunks.clear();
          }
        }
      

      There is no synchroization. 2 threads might be calling this API as part of a MSLAB close. (Once the memstore is flushed). It pass all the chunks used by it. (Those might not have been come from pool also). We try to put back chunks such that it is not crossing maxCount. Suppose maxCount is 10 and currently no chunks in 'reclaimedChunks'. Say both threads at line one. Both see 'maxNumToPutback ' as 10 and that will make 20 chunks being pooled. Similar issue is in putbackChunk(Chunk chunk) API also.

      Attachments

        1. HBASE-16440.patch
          21 kB
          Anoop Sam John
        2. HBASE-16440_V2.patch
          20 kB
          Anoop Sam John
        3. HBASE-16440_V3.patch
          23 kB
          Anoop Sam John
        4. HBASE-16440_V4.patch
          23 kB
          Anoop Sam John

        Activity

          People

            anoop.hbase Anoop Sam John
            anoop.hbase Anoop Sam John
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: