Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20196

Maintain all regions with same size in memstore flusher

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Here is the javadoc for getCopyOfOnlineRegionsSortedByOffHeapSize() :

         *   the biggest.  If two regions are the same size, then the last one found wins; i.e. this
         *   method may NOT return all regions.
      

      Currently value type is HRegion - we only store one region per size.
      I think we should change value type to Collection<HRegion> so that we don't miss any region (potentially with big size).

      e.g. Suppose there are there regions (R1, R2 and R3) with sizes 100, 100 and 1, respectively.
      Using the current data structure, R2 would be stored in the Map, evicting R1 from the Map.
      This means that the current code would choose to flush regions R2 and R3, releasing 101 from memory.
      If value type is changed to Collection<HRegion>, we would flush both R1 and R2. This achieves faster memory reclamation.

      Confirmed with Eshcar Hillel over in HBASE-20090

        Attachments

        1. 20196.v2.txt
          6 kB
          Ted Yu
        2. 20196.v1.txt
          6 kB
          Ted Yu

          Activity

            People

            • Assignee:
              yuzhihong@gmail.com Ted Yu
              Reporter:
              yuzhihong@gmail.com Ted Yu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: