Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.90.5
    • Fix Version/s: 0.92.1, 0.94.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Spawning this from HBASE-2375, I saw that it was much more efficient compaction-wise to check if we can split right after flushing. Much like the ideas that Jon spelled out in the description of that jira, the window is smaller because you don't have to compact and then split right away to only compact again when the daughters open.

      Another thing it improves is while we're normally waiting for the compaction to happen, data that's still coming in will make us go way past the MAX_FILESIZE to a point where for the first region I was seeing a store size 3-4x bigger before it was able to split.

      I targeted this for 0.94, but I'd like to get this into 0.92.1 or .2 too.

        Activity

        Hide
        Hudson added a comment -

        Integrated in HBase-0.92-security #91 (See https://builds.apache.org/job/HBase-0.92-security/91/)
        HBASE-5393 Consider splitting after flushing (Revision 1245726)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Show
        Hudson added a comment - Integrated in HBase-0.92-security #91 (See https://builds.apache.org/job/HBase-0.92-security/91/ ) HBASE-5393 Consider splitting after flushing (Revision 1245726) Result = FAILURE jdcryans : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2665 (See https://builds.apache.org/job/HBase-TRUNK/2665/)
        HBASE-5393 Consider splitting after flushing (Revision 1245727)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2665 (See https://builds.apache.org/job/HBase-TRUNK/2665/ ) HBASE-5393 Consider splitting after flushing (Revision 1245727) Result = FAILURE jdcryans : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #114 (See https://builds.apache.org/job/HBase-TRUNK-security/114/)
        HBASE-5393 Consider splitting after flushing (Revision 1245727)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #114 (See https://builds.apache.org/job/HBase-TRUNK-security/114/ ) HBASE-5393 Consider splitting after flushing (Revision 1245727) Result = FAILURE jdcryans : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.92 #287 (See https://builds.apache.org/job/HBase-0.92/287/)
        HBASE-5393 Consider splitting after flushing (Revision 1245726)

        Result = SUCCESS
        jdcryans :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Show
        Hudson added a comment - Integrated in HBase-0.92 #287 (See https://builds.apache.org/job/HBase-0.92/287/ ) HBASE-5393 Consider splitting after flushing (Revision 1245726) Result = SUCCESS jdcryans : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Hide
        Jean-Daniel Cryans added a comment -

        Committed to trunk and 0.92, thanks for the votes and reviews guys.

        Show
        Jean-Daniel Cryans added a comment - Committed to trunk and 0.92, thanks for the votes and reviews guys.
        Hide
        Lars Hofhansl added a comment -

        +1 on patch and +1 on 0.92.x

        Show
        Lars Hofhansl added a comment - +1 on patch and +1 on 0.92.x
        Hide
        stack added a comment -

        +1 on putting in 0.92 too...

        Show
        stack added a comment - +1 on putting in 0.92 too...
        Hide
        Jean-Daniel Cryans added a comment -

        Patch that I posted in HBASE-2375 that Stack +1'd.

        Show
        Jean-Daniel Cryans added a comment - Patch that I posted in HBASE-2375 that Stack +1'd.

          People

          • Assignee:
            Jean-Daniel Cryans
            Reporter:
            Jean-Daniel Cryans
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development