Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.90.5
    • Fix Version/s: 0.92.1, 0.94.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Spawning this from HBASE-2375, I saw that it was much more efficient compaction-wise to check if we can split right after flushing. Much like the ideas that Jon spelled out in the description of that jira, the window is smaller because you don't have to compact and then split right away to only compact again when the daughters open.

      Another thing it improves is while we're normally waiting for the compaction to happen, data that's still coming in will make us go way past the MAX_FILESIZE to a point where for the first region I was seeing a store size 3-4x bigger before it was able to split.

      I targeted this for 0.94, but I'd like to get this into 0.92.1 or .2 too.

        Activity

        Hide
        hudson Hudson added a comment -

        Integrated in HBase-0.92-security #91 (See https://builds.apache.org/job/HBase-0.92-security/91/)
        HBASE-5393 Consider splitting after flushing (Revision 1245726)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Show
        hudson Hudson added a comment - Integrated in HBase-0.92-security #91 (See https://builds.apache.org/job/HBase-0.92-security/91/ ) HBASE-5393 Consider splitting after flushing (Revision 1245726) Result = FAILURE jdcryans : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Hide
        hudson Hudson added a comment -

        Integrated in HBase-TRUNK #2665 (See https://builds.apache.org/job/HBase-TRUNK/2665/)
        HBASE-5393 Consider splitting after flushing (Revision 1245727)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Show
        hudson Hudson added a comment - Integrated in HBase-TRUNK #2665 (See https://builds.apache.org/job/HBase-TRUNK/2665/ ) HBASE-5393 Consider splitting after flushing (Revision 1245727) Result = FAILURE jdcryans : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Hide
        hudson Hudson added a comment -

        Integrated in HBase-TRUNK-security #114 (See https://builds.apache.org/job/HBase-TRUNK-security/114/)
        HBASE-5393 Consider splitting after flushing (Revision 1245727)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Show
        hudson Hudson added a comment - Integrated in HBase-TRUNK-security #114 (See https://builds.apache.org/job/HBase-TRUNK-security/114/ ) HBASE-5393 Consider splitting after flushing (Revision 1245727) Result = FAILURE jdcryans : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Hide
        hudson Hudson added a comment -

        Integrated in HBase-0.92 #287 (See https://builds.apache.org/job/HBase-0.92/287/)
        HBASE-5393 Consider splitting after flushing (Revision 1245726)

        Result = SUCCESS
        jdcryans :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Show
        hudson Hudson added a comment - Integrated in HBase-0.92 #287 (See https://builds.apache.org/job/HBase-0.92/287/ ) HBASE-5393 Consider splitting after flushing (Revision 1245726) Result = SUCCESS jdcryans : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Hide
        jdcryans Jean-Daniel Cryans added a comment -

        Committed to trunk and 0.92, thanks for the votes and reviews guys.

        Show
        jdcryans Jean-Daniel Cryans added a comment - Committed to trunk and 0.92, thanks for the votes and reviews guys.
        Hide
        lhofhansl Lars Hofhansl added a comment -

        +1 on patch and +1 on 0.92.x

        Show
        lhofhansl Lars Hofhansl added a comment - +1 on patch and +1 on 0.92.x
        Hide
        stack stack added a comment -

        +1 on putting in 0.92 too...

        Show
        stack stack added a comment - +1 on putting in 0.92 too...
        Hide
        jdcryans Jean-Daniel Cryans added a comment -

        Patch that I posted in HBASE-2375 that Stack +1'd.

        Show
        jdcryans Jean-Daniel Cryans added a comment - Patch that I posted in HBASE-2375 that Stack +1'd.

          People

          • Assignee:
            jdcryans Jean-Daniel Cryans
            Reporter:
            jdcryans Jean-Daniel Cryans
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development