Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-10201

Port 'Make flush decisions per column family' to trunk

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0, 2.0.0
    • Component/s: wal
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Adds new flushing policy mechanism. Default, org.apache.hadoop.hbase.regionserver.FlushLargeStoresPolicy, will try to avoid flushing out the small column families in a region, those whose memstores are < hbase.hregion.percolumnfamilyflush.size.lower.bound. To restore the old behavior of flushes writing out all column families, set hbase.regionserver.flush.policy to org.apache.hadoop.hbase.regionserver.FlushAllStoresPolicy either in hbase-default.xml or on a per-table basis by setting the policy to use with HTableDescriptor.getFlushPolicyClassName().
      Show
      Adds new flushing policy mechanism. Default, org.apache.hadoop.hbase.regionserver.FlushLargeStoresPolicy, will try to avoid flushing out the small column families in a region, those whose memstores are < hbase.hregion.percolumnfamilyflush.size.lower.bound. To restore the old behavior of flushes writing out all column families, set hbase.regionserver.flush.policy to org.apache.hadoop.hbase.regionserver.FlushAllStoresPolicy either in hbase-default.xml or on a per-table basis by setting the policy to use with HTableDescriptor.getFlushPolicyClassName().

      Description

      Currently the flush decision is made using the aggregate size of all column families. When large and small column families co-exist, this causes many small flushes of the smaller CF. We need to make per-CF flush decisions.

        Attachments

        1. 10201-addendum.txt
          1 kB
          Ted Yu
        2. 3149-trunk-v1.txt
          89 kB
          Ted Yu
        3. compactions.png
          31 kB
          Michael Stack
        4. count.png
          17 kB
          Michael Stack
        5. HBASE-10201_1.patch
          103 kB
          Duo Zhang
        6. HBASE-10201_10.patch
          128 kB
          Duo Zhang
        7. HBASE-10201_11.patch
          128 kB
          Duo Zhang
        8. HBASE-10201_12.patch
          131 kB
          Duo Zhang
        9. HBASE-10201_13.patch
          135 kB
          Michael Stack
        10. HBASE-10201_13.patch
          135 kB
          Duo Zhang
        11. HBASE-10201_14.patch
          138 kB
          Duo Zhang
        12. HBASE-10201_15.patch
          138 kB
          Duo Zhang
        13. HBASE-10201_16.patch
          138 kB
          Duo Zhang
        14. HBASE-10201_17.patch
          138 kB
          Duo Zhang
        15. HBASE-10201_18.patch
          139 kB
          Duo Zhang
        16. HBASE-10201_19.patch
          144 kB
          Duo Zhang
        17. HBASE-10201_2.patch
          104 kB
          Duo Zhang
        18. HBASE-10201_3.patch
          104 kB
          Duo Zhang
        19. HBASE-10201_4.patch
          105 kB
          Duo Zhang
        20. HBASE-10201_5.patch
          104 kB
          Duo Zhang
        21. HBASE-10201_6.patch
          104 kB
          Duo Zhang
        22. HBASE-10201_7.patch
          124 kB
          Duo Zhang
        23. HBASE-10201_8.patch
          127 kB
          Duo Zhang
        24. HBASE-10201_9.patch
          135 kB
          Duo Zhang
        25. HBASE-10201.patch
          105 kB
          Duo Zhang
        26. HBASE-10201-0.98_1.patch
          114 kB
          Duo Zhang
        27. HBASE-10201-0.98_2.patch
          115 kB
          Duo Zhang
        28. HBASE-10201-0.98.patch
          110 kB
          Duo Zhang
        29. HBASE-10201-0.99.patch
          104 kB
          Duo Zhang
        30. HBASE-10201-addendum_1.patch
          1 kB
          Duo Zhang
        31. io.png
          18 kB
          Michael Stack
        32. memstore.png
          21 kB
          Michael Stack

          Issue Links

            Activity

              People

              • Assignee:
                zhangduo Duo Zhang
                Reporter:
                yuzhihong@gmail.com Ted Yu
              • Votes:
                2 Vote for this issue
                Watchers:
                20 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: