Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-10201

Port 'Make flush decisions per column family' to trunk

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.1.0, 2.0.0
    • wal
    • None
    • Reviewed
    • Hide
      Adds new flushing policy mechanism. Default, org.apache.hadoop.hbase.regionserver.FlushLargeStoresPolicy, will try to avoid flushing out the small column families in a region, those whose memstores are < hbase.hregion.percolumnfamilyflush.size.lower.bound. To restore the old behavior of flushes writing out all column families, set hbase.regionserver.flush.policy to org.apache.hadoop.hbase.regionserver.FlushAllStoresPolicy either in hbase-default.xml or on a per-table basis by setting the policy to use with HTableDescriptor.getFlushPolicyClassName().
      Show
      Adds new flushing policy mechanism. Default, org.apache.hadoop.hbase.regionserver.FlushLargeStoresPolicy, will try to avoid flushing out the small column families in a region, those whose memstores are < hbase.hregion.percolumnfamilyflush.size.lower.bound. To restore the old behavior of flushes writing out all column families, set hbase.regionserver.flush.policy to org.apache.hadoop.hbase.regionserver.FlushAllStoresPolicy either in hbase-default.xml or on a per-table basis by setting the policy to use with HTableDescriptor.getFlushPolicyClassName().

    Description

      Currently the flush decision is made using the aggregate size of all column families. When large and small column families co-exist, this causes many small flushes of the smaller CF. We need to make per-CF flush decisions.

      Attachments

        1. 3149-trunk-v1.txt
          89 kB
          Ted Yu
        2. HBASE-10201-0.98.patch
          110 kB
          Duo Zhang
        3. HBASE-10201-0.98_1.patch
          114 kB
          Duo Zhang
        4. HBASE-10201-0.98_2.patch
          115 kB
          Duo Zhang
        5. HBASE-10201.patch
          105 kB
          Duo Zhang
        6. HBASE-10201_1.patch
          103 kB
          Duo Zhang
        7. HBASE-10201_2.patch
          104 kB
          Duo Zhang
        8. HBASE-10201_3.patch
          104 kB
          Duo Zhang
        9. HBASE-10201-0.99.patch
          104 kB
          Duo Zhang
        10. HBASE-10201_4.patch
          105 kB
          Duo Zhang
        11. HBASE-10201_5.patch
          104 kB
          Duo Zhang
        12. HBASE-10201_6.patch
          104 kB
          Duo Zhang
        13. HBASE-10201_7.patch
          124 kB
          Duo Zhang
        14. HBASE-10201_8.patch
          127 kB
          Duo Zhang
        15. HBASE-10201_9.patch
          135 kB
          Duo Zhang
        16. HBASE-10201_10.patch
          128 kB
          Duo Zhang
        17. HBASE-10201_11.patch
          128 kB
          Duo Zhang
        18. HBASE-10201_12.patch
          131 kB
          Duo Zhang
        19. HBASE-10201_13.patch
          135 kB
          Duo Zhang
        20. HBASE-10201_13.patch
          135 kB
          Michael Stack
        21. memstore.png
          21 kB
          Michael Stack
        22. count.png
          17 kB
          Michael Stack
        23. io.png
          18 kB
          Michael Stack
        24. compactions.png
          31 kB
          Michael Stack
        25. HBASE-10201_14.patch
          138 kB
          Duo Zhang
        26. HBASE-10201_15.patch
          138 kB
          Duo Zhang
        27. HBASE-10201_16.patch
          138 kB
          Duo Zhang
        28. HBASE-10201_17.patch
          138 kB
          Duo Zhang
        29. HBASE-10201_18.patch
          139 kB
          Duo Zhang
        30. HBASE-10201_19.patch
          144 kB
          Duo Zhang
        31. 10201-addendum.txt
          1 kB
          Ted Yu
        32. HBASE-10201-addendum_1.patch
          1 kB
          Duo Zhang

        Issue Links

          relates to

          New Feature - A new feature of the product, which has yet to be developed. HBASE-13408 HBase In-Memory Memstore Compaction

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-3149 Make flush decisions per column family

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-12405 WAL accounting by Store

          • Major - Major loss of function.
          • Closed
          links to

          Web Link reviewboard

          Activity

            People

              zhangduo Duo Zhang
              yuzhihong@gmail.com Ted Yu
              Votes:
              2 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: