Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14969

Add throughput controller for flush

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0, 2.0.0
    • Fix Version/s: 1.3.0, 2.0.0
    • Component/s: regionserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Adds means of throttling flush throughput. By default there is no limit; we use NoLimitThroughputController. An alternative controller, PressureAwareFlushThroughputController, allows specifying throughput bounds. A new simple factor, flush pressure, influences throughput. See PressureAwareFlushThroughputController.java class for detail.
      Show
      Adds means of throttling flush throughput. By default there is no limit; we use NoLimitThroughputController. An alternative controller, PressureAwareFlushThroughputController, allows specifying throughput bounds. A new simple factor, flush pressure, influences throughput. See PressureAwareFlushThroughputController.java class for detail.

      Description

      In HBASE-8329 we added a throughput controller for compaction, to avoid spike caused by huge IO pressure like network/disk overflow. However, even with this control on, we are still observing disk utils near 100%, and by analysis we think this is caused by flush, especially when we increase the setting of hbase.hstore.flusher.count

      In this JIRA, we propose to add throughput control feature for flush, as a supplement of HBASE-8329 to better control IO pressure.

        Attachments

        1. fd78628_9909808_compat_report.html
          121 kB
          Yu Li
        2. HBASE-14969_v10.patch
          169 kB
          Yu Li
        3. HBASE-14969_v2.patch
          110 kB
          Yu Li
        4. HBASE-14969_v3.patch
          110 kB
          Yu Li
        5. HBASE-14969_v4.patch
          110 kB
          Yu Li
        6. HBASE-14969_v5.patch
          111 kB
          Yu Li
        7. HBASE-14969_v6.patch
          111 kB
          Yu Li
        8. HBASE-14969_v9.patch
          168 kB
          Yu Li
        9. HBASE-14969.branch-1.patch
          162 kB
          Yu Li
        10. HBASE-14969.patch
          109 kB
          Yu Li
        11. HBASE-14969-addendum-branch-1.patch
          2 kB
          Duo Zhang
        12. load-nothrottling.log
          261 kB
          Yu Li
        13. load-throttling.log
          260 kB
          Yu Li

          Issue Links

            Activity

              People

              • Assignee:
                liyu Yu Li
                Reporter:
                liyu Yu Li
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: