Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14388

Compaction improvements: Avoid flush storms by jittering flush interval and max log files

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Hide
      Periodic memstore flush changes

      New configuration option: hbase.regionserver.flush.jitter. Default value: 0.5. This affects memstore flush interval (hbase.regionserver.optionalcacheflushinterval) and maximum changes per flush (hbase.regionserver.flush.per.changes) by randomizing them. After every flush operation, the new random value is assigned to memstore flush interval and to maximum changes per flush.

      Maximum WAL files changes

      The configuration option hbase.regionserver.maxlogs is obsolete. The maximum number of WAL files is defined now dynamically:

       maxLogs = HEAP_SIZE * memstoreRatio * 2/ LogRollSize
      Show
      Periodic memstore flush changes New configuration option: hbase.regionserver.flush.jitter. Default value: 0.5. This affects memstore flush interval (hbase.regionserver.optionalcacheflushinterval) and maximum changes per flush (hbase.regionserver.flush.per.changes) by randomizing them. After every flush operation, the new random value is assigned to memstore flush interval and to maximum changes per flush. Maximum WAL files changes The configuration option hbase.regionserver.maxlogs is obsolete. The maximum number of WAL files is defined now dynamically:  maxLogs = HEAP_SIZE * memstoreRatio * 2/ LogRollSize

      Attachments

      1. HBASE-14388-v2.patch
        8 kB
        Vladimir Rodionov
      2. HBASE-14388-v1.patch
        8 kB
        Vladimir Rodionov

        Issue Links

        is part of

        Improvement - An improvement or enhancement to an existing feature or task. HBASE-14383 Compaction improvements

        • Major - Major loss of function.
        • Open

          Activity

            People

            • Assignee:
              vrodionov Vladimir Rodionov
              Reporter:
              vrodionov Vladimir Rodionov
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated: