Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14457

Umbrella: Improve Multiple WAL for production usage

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.3.0, 2.0.0
    • None
    • None

    Description

      HBASE-5699 proposed the idea to run with multiple WAL in regionserver and did a great initial work there, but when trying to use it in our production cluster, we still found several issues to resolve, like tracking multiple WAL paths in replication (HBASE-6617), fixing UT with multiwal provider (HBASE-14411), introducing a namespace-based strategy for RegionGroupingProvider (HBASE-14456), etc. This is an umbrella including(but not limited of) all these works and efforts to make multiple wal ready for production usage and give user a clear picture about it.

      Besides the developing works done, I'd also like to share some scenarios and testing/online data in this JIRA about our usage/performance of multiple wal, to(hopefully) help people better judge whether to enable multiple wal or not in their own cluster and what they might gain.

      Attachments

        1. Action in Multiple WAL.pdf
          5.31 MB
          Yu Li
        2. Action in Multiple WAL.pdf
          5.32 MB
          Yu Li

        Issue Links

          contains

          Bug - A problem which impairs or prevents the functions of the product. HBASE-14411 Fix unit test failures when using multiwal as default WAL provider

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. HBASE-14456 Implement a namespace-based region grouping strategy for RegionGroupingProvider

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-6617 ReplicationSourceManager should be able to track multiple WAL paths

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-14306 Refine RegionGroupingProvider: fix issues and make it more scalable

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-14448 Refine RegionGroupingProvider Phase-2: remove provider nesting and formalize wal group name

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-15093 Replication can report incorrect size of log queue for the global source when multiwal is enabled

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HBASE-15131 Turn on multi-WAL by default

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-5699 Run with > 1 WAL in HRegionServer

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              liyu Yu Li
              liyu Yu Li
              Votes:
              0 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: