Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14790 Implement a new DFSOutputStream for logging WAL only
  3. HBASE-15536

Make AsyncFSWAL as our default WAL

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 2.0.0
    • 2.0.0-beta-1, 2.0.0
    • wal
    • None
    • Reviewed
    • Hide
      Now the default WALProvider is AsyncFSWALProvider, i.e. 'asyncfs'.
      If you want to change back to use FSHLog, please add this in hbase-site.xml
      {code}
      <property>
      <name>hbase.wal.provider</name>
      <value>filesystem</value>
      </property>
      {code}
      If you want to use FSHLog with multiwal, please add this in hbase-site.xml
      {code}
      <property>
      <name>hbase.wal.regiongrouping.delegate.provider</name>
      <value>filesystem</value>
      </property>
      {code}

      This patch also sets hbase.wal.async.use-shared-event-loop to false so WAL has its own netty event group.
      Show
      Now the default WALProvider is AsyncFSWALProvider, i.e. 'asyncfs'. If you want to change back to use FSHLog, please add this in hbase-site.xml {code} <property> <name>hbase.wal.provider</name> <value>filesystem</value> </property> {code} If you want to use FSHLog with multiwal, please add this in hbase-site.xml {code} <property> <name>hbase.wal.regiongrouping.delegate.provider</name> <value>filesystem</value> </property> {code} This patch also sets hbase.wal.async.use-shared-event-loop to false so WAL has its own netty event group.

    Description

      As it should be predicated on passing basic cluster ITBLL

      Attachments

        1. 15536.addendum2.enable.asyncfswal.by.default.2.txt
          3 kB
          Duo Zhang
        2. 15536.addendum2.enable.asyncfswal.by.default.2.txt
          3 kB
          Michael Stack
        3. 15536.addendum2.enable.asyncfswal.by.default.3.txt
          3 kB
          Michael Stack
        4. 15536.addendum2.enable.asyncfswal.by.default.txt
          2 kB
          Duo Zhang
        5. 15536.addendum2.enable.asyncfswal.by.default.txt
          2 kB
          Duo Zhang
        6. 15536.addendum2.enable.asyncfswal.by.default.txt
          2 kB
          Duo Zhang
        7. 15536.addendum2.enable.asyncfswal.by.default.txt
          2 kB
          Michael Stack
        8. 15536.minor.addendum.patch
          1 kB
          Michael Stack
        9. HBASE-15536.patch
          135 kB
          Duo Zhang
        10. HBASE-15536-v1.patch
          141 kB
          Duo Zhang
        11. HBASE-15536-v2.patch
          140 kB
          Duo Zhang
        12. HBASE-15536-v3.patch
          274 kB
          Duo Zhang
        13. HBASE-15536-v4.patch
          276 kB
          Duo Zhang
        14. HBASE-15536-v5.patch
          274 kB
          Duo Zhang
        15. latesttrunk_asyncWAL_50threads_10cols.jfr
          8.53 MB
          ramkrishna.s.vasudevan
        16. latesttrunk_defaultWAL_50threads_10cols.jfr
          10.89 MB
          ramkrishna.s.vasudevan

        Issue Links

          is blocked by

          Sub-task - The sub-task of the issue HBASE-16890 Analyze the performance of AsyncWAL and fix the same

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Sub-task - The sub-task of the issue HBASE-15709 Handle large edits for asynchronous WAL

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Sub-task - The sub-task of the issue HBASE-15628 Implement an AsyncOutputStream which can work with any FileSystem implementation

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HBASE-19510 TestDistributedLogSplitting is flakey for AsyncFSWAL

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HBASE-19493 Make TestWALMonotonicallyIncreasingSeqId also work with AsyncFSWAL

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HBASE-19503 Fix TestWALOpenAfterDNRollingStart for AsyncFSWAL

          • Major - Major loss of function.
          • Closed

          Activity

            People

              zhangduo Duo Zhang
              zhangduo Duo Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: