Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14790 Implement a new DFSOutputStream for logging WAL only
  3. HBASE-15536

Make AsyncFSWAL as our default WAL

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 2.0.0
    • 2.0.0-beta-1, 2.0.0
    • wal
    • None
    • Reviewed
    • Hide
      Now the default WALProvider is AsyncFSWALProvider, i.e. 'asyncfs'.
      If you want to change back to use FSHLog, please add this in hbase-site.xml
      {code}
      <property>
      <name>hbase.wal.provider</name>
      <value>filesystem</value>
      </property>
      {code}
      If you want to use FSHLog with multiwal, please add this in hbase-site.xml
      {code}
      <property>
      <name>hbase.wal.regiongrouping.delegate.provider</name>
      <value>filesystem</value>
      </property>
      {code}

      This patch also sets hbase.wal.async.use-shared-event-loop to false so WAL has its own netty event group.
      Show
      Now the default WALProvider is AsyncFSWALProvider, i.e. 'asyncfs'. If you want to change back to use FSHLog, please add this in hbase-site.xml {code} <property> <name>hbase.wal.provider</name> <value>filesystem</value> </property> {code} If you want to use FSHLog with multiwal, please add this in hbase-site.xml {code} <property> <name>hbase.wal.regiongrouping.delegate.provider</name> <value>filesystem</value> </property> {code} This patch also sets hbase.wal.async.use-shared-event-loop to false so WAL has its own netty event group.

    Description

      As it should be predicated on passing basic cluster ITBLL

      Attachments

        1. latesttrunk_defaultWAL_50threads_10cols.jfr
          10.89 MB
          ramkrishna.s.vasudevan
        2. latesttrunk_asyncWAL_50threads_10cols.jfr
          8.53 MB
          ramkrishna.s.vasudevan
        3. HBASE-15536-v5.patch
          274 kB
          Duo Zhang
        4. HBASE-15536-v4.patch
          276 kB
          Duo Zhang
        5. HBASE-15536-v3.patch
          274 kB
          Duo Zhang
        6. HBASE-15536-v2.patch
          140 kB
          Duo Zhang
        7. HBASE-15536-v1.patch
          141 kB
          Duo Zhang
        8. HBASE-15536.patch
          135 kB
          Duo Zhang
        9. 15536.minor.addendum.patch
          1 kB
          Michael Stack
        10. 15536.addendum2.enable.asyncfswal.by.default.txt
          2 kB
          Michael Stack
        11. 15536.addendum2.enable.asyncfswal.by.default.txt
          2 kB
          Duo Zhang
        12. 15536.addendum2.enable.asyncfswal.by.default.txt
          2 kB
          Duo Zhang
        13. 15536.addendum2.enable.asyncfswal.by.default.txt
          2 kB
          Duo Zhang
        14. 15536.addendum2.enable.asyncfswal.by.default.3.txt
          3 kB
          Michael Stack
        15. 15536.addendum2.enable.asyncfswal.by.default.2.txt
          3 kB
          Michael Stack
        16. 15536.addendum2.enable.asyncfswal.by.default.2.txt
          3 kB
          Duo Zhang

        Issue Links

          is blocked by

          Sub-task - The sub-task of the issue HBASE-16890 Analyze the performance of AsyncWAL and fix the same

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Sub-task - The sub-task of the issue HBASE-15709 Handle large edits for asynchronous WAL

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Sub-task - The sub-task of the issue HBASE-15628 Implement an AsyncOutputStream which can work with any FileSystem implementation

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HBASE-19510 TestDistributedLogSplitting is flakey for AsyncFSWAL

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HBASE-19493 Make TestWALMonotonicallyIncreasingSeqId also work with AsyncFSWAL

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HBASE-19503 Fix TestWALOpenAfterDNRollingStart for AsyncFSWAL

          • Major - Major loss of function.
          • Closed

          Activity

            People

              zhangduo Duo Zhang
              zhangduo Duo Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: