HBase
  1. HBase
  2. HBASE-5699

Run with > 1 WAL in HRegionServer

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.0.0, 2.0.0, 1.1.0
    • Component/s: Performance, wal
    • Labels:
      None
    • Release Note:
      Hide
      HBase's write-ahead-log (WAL) can now be configured to use multiple HDFS pipelines in parallel to provide better write throughput for clusters by using additional disks. By default, HBase will still use only a single HDFS-based WAL.

      To run with multiple WALs, alter the hbase-site.xml property "hbase.wal.provider" to have the value "multiwal". To return to having HBase determine what kind of WAL implementation to use either remove the property all together or set it to "defaultProvider".

      Altering the WAL provider used by a particular RegionServer requires restarting that instance. RegionServers using the original WAL implementation and those using the "multiwal" implementation can each handle recovery of either set of WALs, so a zero-downtime configuration update is possible through a rolling restart.
      Show
      HBase's write-ahead-log (WAL) can now be configured to use multiple HDFS pipelines in parallel to provide better write throughput for clusters by using additional disks. By default, HBase will still use only a single HDFS-based WAL. To run with multiple WALs, alter the hbase-site.xml property "hbase.wal.provider" to have the value "multiwal". To return to having HBase determine what kind of WAL implementation to use either remove the property all together or set it to "defaultProvider". Altering the WAL provider used by a particular RegionServer requires restarting that instance. RegionServers using the original WAL implementation and those using the "multiwal" implementation can each handle recovery of either set of WALs, so a zero-downtime configuration update is possible through a rolling restart.

      Issue Links

        Activity

        Enis Soztutar made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Enis Soztutar made changes -
        Fix Version/s 1.0.0 [ 12325852 ]
        Sean Busbey made changes -
        Release Note HBase's write-ahead-log (WAL) can now be configured to use multiple HDFS pipelines in parallel to provide better write throughput for clusters by using additional disks. By default, HBase will still use only a single HDFS-based WAL.

        To run with multiple WALs, alter the hbase-site.xml property "hbase.wal.provider" to have the value "multiwal". To return to having HBase determine what kind of WAL implementation to use either remove the property all together or set it to "defaultProvider."

        Altering the WAL provider used by a particular RegionServer requires restarting that instance. RegionServers using the original WAL implementation and those using the "multiwal" implementation can each handle recovery of either set of WALs, so a zero-downtime configuration update is possible through a rolling restart.
        HBase's write-ahead-log (WAL) can now be configured to use multiple HDFS pipelines in parallel to provide better write throughput for clusters by using additional disks. By default, HBase will still use only a single HDFS-based WAL.

        To run with multiple WALs, alter the hbase-site.xml property "hbase.wal.provider" to have the value "multiwal". To return to having HBase determine what kind of WAL implementation to use either remove the property all together or set it to "defaultProvider".

        Altering the WAL provider used by a particular RegionServer requires restarting that instance. RegionServers using the original WAL implementation and those using the "multiwal" implementation can each handle recovery of either set of WALs, so a zero-downtime configuration update is possible through a rolling restart.
        Sean Busbey made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 2.0.0 [ 12327188 ]
        Fix Version/s 1.1.0 [ 12329043 ]
        Resolution Fixed [ 1 ]
        Sean Busbey made changes -
        Release Note HBase's write-ahead-log (WAL) can now be configured to use multiple HDFS pipelines in parallel to provide better write throughput for clusters by using additional disks. By default, HBase will still use only a single HDFS-based WAL.

        To run with multiple WALs, alter the hbase-site.xml property "hbase.wal.provider" to have the value "multiwal". To return to having HBase determine what kind of WAL implementation to use either remove the property all together or set it to "defaultProvider."

        Altering the WAL provider used by a particular RegionServer requires restarting that instance. RegionServers using the original WAL implementation and those using the "multiwal" implementation can each handle recovery of either set of WALs, so a zero-downtime configuration update is possible through a rolling restart.
        Sean Busbey made changes -
        Sean Busbey made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Sean Busbey made changes -
        Attachment HBASE-5699.4.patch.txt [ 12687232 ]
        Sean Busbey made changes -
        Attachment HBASE-5699.3.patch.txt [ 12686318 ]
        Sean Busbey made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Sean Busbey made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Sean Busbey made changes -
        Remote Link This issue links to "reviewboard (Web Link)" [ 20284 ]
        Sean Busbey made changes -
        Link This issue relates to HBASE-10378 [ HBASE-10378 ]
        Sean Busbey made changes -
        Link This issue depends upon HBASE-10378 [ HBASE-10378 ]
        Sean Busbey made changes -
        Assignee Li Pi [ li ] Sean Busbey [ busbey ]
        Sean Busbey made changes -
        Link This issue relates to HBASE-10378 [ HBASE-10378 ]
        Sean Busbey made changes -
        Link This issue relates to ACCUMULO-1083 [ ACCUMULO-1083 ]
        Anoop Sam John made changes -
        Component/s wal [ 12313572 ]
        Sean Busbey made changes -
        Link This issue is duplicated by HBASE-6981 [ HBASE-6981 ]
        Anoop Sam John made changes -
        Link This issue is related to HBASE-11270 [ HBASE-11270 ]
        stack made changes -
        Link This issue relates to HBASE-10278 [ HBASE-10278 ]
        stack made changes -
        Priority Major [ 3 ] Critical [ 2 ]
        Component/s Performance [ 12314193 ]
        Anoop Sam John made changes -
        Link This issue relates to HBASE-8610 [ HBASE-8610 ]
        Gavin made changes -
        Link This issue is related to HBASE-5937 [ HBASE-5937 ]
        Gavin made changes -
        Link This issue is related to HBASE-5937 [ HBASE-5937 ]
        stack made changes -
        Link This issue is depended upon by HBASE-8338 [ HBASE-8338 ]
        Jonathan Hsieh made changes -
        Link This issue is related too HBASE-5937 [ HBASE-5937 ]
        Lars Hofhansl made changes -
        Link This issue is related to HBASE-6116 [ HBASE-6116 ]
        ramkrishna.s.vasudevan made changes -
        Attachment PerfHbase.txt [ 12530779 ]
        Ted Yu made changes -
        Comment [ This seems interesting. I'll take a look at doing this. ]
        Li Pi made changes -
        Assignee Li Pi [ li ]
        stack made changes -
        Field Original Value New Value
        Summary Should we use muti HLog or Writer in HLog in a HRegionServer Run with > 1 WAL in HRegionServer
        binlijin created issue -

          People

          • Assignee:
            Sean Busbey
            Reporter:
            binlijin
          • Votes:
            0 Vote for this issue
            Watchers:
            55 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development