Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-22628

Document the custom WAL directory (hbase.wal.dir) usage

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0
    • Component/s: documentation, wal
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Custom WAL directory usage must be documented, otherwise it may lead to inconsistent data during migrating to new WAL dir path.

       

      You can consider below scenario while migrating to custom WAL directory.

      1. Setup HBase cluster with the default setting (all WAL files are under the root directory ie. /hbase/WALs).
      2. Create table 't1' and insert few records
      3. Flush meta table (so that table region entries persist in FS)
      4. Forcibly kill HBase processes (HM & RS).
      5. Configure the hbase.wal.dir to outside the root dir (say /hbaseWAL)
      6. Start the HBase servers
      7. Scan 't1'

      Ideally HMaster should submit split task of old RS(s) WAL files (created under /hbase/WALs) and old data should be replayed. But currently, during HM startup we populate the previous dead servers from the current WAL dir ( hbase.wal.dir -> /hbaseWAL).

       

      Since WAL dir path is new, so you need to copy RegionServer WAL directories manualy from old WAL dir to new path. 

        Attachments

        1. Screenshot-HBASE-22628.png
          93 kB
          Pankaj Kumar
        2. HBASE-22628.patch
          2 kB
          Pankaj Kumar
        3. HBASE-22628.002.patch
          2 kB
          Pankaj Kumar
        4. 002-view.png
          182 kB
          Reid Chan

          Issue Links

            Activity

              People

              • Assignee:
                pankaj2461 Pankaj Kumar
                Reporter:
                pankaj2461 Pankaj Kumar
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: