Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14350 Procedure V2 Phase 2: Assignment Manager
  3. HBASE-16524

Procedure v2 - Compute WALs cleanup on wal modification and not on every sync

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.0.0
    • 2.0.0
    • proc-v2
    • None

    Description

      Fix performance regression introduced by HBASE-16094.

      Instead of scanning all the wals every time, we can rely on the insert/update/delete events we have.
      and since we want to delete the wals in order we can keep track of what is "holding" that wal, and take a hit on scanning all the trackers only when we remove the first log in the queue.
      e.g.
      WAL-1 [1, 2]
      WAL-2 [1] -> "[2] is holding WAL-1"
      WAL-3 [2] -> "WAL 1 can be removed, recompute what is holding WAL-2"

      Attachments

        1. flame1.svg
          78 kB
          Apekshit Sharma
        2. HBASE-16524.master.001.patch
          3 kB
          Apekshit Sharma
        3. HBASE-16524.master.002.patch
          39 kB
          Michael Stack
        4. HBASE-16524-v2.patch
          29 kB
          Matteo Bertozzi
        5. HBASE-16524-v3.patch
          37 kB
          Matteo Bertozzi
        6. HBASE-16524-v4.patch
          37 kB
          Matteo Bertozzi
        7. HBASE-16524-v5.patch
          37 kB
          Matteo Bertozzi
        8. HBASE-16524-v6.patch
          39 kB
          Matteo Bertozzi

        Issue Links

          Activity

            People

              mbertozzi Matteo Bertozzi
              appy Apekshit Sharma
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: