Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14350 Procedure V2 Phase 2: Assignment Manager
  3. HBASE-16094

Procedure v2 - Improve cleaning up of proc wals

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0
    • proc-v2
    • None

    Description

      Avoid accumulating too many wals.
      We remove logs in 3 cases:

      • No procedures are running. We can remove all the logs.
      • All procedures are updated/written in the last log. We can remove all the logs, aside the active one.
      • Remove log if does not contains “active” procIds

      https://github.com/apache/hbase/blob/master/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java#L865
      The last point, Remove log if does not contains “active” procIds, needs to be improved. Basically, even if a log contains state of an active proc, we can delete it if a later log contains a newer state.

      Attachments

        1. HBASE-16094.master.006.patch
          32 kB
          Apekshit Sharma
        2. HBASE-16094.master.005.patch
          32 kB
          Apekshit Sharma
        3. HBASE-16094.master.004.patch
          32 kB
          Apekshit Sharma
        4. HBASE-16094.master.003.patch
          32 kB
          Apekshit Sharma
        5. HBASE-16094.master.002.patch
          26 kB
          Apekshit Sharma
        6. HBASE-16094.master.001.patch
          23 kB
          Apekshit Sharma

        Issue Links

          Activity

            People

              appy Apekshit Sharma
              appy Apekshit Sharma
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: