Uploaded image for project: 'Jackrabbit Oak'
  1. Jackrabbit Oak
  2. OAK-3349

Partial compaction

    XMLWordPrintableJSON

Details

    Description

      On big repositories compaction can take quite a while to run as it needs to create a full deep copy of the current root node state. For such cases it could be beneficial if we could partially compact the repository thus splitting full compaction over multiple cycles.
      Partial compaction would run compaction on a sub-tree just like we now run it on the full tree. Afterwards it would create a new root node state by referencing the previous root node state replacing said sub-tree with the compacted one.

      Todo: Asses feasibility and impact, implement prototype.

      Attachments

        1. post-gc-size.png
          33 kB
          Michael Dürig
        2. cycle-count.png
          36 kB
          Michael Dürig
        3. compaction-time.png
          37 kB
          Michael Dürig

        Issue Links

          blocks

          Task - A task that needs to be done. OAK-6399 Re-establish full offline compaction functionality

          • Major - Major loss of function.
          • Closed
          depends upon

          Improvement - An improvement or enhancement to an existing feature or task. OAK-6009 Simplify cancellation of compaction by timeout

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. OAK-5996 Revisions.setHead(Function) should return the new head or null instead of boolean

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. OAK-6005 Add record id of the compacted root to the GC journal

          • Major - Major loss of function.
          • Closed
          is blocked by

          New Feature - A new feature of the product, which has yet to be developed. OAK-4122 Replace the commit semaphore in the segment node store with a scheduler

          • Major - Major loss of function.
          • Closed
          is related to

          New Feature - A new feature of the product, which has yet to be developed. OAK-6456 Include the tail generation in the TAR index

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. OAK-6457 Increment the segment version number

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. OAK-6458 Increment the manifest version number

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. OAK-6468 Include the tail generation in the binary references index

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. OAK-6484 Include tail generation in the GC journal

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. OAK-6485 Expose tail and full compaction via different methods

          • Major - Major loss of function.
          • Closed
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. OAK-6520 Improve tail compactions resilience when base state cannot be determined

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. OAK-6519 Properly handle tail compactions in deduplication caches

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. OAK-6522 Implement unit tests for OnlineCompactor

          • Major - Major loss of function.
          • Closed
          links to

          Web Link GitHub Pull Request #773

          Web Link GitHub Pull Request #1216

          Activity

            People

              frm Francesco Mari
              mduerig Michael Dürig
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: