Uploaded image for project: 'Jackrabbit Oak'
  1. Jackrabbit Oak
  2. OAK-4256

CLONE - Cross gc sessions might introduce references to pre-compacted segments

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Critical
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: segmentmk

      Description

      I suspect that certain write operations during compaction can cause references from compacted segments to pre-compacted ones. This would effectively prevent the pre-compacted segments from getting evicted in subsequent cleanup phases.

      The scenario is as follows:

      • A session is opened and a lot of content is written to it such that the update limit is exceeded. This causes the changes to be written to disk.
      • Revision gc runs causing a new, compacted root node state to be written to disk.
      • The session saves its changes. This causes rebasing of its changes onto the current root (the compacted one). At this point any node that has been added will be added again in the sub-tree rooted at the current root. Such nodes however might have been written to disk before revision gc ran and might thus be contained in pre-compacted segments. As I suspect the node-add operation in the rebasing process not to create a deep copy of such nodes but to rather create a reference to them, a reference to a pre-compacted segment is introduced here.

      Going forward we need to validate above hypothesis, assess its impact if necessary come up with a solution.

        Attachments

        1. repo-size.png
          43 kB
          Michael Dürig
        2. image.png
          6 kB
          Michael Dürig
        3. cross-gc-refs.pdf
          774 kB
          Michael Dürig
        4. compaction-time.png
          43 kB
          Michael Dürig
        5. cleanup-time.png
          44 kB
          Michael Dürig
        6. SCIT.patch
          3 kB
          Michael Dürig
        7. OAK-3348.patch
          7 kB
          Michael Dürig
        8. OAK-3348-2.patch
          24 kB
          Michael Dürig
        9. OAK-3348-1.patch
          16 kB
          Michael Dürig

          Issue Links

            Activity

              People

              • Assignee:
                mduerig Michael Dürig
                Reporter:
                mduerig Michael Dürig
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: