Uploaded image for project: 'Jackrabbit Oak'
  1. Jackrabbit Oak
  2. OAK-4949

SegmentWriter buffers child node list changes

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Not A Bug
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: segment-tar
    • Labels:

      Description

      The SegmentWriter currently buffers the list of child nodes changed on a nodestate update [0] (new node or updated node). This can be problematic in a scenario where there are a large number of children added to a node (ie. unique index size seen to spike above 10MM in one case).

      To have a reference for the impact of this, at the SegmentWriter level, for a list of map entries of almost 3MM items, I saw it take up around 245MB heap.

      This issue serves to track a possible improvement here in how we handle this update scenario.

      [0] https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/SegmentWriter.java#L516

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mduerig Michael Dürig
                Reporter:
                stillalex Alex Deparvu
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: