Uploaded image for project: 'Jackrabbit Oak'
  1. Jackrabbit Oak
  2. OAK-2862

CompactionMap#compress() inefficient for large compaction maps

    XMLWordPrintableJSON

Details

    Description

      I've seen CompactionMap#compress() take up most of the time spent in compaction. With 40M record ids in the compaction map compressing runs for hours.

      I will back this with numbers as soon as I have a better grip on the issue.

      Attachments

        1. OAK-2862-memory.png
          532 kB
          Michael Dürig
        2. OAK-2862.png
          330 kB
          Michael Dürig
        3. benchLargeMap.xlsx
          100 kB
          Michael Dürig

        Issue Links

          is depended upon by

          Bug - A problem which impairs or prevents the functions of the product. OAK-3139 SNFE in persisted comapation map when using CLEAN_OLD

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. OAK-3179 Deadlock between persisted compaction map and cleanup

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. OAK-3264 Deadlock between persisted compaction map and cleanup 2

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. OAK-3317 ConcurrentModificationException when running SegmentOverflowExceptionIT

          • Major - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. OAK-2896 Putting many elements into a map results in many small segments.

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. OAK-2880 NPE in SegmentWriter.writeMap

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. OAK-3123 NPE in RecordIdMap

          • Major - Major loss of function.
          • Closed
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. OAK-2713 High memory usage of CompactionMap

          • Major - Major loss of function.
          • Closed

          Activity

            People

              mduerig Michael Dürig
              mduerig Michael Dürig
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: