[OAK-7838] oak-run check crashes JVM

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.10.0, 1.9.10
• Component/s: run, segment-tar
• Assignee: Michael Dürig
• Reporter: Michael Dürig
• Created: 16/Oct/18 08:17
• Updated: 04/Oct/19 17:50
• Resolved: 30/Oct/18 12:59

Description

I had a case where running oak-run check on a repository with many revisions would reliably crash the JVM.

Apparently there is a problem with the Scheduler instances in org.apache.jackrabbit.oak.segment.CommitsTracker: when many instances of that class are created in fast succession it will leave many daemon threads lingering around for a while. In my case this was sufficient to kill the JVM.
To verify I simply removed the scheduler and everything was just fine:

===================================================================
--- oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/CommitsTracker.java    (date 1539358293000)
+++ oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/CommitsTracker.java    (date 1539670356000)
@@ -19,8 +19,6 @@

package org.apache.jackrabbit.oak.segment;

-import static java.util.concurrent.TimeUnit.MINUTES;
-
import java.io.Closeable;
import java.util.HashMap;
import java.util.Map;
@@ -29,7 +27,6 @@
import java.util.stream.Stream;

import com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap;
-import org.apache.jackrabbit.oak.segment.file.Scheduler;

/**
 * A simple tracker for the source of commits (writes) in
@@ -49,7 +46,6 @@
    private final ConcurrentMap<String, Long> commitsCountPerThreadGroup;
    private final ConcurrentMap<String, Long> commitsCountOtherThreads;
    private final ConcurrentMap<String, Long> commitsCountPerThreadGroupLastMinute;
-    private final Scheduler commitsTrackerScheduler = new Scheduler("CommitsTracker background tasks");

    CommitsTracker(String[] threadGroups, int otherWritersLimit, boolean collectStackTraces) {
        this.threadGroups = threadGroups;
@@ -60,8 +56,6 @@
                .maximumWeightedCapacity(otherWritersLimit).build();
        this.queuedWritersMap = new ConcurrentHashMap<>();

-        commitsTrackerScheduler.scheduleWithFixedDelay("TarMK commits tracker stats resetter", 1, MINUTES,
-                this::resetStatistics);
    }

    public void trackQueuedCommitOf(Thread t) {
@@ -112,7 +106,7 @@

    @Override
    public void close() {
-        commitsTrackerScheduler.close();
+
    }

cc dulceanu

Issue Links

• causes: OAK-7885 Performance regression in FlatTreeUpdateTest