[OAK-7132] SNFE after full compaction

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.8.0
• Fix Version/s: 1.9.0, 1.10.0, 1.8.1
• Component/s: segment-tar
• Labels: compaction
• Epic Link: TarMK GC scalability
• Assignee: Michael Dürig
• Reporter: Michael Dürig
• Created: 10/Jan/18 09:44
• Updated: 25/Jun/19 06:32
• Resolved: 22/Jan/18 12:20

Description

In some cases we observed a SNFE right after a the cleanup following a full compaction:

31.12.2017 04:25:19.816 *ERROR* [pool-17-thread-22] org.apache.jackrabbit.oak.segment.SegmentNotFoundExceptionListener Segment not found: a82a99a3-f1e9-49b7-a1e0-55e7fec80c41. SegmentId age=609487478ms,segment-generation=GCGeneration{generation=4,fullGeneration=2,isCompacted=true}
org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment a82a99a3-f1e9-49b7-a1e0-55e7fec80c41 not found
        at org.apache.jackrabbit.oak.segment.file.AbstractFileStore.readSegmentUncached(AbstractFileStore.java:276)
        at org.apache.jackrabbit.oak.segment.file.FileStore.lambda$readSegment$5(FileStore.java:478)
        at org.apache.jackrabbit.oak.segment.SegmentCache.lambda$getSegment$0(SegmentCache.java:116)
        at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4724)
        at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3522)
        at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2315)
        at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2278)
        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2193)
        at com.google.common.cache.LocalCache.get(LocalCache.java:3932)
        at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4721)
        at org.apache.jackrabbit.oak.segment.SegmentCache.getSegment(SegmentCache.java:113)
        at org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:478)
        at org.apache.jackrabbit.oak.segment.SegmentId.getSegment(SegmentId.java:154)
        at org.apache.jackrabbit.oak.segment.CachingSegmentReader$1.apply(CachingSegmentReader.java:94)
        at org.apache.jackrabbit.oak.segment.CachingSegmentReader$1.apply(CachingSegmentReader.java:90)
        at org.apache.jackrabbit.oak.segment.ReaderCache.get(ReaderCache.java:118)
        at org.apache.jackrabbit.oak.segment.CachingSegmentReader.readString(CachingSegmentReader.java:90)
        at org.apache.jackrabbit.oak.segment.MapRecord.getEntry(MapRecord.java:220)
        at org.apache.jackrabbit.oak.segment.MapRecord.getEntry(MapRecord.java:173)
        at org.apache.jackrabbit.oak.segment.SegmentNodeState.getChildNode(SegmentNodeState.java:423)
        at org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.<init>(MemoryNodeBuilder.java:143)
        at org.apache.jackrabbit.oak.segment.SegmentNodeBuilder.<init>(SegmentNodeBuilder.java:93)
        at org.apache.jackrabbit.oak.segment.SegmentNodeBuilder.createChildBuilder(SegmentNodeBuilder.java:148)
        at org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.getChildNode(MemoryNodeBuilder.java:331)
        at org.apache.jackrabbit.oak.core.SecureNodeBuilder.<init>(SecureNodeBuilder.java:112)
        at org.apache.jackrabbit.oak.core.SecureNodeBuilder.getChildNode(SecureNodeBuilder.java:329)
        at org.apache.jackrabbit.oak.core.MutableTree.getTree(MutableTree.java:290)
        at org.apache.jackrabbit.oak.core.MutableRoot.getTree(MutableRoot.java:220)
        at org.apache.jackrabbit.oak.core.MutableRoot.getTree(MutableRoot.java:69)
        at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getItem(SessionDelegate.java:442)
        at org.apache.jackrabbit.oak.jcr.session.SessionImpl.getItemInternal(SessionImpl.java:167)
        at org.apache.jackrabbit.oak.jcr.session.SessionImpl.access$400(SessionImpl.java:82)
        at org.apache.jackrabbit.oak.jcr.session.SessionImpl$3.performNullable(SessionImpl.java:229)
        at org.apache.jackrabbit.oak.jcr.session.SessionImpl$3.performNullable(SessionImpl.java:226)
        at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performNullable(SessionDelegate.java:243)
        at org.apache.jackrabbit.oak.jcr.session.SessionImpl.getItemOrNull(SessionImpl.java:226)

Issue Links

• depends upon: OAK-7157 Minimize the amount of generations retained by the Cold Standby
• depends upon: OAK-7158 Users shouldn't be able to change the number of retained generations