[OAK-10636] Tar recovery unsuccessful in oak-segment-tar during initialisation of FileStore

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.60.0
• Fix Version/s: 1.62.0
• Component/s: segment-tar
• Assignee: Miroslav Smiljanic
• Reporter: Miroslav Smiljanic
• Created: 01/Feb/24 09:04
• Updated: 17/May/24 11:22
• Resolved: 26/Feb/24 13:50

Description

The initialization of FileStore is unsuccessful if tar archive is not closed properly and is missing index, graph and binary reference. 

java.lang.NullPointerException: null
	at org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:517)
	at org.apache.jackrabbit.oak.segment.SegmentId.getSegment(SegmentId.java:153)
	at org.apache.jackrabbit.oak.segment.RecordId.getSegment(RecordId.java:98)
	at org.apache.jackrabbit.oak.segment.SegmentBlob.readLongBlobId(SegmentBlob.java:236)
	at org.apache.jackrabbit.oak.segment.SegmentBlob.readBlobId(SegmentBlob.java:174)
	at org.apache.jackrabbit.oak.segment.file.AbstractFileStore.lambda$populateTarBinaryReferences$0(AbstractFileStore.java:253)
	at org.apache.jackrabbit.oak.segment.Segment.forEachRecord(Segment.java:601)
	at org.apache.jackrabbit.oak.segment.file.AbstractFileStore.populateTarBinaryReferences(AbstractFileStore.java:251)
	at org.apache.jackrabbit.oak.segment.file.AbstractFileStore.writeSegment(AbstractFileStore.java:237)
	at org.apache.jackrabbit.oak.segment.file.AbstractFileStore$1.recoverEntry(AbstractFileStore.java:122)
	at org.apache.jackrabbit.oak.segment.file.tar.TarReader.generateTarFile(TarReader.java:183)
	at org.apache.jackrabbit.oak.segment.file.tar.TarReader.open(TarReader.java:106)
	at org.apache.jackrabbit.oak.segment.file.tar.TarFiles.<init>(TarFiles.java:396)
	at org.apache.jackrabbit.oak.segment.file.tar.TarFiles$Builder.build(TarFiles.java:214)
	at org.apache.jackrabbit.oak.segment.file.FileStore.<init>(FileStore.java:172)
	at org.apache.jackrabbit.oak.segment.file.FileStoreBuilder.build(FileStoreBuilder.java:445)
	at org.apache.jackrabbit.oak.segment.file.FileStoreTest.testRecovery_FileStore_withExternalBlobStore(FileStoreTest.java:201)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)

I was able to reproduce the issue when, in the segment, there are records holding long ID blob references.
Test case demonstrating the issue:
OAK-10636_test.patch

Issue Links

• is related to: OAK-10814 Tar file recovery still fails with SNFE in some cases.
• links to: GitHub PR