[OAK-6604] Oak Blob Cloud is not used by oak-upgrade

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.7.6
• Fix Version/s: 1.7.7, 1.6.6, 1.8.0
• Component/s: upgrade
• Assignee: Tomek Rękawek
• Reporter: Arek Kita
• Created: 31/Aug/17 08:40
• Updated: 12/Oct/17 09:27
• Resolved: 01/Sep/17 07:31

Description

Watching at oak-upgrade pom.xml file I had a clue that Oak Blob Cloud is used for migration.

However, I noticed that there is a wrong import for S3DataStoreFactory file:

import org.apache.jackrabbit.oak.blob.cloud.aws.s3.S3DataStore;

and instead we should use:

import org.apache.jackrabbit.oak.blob.cloud.s3.S3DataStore;

...to be able to get the newest Oak Blob Cloud features for S3 and oak-upgrade.

Additionally the class org.apache.jackrabbit.core.data.CachingDataStore; should not be used anymore and it is not needed as the org.apache.jackrabbit.oak.blob.cloud.s3.S3DataStore itself provides caching capabilities.

This should help to get rid of the following problems when migrating S3 backed DataStores:

31.08.2017 08:16:13.886 *ERROR* [org.apache.jackrabbit.core.data.AbstractDataStore] AbstractDataStore.java:105  Failed to hash identifier using MAC (Message Authentication Code) algorithm.
java.lang.NullPointerException: null
        at org.apache.jackrabbit.core.data.CachingDataStore.getOrCreateReferenceKey(CachingDataStore.java:685)
        at org.apache.jackrabbit.core.data.AbstractDataStore.getReferenceKey(AbstractDataStore.java:141)
        at org.apache.jackrabbit.core.data.AbstractDataStore.getReferenceFromIdentifier(AbstractDataStore.java:100)
        at org.apache.jackrabbit.core.data.AbstractDataRecord.getReference(AbstractDataRecord.java:60)
        at org.apache.jackrabbit.oak.plugins.blob.datastore.DataStoreBlobStore.getReference(DataStoreBlobStore.java:306)
        at org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.getReference(SegmentBlob.java:132)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeBlob(DefaultSegmentWriter.java:573)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeProperty(DefaultSegmentWriter.java:697)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeProperty(DefaultSegmentWriter.java:683)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeNodeUncached(DefaultSegmentWriter.java:901)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeNode(DefaultSegmentWriter.java:805)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeNodeUncached(DefaultSegmentWriter.java:873)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeNode(DefaultSegmentWriter.java:805)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeNodeUncached(DefaultSegmentWriter.java:868)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeNode(DefaultSegmentWriter.java:805)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeNodeUncached(DefaultSegmentWriter.java:868)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeNode(DefaultSegmentWriter.java:805)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeNodeUncached(DefaultSegmentWriter.java:868)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeNode(DefaultSegmentWriter.java:805)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeNodeUncached(DefaultSegmentWriter.java:868)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.writeNode(DefaultSegmentWriter.java:805)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$SegmentWriteOperation.access$800(DefaultSegmentWriter.java:258)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter$8.execute(DefaultSegmentWriter.java:247)
        at org.apache.jackrabbit.oak.segment.SegmentBufferWriterPool.execute(SegmentBufferWriterPool.java:101)
        at org.apache.jackrabbit.oak.segment.DefaultSegmentWriter.writeNode(DefaultSegmentWriter.java:243)
        at org.apache.jackrabbit.oak.segment.SegmentWriter.writeNode(SegmentWriter.java:141)
        at org.apache.jackrabbit.oak.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:132)
        at org.apache.jackrabbit.oak.segment.scheduler.Commit.hasChanges(Commit.java:102)
        at org.apache.jackrabbit.oak.segment.scheduler.LockBasedScheduler.execute(LockBasedScheduler.java:249)
        at org.apache.jackrabbit.oak.segment.scheduler.LockBasedScheduler.schedule(LockBasedScheduler.java:227)
        at org.apache.jackrabbit.oak.segment.SegmentNodeStore.merge(SegmentNodeStore.java:195)
        at org.apache.jackrabbit.oak.spi.state.ProxyNodeStore.merge(ProxyNodeStore.java:43)
        at org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.migrateWithCheckpoints(RepositorySidegrade.java:374)
        at org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.copyState(RepositorySidegrade.java:337)
        at org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.copy(RepositorySidegrade.java:301)
        at org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.sidegrade(OakUpgrade.java:92)
        at org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.migrate(OakUpgrade.java:78)
...

/cc amjain

Issue Links

• is required by: OAK-6611 [upgrade][oak-blob-cloud] Many S3DataStore errors during migration with oak-upgrade