[OAK-4903] Async uploads in S3 causes issues in a cluster

• Type: Improvement
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.5.15, 1.6.0
• Component/s: blob
• Assignee: Amit Jain
• Reporter: Amit Jain
• Created: 07/Oct/16 06:15
• Updated: 08/Oct/19 15:22
• Resolved: 29/Nov/16 08:41

Description

S3DataStore and CachingFDS through the CachingDataStore enable async uploads. This causes problems in clustered setups where uploads can sometimes be visible after a delay. During this time any request for the corresponding asset/file would return errors.

Issue Links

• is duplicated by: OAK-4440 Async upload in S3DataStore can lead to intermittent binary missing exception in cluster
• is related to: OAK-5146 S3 getOrCreateReferenceKey() should provide a default implementation

Sub-Tasks

• Introduce API to pass options while writing blobs in BlobStore/DataStore: Closed
• Pass option for synchronized uploads when writing lucene blobs: Closed