[OAK-7091] Avoid streaming data twice in composite data store

• Type: Task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: blob, blob-cloud, blob-cloud-azure, blob-plugins
• Assignee: Unassigned
• Reporter: Matt Ryan
• Created: 19/Dec/17 19:02
• Updated: 05/Jan/18 16:45

Description

When adding a new record to an Oak instance that is using composite data store, the blob stream will be read twice before it is stored - once by the composite data store (to determine the blob ID) and again by the delegate. This necessary because if there are multiple writable delegates and one delegate already has a matching blob, the composite should call addRecord() on the delegate that has the matching blob, which may not be the highest priority delegate. So we need to know the blob ID in order to select the correct writable delegate.

We could add a method to the CompositeDataStoreAware interface wherein the data store can be told which blob ID to use (from the composite) so that it doesn't have to process the stream again. Then the composite data store, after having read the stream to a temporary file, can pass an input stream from the temporary file to the delegate along with the computed blob ID, to avoid reading the stream twice.

Issue Links

• is a child of: OAK-5960 Multi blob store support