[OAK-8574] Minimize network calls required when completing direct upload

• Type: Technical task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.16.0, 1.10.4
• Fix Version/s: 1.18.0
• Component/s: blob-cloud, blob-cloud-azure
• Assignee: Matt Ryan
• Reporter: Matt Ryan
• Created: 23/Aug/19 21:25
• Updated: 30/Sep/19 07:35
• Resolved: 27/Aug/19 07:31

Description

The completeHttpUpload() in the cloud data store backends can be improved in terms of quantity of cloud storage service API calls.

Suggestions include:

• Try a single getRecord() call at the beginning instead of calling exists().  If an exception is thrown, catch it - this means the record doesn't exist and can be written.  If a record is returned, we don't write - and instead return this record.
• Don't check for existence after writing the record; instead assume that the record is written correctly if no error or exception from SDK.  Verify this behavior in unit tests.
• After writing the record, construct the record directly instead of calling getRecord() to do it.

This removes two network API calls if the record is written and one if the record already exists.