[OAK-3148] Online migration process for the binaries

• Type: New Feature
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.3.7, 1.4
• Component/s: blob, upgrade
• Assignee: Thomas Mueller
• Reporter: Tomek Rękawek
• Created: 28/Jul/15 08:13
• Updated: 08/Oct/19 15:21
• Resolved: 17/Sep/15 10:28

Description

For clients that want to migrate their blob stores, let's add a new feature that allows copy them in the background.

AC:

1. SplitBlobStore
   1. Administrator can configure Oak to use the SplitBlobStore that references the source (old) and the destination (new) blob store.
   2. Data stores can be used as well via the DataStoreBlobStore.
   3. On the read operation, if the requested blob exists on the new store, SplitBlobStore will return it.
   4. Otherwise, SplitBlobStore will try to read the blob from the old store.
   5. All write requests will be directed to the new blob store.
2. Copy process
   1. Administrator can start, stop and resume the copy process using JMX command.
   2. Administrator can see the progress in JMX and logs
   3. The process will read the SplitBlobStore configuration and copy the binaries from source to destination
   4. Once a binary is moved, its reference in the NodeStore is updated and commited.
   5. Only the head revision has to be updated.

The idea is that after all binaries are copied, the old revisions will be gradually removed by the compaction mechanisms and then binaries will be removed from the source store by the blob garbage collector. Future improvements are possible, eg. to invoke the compaction and GC manually.

Issue Links

• is related to: OAK-3402 Multiplexing NodeStore support in Oak layer
• supercedes: OAK-3122 Direct copy of the chunked data between blob stores