[OAK-5238] IndexCopier causes concurrent update on NodeBuilder

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.15, 1.2.3, 1.4.0
• Fix Version/s: 1.5.16, 1.6.0, 1.4.18, 1.0.40, 1.2.28
• Component/s: lucene
• Epic Link: indexer resilience 1.6
• Assignee: Marcel Reutegger
• Reporter: Marcel Reutegger
• Created: 07/Dec/16 17:24
• Updated: 23/Nov/17 08:16
• Resolved: 14/Dec/16 14:52

Description

OAK-2247 introduced the copy-on-write feature for lucene index in Oak. This feature may result in a NodeBuilder updated by multiple threads concurrently. New index files are first stored on the local filesystem and then copied asynchronously into the repository. At the same time the async index update thread manipulates the node builders as well.

With MongoMK this results in unexpected conflicts and failed async index updates.

Issue Links

• depends upon: OAK-5300 OakDirectory.fileLength() must throw FileNotFoundException if file does not exist
• depends upon: OAK-5175 Pass option for synchronized uploads when writing lucene blobs
• depends upon: OAK-5075 Refactor IndexCopier to make it more modular
• is related to: OAK-6976 CopyOnWriteDirectory_Segment_Test failure
• relates to: OAK-2247 CopyOnWriteDirectory implementation for Lucene for use in indexing

Sub-Tasks

• Introduce BlobFactory in OakDirectory: Closed