[OAK-4060] Allow use of pre extrcated text cache for incremental indexing

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.2.12, 1.4, 1.4.0
• Component/s: lucene
• Labels: docs-impacting
• Assignee: Chetan Mehrotra
• Reporter: Chetan Mehrotra
• Created: 25/Feb/16 09:26
• Updated: 07/Mar/16 05:35
• Resolved: 25/Feb/16 10:48

Description

Pre Extraction support was implemented with an assumption that such big indexing would happen as part of reindex so it was used in reindex phase only. Reason to avoid using it in incremental indexing (non reindex case) were

1. Incremental index would does not have text for newly added files. So checking with pre extracted cache would not be useful
2. PreExtraction logic keeps in memory state (blobs_empty.txt,blobs_error.txt) which would then unnecessary hog memory.

However in some cases people make use of new incremental migration feature in upgrade. Which would lead to one big incremental indexing step once next migration is done and that would then not able to make use of pre extraction support.

So as a fix we should provide a policy option to ignore the reindex clause per admin setting