[ATLAS-4340] Set Solr wait-searcher property to false by default to make Solr commits async

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1.0
• Fix Version/s: 3.0.0, 2.2.0
• Component/s: atlas-core
• Labels: perfomance solr
• Assignee: Sarath Subramanian
• Reporter: Sarath Subramanian
• Created: 22/Jun/21 16:21
• Updated: 27/Jul/21 22:00
• Resolved: 27/Jul/21 22:00

Description

 In Atlas when a transaction is committed, the entries are committed to HBase (primary storage) and Solr (indexing storage). A transaction is rolled-back if the primary storage commit fails, on the other hand when the secondary commit fails (solr), the transaction is not-rolled back and logged as warning and it is recommended to use reindex to repair the missing index documents. This behavior is due to the fact that the primary storage is the source of truth and indexes can be rebuild.

In Janusgraph, there is a property for Solr to make solr commits async. This is set to true in Atlas making every commit to wait until the solr commit is successful. This will have a negative impact on performance and is recommended to be false by default.

Property: index.[X].solr.wait-searcher

When mutating - wait for the index to reflect new mutations before returning. This can have a negative impact on performance.

 

This Jira is about setting the default value for above property to FALSE and can be overridden if need arises. 

The solution should use the StandardTransactionLogProcessor provided within JanusGraph to track failures to indexes (secondary storage in JanusGraph parlance) during commit. Using this would provide recovery mechanism in case of failures during transaction commit.

Issue Links

• links to: Code review
• links to: Precommit