[SOLR-14923] Indexing performance is unacceptable when child documents are involved

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 8.3, 8.4, 8.5, 8.6, 8.7, 9.0
• Fix Version/s: 8.8
• Component/s: update, UpdateRequestProcessors
• Labels: performance pull-request-available
• Assignee: David Smiley
• Reporter: Thomas Wöckinger
• Created: 12/Oct/20 13:16
• Updated: 22/Aug/22 16:57
• Resolved: 08/Jan/21 05:13

Description

Parallel indexing does not make sense at moment when child documents are used.

The org.apache.solr.update.processor.DistributedUpdateProcessor checks at the end of the method doVersionAdd if Ulog caches should be refreshed.

This check will return true if any child document is included in the AddUpdateCommand.

If so ulog.openRealtimeSearcher(); is called, this call is very expensive, and executed in a synchronized block of the UpdateLog instance, therefore all other operations on the UpdateLog are blocked too.

Because every important UpdateLog method (add, delete, ...) is done using a synchronized block almost each operation is blocked.

This reduces multi threaded index update to a single thread behavior.

The described behavior is not depending on any option of the UpdateRequest, so it does not make any difference if 'waitFlush', 'waitSearcher' or 'softCommit'  is true or false.

The described behavior makes the usage of ChildDocuments useless, because the performance is unacceptable.

 

 

Issue Links

• causes: SOLR-15468 Regression: Adding _root_ schema field to existing index causes atomic update failures
• is caused by: SOLR-12638 Support atomic updates of nested/child documents for nested-enabled schema
• is related to: SOLR-15192 Atomic update error when RTG component is involved
• relates to: SOLR-15064 Atomic/partial updates to nested docs should not assume _route_ param is the root ID
• requires: SOLR-15069 child doc transformer should make parentFilter obsolete
• links to: GitHub Pull Request #2142
• links to: GitHub Pull Request #2159