[SOLR-9592] decorateDocValues cause serious performance issue because of using slowCompositeReaderWrapper

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 6.0, 6.1, 6.2
• Fix Version/s: 6.3, 7.0
• Component/s: Response Writers, search
• Labels: performance
• Assignee: Yonik Seeley
• Reporter: Takahiro Ishikawa
• Created: 04/Oct/16 04:37
• Updated: 08/Jun/19 15:31
• Resolved: 05/Oct/16 19:28

Description

I have serious performance issue using AtomicUpdate (and RealtimeGet) with non stored docValues.
Because decorateDocValues try to merge each leafLeader on the fly via slowCompositeReaderWrapper and it’s extremely slow (> 10sec).
Simply access docValues via nonCompositeReader could resolve this issue.(patch)

AtomicUpdate performance(or RealtimeGet performance)

• Environment
  • solr version : 6.0.0
  • schema ~ 100 fields(90% docValues, some of those are multi valued)
  • index : 5,000,000
• Performance
  • original : > 10sec per query
  • patched : at least 100msec per query

This patch will also enhance search performance, because DocStreamer also fetch docValues via decorateDocValues.
Though it depends on each environment, I could take 20% search performance gain.

(This patch originally written for solr 6.0.0, and now rewritten for master)