[SOLR-9612] Stored field access should be avoided when it's not needed

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 6.0, 6.1, 6.2
• Fix Version/s: 6.7, 7.0
• Component/s: Response Writers, search
• Labels: performance
• Assignee: Unassigned
• Reporter: Takahiro Ishikawa
• Created: 08/Oct/16 05:56
• Updated: 08/Jun/19 15:31

Description

This is a small enhancement. (unneeded stored access spend 5% in my profile result)
All fields which is written in fl parameter(some of them are only doc values not stored) are iterated over from stored fields and it's inefficient. Further when fl parameters are only docValues, we should avoid accessing stored.

I'm going to update a conservative patch.
This patch exclude nonStoredDocValues fields from stored field list, and if we don't need access stored, we skip it.
What 'conservative' means is that when schema is dynamically changed this patch not change behaviors.(ex. stored field 'a' is removed from schema, and user search fl=a, then a is returned from DocStreamer.)
But I'm not sure how should solr behaves when schema is dynamically changed.

I think better approach is
Each fields are classified 3 types from schema and process each.

• stored -> fetch from stored
• nonStoredDocValues -> fetch from docValues
• unknown -> error or lazy field(distinguishable?)
  But this might break backward compatibility.(like mentioned above)

Any comments are welcome.

Issue Links

• is related to: SOLR-8220 Read field from docValues for non stored fields