Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9612

Stored field access should be avoided when it's not needed

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      This is a small enhancement. (unneeded stored access spend 5% in my profile result)
      All fields which is written in fl parameter(some of them are only doc values not stored) are iterated over from stored fields and it's inefficient. Further when fl parameters are only docValues, we should avoid accessing stored.

      I'm going to update a conservative patch.
      This patch exclude nonStoredDocValues fields from stored field list, and if we don't need access stored, we skip it.
      What 'conservative' means is that when schema is dynamically changed this patch not change behaviors.(ex. stored field 'a' is removed from schema, and user search fl=a, then a is returned from DocStreamer.)
      But I'm not sure how should solr behaves when schema is dynamically changed.

      I think better approach is
      Each fields are classified 3 types from schema and process each.

      • stored -> fetch from stored
      • nonStoredDocValues -> fetch from docValues
      • unknown -> error or lazy field(distinguishable?)
        But this might break backward compatibility.(like mentioned above)

      Any comments are welcome.

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              takaishi Takahiro Ishikawa

              Dates

              • Created:
                Updated:

                Issue deployment