Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-6259

Performance issue with large number of fields and values when using copyFields

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 4.8.1
    • Fix Version/s: 4.10, 6.0
    • Component/s: None
    • Labels:
      None

      Description

      When you have schema with a large enough number of fields (in my case around 250 fields) and you use copyFields to populate a number of fields (very few in my case 3-4) you see a severe degradation in the performance of ingestion.

      Tracking this down using a profiler found that in the lucene Document.getField() was using 87% of all CPU time. As it turns out getField() does an iteration over the list of fields in the Document returning the field if the name matches.. Anyway in the case of copyFields with lots of values getField() gets called alot...

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              shalin Shalin Shekhar Mangar
              Reporter:
              sbower Steven Bower

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment