[SOLR-6259] Performance issue with large number of fields and values when using copyFields

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 4.8.1
• Fix Version/s: 4.10, 6.0
• Assignee: Shalin Shekhar Mangar
• Reporter: Steven Bower
• Created: 19/Jul/14 03:44
• Updated: 09/May/16 18:52
• Resolved: 19/Jul/14 07:13

Description

When you have schema with a large enough number of fields (in my case around 250 fields) and you use copyFields to populate a number of fields (very few in my case 3-4) you see a severe degradation in the performance of ingestion.

Tracking this down using a profiler found that in the lucene Document.getField() was using 87% of all CPU time. As it turns out getField() does an iteration over the list of fields in the Document returning the field if the name matches.. Anyway in the case of copyFields with lots of values getField() gets called alot...

Issue Links

• breaks: SOLR-7335 Multivalue field that is boosted on indexing time has wrong norm.