[LUCENE-8558] Adding NumericDocValuesFields is slowing down the indexing process significantly

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 7.4, 7.5
• Fix Version/s: 7.6, 8.0
• Component/s: core/index
• Labels: patch performance
• Lucene Fields: New
• Assignee: Simon Willnauer
• Reporter: Kranthi
• Created: 03/Nov/18 00:14
• Updated: 28/Aug/22 15:37
• Resolved: 07/Nov/18 19:36

Description

The indexing time for my ~2M documents has gone up significantly when I started adding fields of type NumericDocValuesField.

 

Upon debugging found the bottleneck to be in the PerFieldMergeState#FilterFieldInfos constructor. The contains check in the below code snippet was the culprit. 

this.filteredNames = new HashSet<>(filterFields);
this.filtered = new ArrayList<>(filterFields.size());
for (FieldInfo fi : src) {
  if (filterFields.contains(fi.name)) {

A simple change as below seems to have fixed my issue

this.filteredNames = new HashSet<>(filterFields);
this.filtered = new ArrayList<>(filterFields.size());
for (FieldInfo fi : src) {
  if (this.filteredNames.contains(fi.name)) {