Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-6906

Lucene54DocValuesFormat bug when a large segment does not have values

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.4, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Mike reported getting the following stack trace during a merge:

      java.lang.ArrayIndexOutOfBoundsException: -1
              at org.apache.lucene.util.packed.DirectMonotonicWriter.flush(DirectMonotonicWriter.java:64)
              at org.apache.lucene.util.packed.DirectMonotonicWriter.finish(DirectMonotonicWriter.java:126)
              at org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.writeSparseMissingBitset(Lucene54DocValuesConsumer.java:399)
              at org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.addNumericField(Lucene54DocValuesConsumer.java:207)
              at org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.addSortedField(Lucene54DocValuesConsumer.java:614)
              at org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.addSortedSetField(Lucene54DocValuesConsumer.java:653)
              at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addSortedSetField(PerFieldDocValuesFormat.java:131)
              at org.apache.lucene.codecs.DocValuesConsumer.mergeSortedSetField(DocValuesConsumer.java:736)
              at org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:219)
              at org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:150)
              at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:105)
              at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4075)
              at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
              at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
              at org.elasticsearch.index.engine.ElasticsearchConcurrentMergeScheduler.doMerge(ElasticsearchConcurrentMergeScheduler.java:95)
              at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
      

      This is due to the fact that the segment did not have any values and made DirectMonotonicWriter fail since it assumes at least one value (without explicitly testing it).

        Attachments

          Activity

            People

            • Assignee:
              jpountz Adrien Grand
              Reporter:
              jpountz Adrien Grand
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: