Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-6906

Lucene54DocValuesFormat bug when a large segment does not have values

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 5.4, 6.0
    • None
    • None
    • New

    Description

      Mike reported getting the following stack trace during a merge:

      java.lang.ArrayIndexOutOfBoundsException: -1
              at org.apache.lucene.util.packed.DirectMonotonicWriter.flush(DirectMonotonicWriter.java:64)
              at org.apache.lucene.util.packed.DirectMonotonicWriter.finish(DirectMonotonicWriter.java:126)
              at org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.writeSparseMissingBitset(Lucene54DocValuesConsumer.java:399)
              at org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.addNumericField(Lucene54DocValuesConsumer.java:207)
              at org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.addSortedField(Lucene54DocValuesConsumer.java:614)
              at org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.addSortedSetField(Lucene54DocValuesConsumer.java:653)
              at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addSortedSetField(PerFieldDocValuesFormat.java:131)
              at org.apache.lucene.codecs.DocValuesConsumer.mergeSortedSetField(DocValuesConsumer.java:736)
              at org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:219)
              at org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:150)
              at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:105)
              at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4075)
              at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
              at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
              at org.elasticsearch.index.engine.ElasticsearchConcurrentMergeScheduler.doMerge(ElasticsearchConcurrentMergeScheduler.java:95)
              at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
      

      This is due to the fact that the segment did not have any values and made DirectMonotonicWriter fail since it assumes at least one value (without explicitly testing it).

      Attachments

        1. LUCENE-6906.patch
          14 kB
          Adrien Grand

        Activity

          People

            jpountz Adrien Grand
            jpountz Adrien Grand
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: