[LUCENE-7460] Should SortedNumericDocValues expose a per-document random-access API?

• Type: Wish
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Adrien Grand
• Created: 23/Sep/16 07:46
• Updated: 28/Aug/22 15:03
• Resolved: 30/Jan/17 10:25

Description

Sorted numerics used to expose a per-document random-access API so that accessing the median or max element would be cheap. The new SortedNumericDocValues still exposes the number of values a document has, but the only way to read values is to use

{nextValue}

, which forces to read all values in order to read the max value.

For instance, SortedNumericSelector.MAX does the following in master (the important part is the for-loop):

    private void setValue() throws IOException {
      int count = in.docValueCount();
      for(int i=0;i<count;i++) {
        value = in.nextValue();
      }
    }

    @Override
    public int nextDoc() throws IOException {
      int docID = in.nextDoc();
      if (docID != NO_MORE_DOCS) {
        setValue();
      }
      return docID;
    }

while it used to simply look up the value at index count-1 in 6.x:

    @Override
    public long get(int docID) {
      in.setDocument(docID);
      final int count = in.count();
      if (count == 0) {
        return 0; // missing
      } else {
        return in.valueAt(count-1);
      }
    }

This could be a conscious decision since a sequential API gives more opportunities to the codec to compress efficiently, but on the other hand this API prevents sorting by max or median values to be efficient.

On my end I have a preference for the random-access API.

Issue Links

• is related to: LUCENE-7407 Explore switching doc values to an iterator API