[LUCENE-3390] Incorrect sort by Numeric values for documents missing the sorting field

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.3
• Fix Version/s: 3.4, 3.5
• Component/s: core/search
• Labels: double float int long numeric sort
• Lucene Fields: Patch Available
• Assignee: Uwe Schindler
• Reporter: Gilad Barkai
• Created: 22/Aug/11 11:32
• Updated: 28/Aug/22 12:55
• Resolved: 21/Sep/11 16:26

Description

While sorting results over a numeric field, documents which do not contain a value for the sorting field seem to get 0 (ZERO) value in the sort. (Tested against Double, Float, Int & Long numeric fields ascending and descending order).
This behavior is unexpected, as zero is "comparable" to the rest of the values. A better solution would either be allowing the user to define such a "non-value" default, or always bring those document results as the last ones.

Example scenario:
Adding 3 documents, 1st with value 3.5d, 2nd with -10d, and 3rd without any value.
Searching with MatchAllDocsQuery, with sort over that field in descending order yields the docid results of 0, 2, 1.

Asking for the top 2 documents brings the document without any value as the 2nd result - which seems as a bug?

Issue Links

• is depended upon by: LUCENE-3443 Port 3.x FieldCache.getDocsWithField() to trunk
• is related to: LUCENE-2671 Add sort missing first/last ability to SortField and ValueComparator
• relates to: LUCENE-831 Complete overhaul of FieldCache API/Implementation
• relates to: LUCENE-2665 Rework FieldCache to be more flexible/general
• relates to: LUCENE-2649 FieldCache should include a BitSet for matching docs
• relates to: SOLR-2134 Trie* fields should support sortMissingLast=true, and deprecate Sortable* Field Types