Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-3582

NumericUtils.floatToSortableInt/doubleToSortableLong does not sort certain NaN ranges correctly and NumericRangeQuery produces wrong results for NaNs with half-open ranges

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.5, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      The current implementation of floatToSortableInt does not account for different NaN ranges which may result in NaNs sorted before -Infinity and after +Infinity. The default Java ordering is: all NaNs after Infinity.

      A possible fix is to make all NaNs canonic "quiet NaN" as in:

      // Canonicalize NaN ranges. I assume this check will be faster here than 
      // (v == v) == false on the FPU? We don't distinguish between different
      // flavors of NaNs here (see http://en.wikipedia.org/wiki/NaN). I guess
      // in Java this doesn't matter much anyway.
      if ((v & 0x7fffffff) > 0x7f800000) {
        // Apply the logic below to a canonical "quiet NaN"
        return 0x7fc00000 ^ 0x80000000;
      }
      

      I don't commit because I don't know how much of the existing stuff relies on this (nobody should be keeping different NaNs in their indexes, but who knows...).

        Attachments

        1. LUCENE-3582.patch
          21 kB
          Uwe Schindler
        2. LUCENE-3582.patch
          16 kB
          Uwe Schindler
        3. LUCENE-3582.patch
          2 kB
          Dawid Weiss

          Activity

            People

            • Assignee:
              thetaphi Uwe Schindler
              Reporter:
              dweiss Dawid Weiss
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: