Lucene - Core
  1. Lucene - Core
  2. LUCENE-3582

NumericUtils.floatToSortableInt/doubleToSortableLong does not sort certain NaN ranges correctly and NumericRangeQuery produces wrong results for NaNs with half-open ranges

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.5, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      The current implementation of floatToSortableInt does not account for different NaN ranges which may result in NaNs sorted before -Infinity and after +Infinity. The default Java ordering is: all NaNs after Infinity.

      A possible fix is to make all NaNs canonic "quiet NaN" as in:

      // Canonicalize NaN ranges. I assume this check will be faster here than 
      // (v == v) == false on the FPU? We don't distinguish between different
      // flavors of NaNs here (see http://en.wikipedia.org/wiki/NaN). I guess
      // in Java this doesn't matter much anyway.
      if ((v & 0x7fffffff) > 0x7f800000) {
        // Apply the logic below to a canonical "quiet NaN"
        return 0x7fc00000 ^ 0x80000000;
      }
      

      I don't commit because I don't know how much of the existing stuff relies on this (nobody should be keeping different NaNs in their indexes, but who knows...).

      1. LUCENE-3582.patch
        21 kB
        Uwe Schindler
      2. LUCENE-3582.patch
        16 kB
        Uwe Schindler
      3. LUCENE-3582.patch
        2 kB
        Dawid Weiss

        Activity

        Dawid Weiss created issue -
        Dawid Weiss made changes -
        Field Original Value New Value
        Attachment LUCENE-3582.patch [ 12504266 ]
        Uwe Schindler made changes -
        Assignee Uwe Schindler [ thetaphi ]
        Uwe Schindler made changes -
        Attachment LUCENE-3582.patch [ 12504308 ]
        Uwe Schindler made changes -
        Fix Version/s 3.5 [ 12317877 ]
        Uwe Schindler made changes -
        Summary NumericUtils.floatToSortableInt does not sort certain NaN ranges correctly. NumericUtils.floatToSortableInt/doubleToSortableLong does not sort certain NaN ranges correctly and NumericRangeQuery produces wrong results for NaNs with half-open ranges
        Uwe Schindler made changes -
        Attachment LUCENE-3582.patch [ 12504352 ]
        Uwe Schindler made changes -
        Attachment LUCENE-3582.patch [ 12504353 ]
        Uwe Schindler made changes -
        Attachment LUCENE-3582.patch [ 12504352 ]
        Uwe Schindler made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Uwe Schindler made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Uwe Schindler
            Reporter:
            Dawid Weiss
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development