Solr
  1. Solr
  2. SOLR-449

python (and presumably ruby) writer can generate NaN

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 1.3
    • Fix Version/s: 1.3
    • Component/s: None
    • Labels:
      None

      Description

      The JSON response writer can omit "NaN" as float literal; this is fine for JSON but breaks eval() in python (not sure if this is a problem in ruby).

      1. nan.patch
        4 kB
        Mike Klaas

        Activity

        Hide
        Mike Klaas added a comment -

        Fix that serializes NaN/Inf as

        float('NaN')
        float('Inf')

        which will produce the proper values if the underlying c library is sufficiently smart. To do it more portably requires importing modules, so isn't particularly feasible.

        I'll probably commit this soon. ruby might also suffer from the same problem, though I don't know how to represent NaN properly there.

        Show
        Mike Klaas added a comment - Fix that serializes NaN/Inf as float('NaN') float('Inf') which will produce the proper values if the underlying c library is sufficiently smart. To do it more portably requires importing modules, so isn't particularly feasible. I'll probably commit this soon. ruby might also suffer from the same problem, though I don't know how to represent NaN properly there.
        Hide
        Hoss Man added a comment -

        when the description says "omit" i think you mean "emit" right?

        I don't know squat about Ruby, but the docs seem to suggest that...

        NaN

        ...is the proper way to express "not a number" as a float literal...

        http://ruby-doc.org/core/classes/Float.html#M000244

        Show
        Hoss Man added a comment - when the description says "omit" i think you mean "emit" right? I don't know squat about Ruby, but the docs seem to suggest that... NaN ...is the proper way to express "not a number" as a float literal... http://ruby-doc.org/core/classes/Float.html#M000244
        Hide
        Mike Klaas added a comment -

        Yes, "emit"

        Using an online ruby interpreter, "NaN" and "Infinity" aren't literals. But I can produce them using (0.0/0.0) and (1.0/1e-3000), so I'll throw that into the patch too.

        Show
        Mike Klaas added a comment - Yes, "emit" Using an online ruby interpreter, "NaN" and "Infinity" aren't literals. But I can produce them using (0.0/0.0) and (1.0/1e-3000), so I'll throw that into the patch too.
        Hide
        Yonik Seeley added a comment -

        Rather than rely on an overflow for infinity, one can divide by zero, right?

        irb(main):022:0> 1.0/0.0
        => Infinity
        irb(main):023:0>
        irb(main):024:0* -1.0/0.0
        => -Infinity

        Show
        Yonik Seeley added a comment - Rather than rely on an overflow for infinity, one can divide by zero, right? irb(main):022:0> 1.0/0.0 => Infinity irb(main):023:0> irb(main):024:0* -1.0/0.0 => -Infinity
        Hide
        Mike Klaas added a comment -

        so you can! I tried the integer version (1/0), which complains bitterly.

        Show
        Mike Klaas added a comment - so you can! I tried the integer version (1/0), which complains bitterly.
        Hide
        Hoss Man added a comment -

        Yonik's comment lead me to discover that "irb" is the interactive ruby shell (who knew!) which lead me to spend 20 minutes banging my head against my desk (and google) attempting to find literals or constants that express NaN or Infinity.

        I can not fathom the existence a language written in this millennium that understands the concept of NaN but has no literal way for you to express NaN without doing a computation. I would say "i've lost all respect for Ruby", but since i didn't really know enough about ruby to respest it before, let me just say "i've lost the ability to gain respect for Ruby in the future".

        I can't believe i'm saying this, but: would it be worth while for the RubyResponseWriter to output code that declares variables for NaN, Infinite and -Infinite and only does those 3 computations once, instead of each time they occur? (i'm assuming that if one document has a NaN value for a field value, then other docs will probably have a NaN value).

        Show
        Hoss Man added a comment - Yonik's comment lead me to discover that "irb" is the interactive ruby shell (who knew!) which lead me to spend 20 minutes banging my head against my desk (and google) attempting to find literals or constants that express NaN or Infinity. I can not fathom the existence a language written in this millennium that understands the concept of NaN but has no literal way for you to express NaN without doing a computation. I would say "i've lost all respect for Ruby", but since i didn't really know enough about ruby to respest it before, let me just say "i've lost the ability to gain respect for Ruby in the future". I can't believe i'm saying this, but: would it be worth while for the RubyResponseWriter to output code that declares variables for NaN, Infinite and -Infinite and only does those 3 computations once, instead of each time they occur? (i'm assuming that if one document has a NaN value for a field value, then other docs will probably have a NaN value).
        Hide
        Mike Klaas added a comment -

        I don't think that declaring a viable approach. At least in python, it would create a series of statements rather than a single expression, which would make parsing more difficult (using exec instead of eval(), etc). Ruby might behave similarly.

        I'm not sure that repeating the computation is a justifiable worry though: even floating point division is blazingly fast compared to anything ruby will try to do. Also, I'm not sure if it is possible to store a NaN or infinity in a document field, is it? I'm too lazy to check if Float.parseFloat("NaN") does the right thing in java.

        It came up for me because of a bug in a custom queryscorer that gamely attempted a division by zero. I've checked in a the fix that we discussed for python and ruby--I'll leave the issue open in case our resident rubyista has a better solution.

        Show
        Mike Klaas added a comment - I don't think that declaring a viable approach. At least in python, it would create a series of statements rather than a single expression, which would make parsing more difficult (using exec instead of eval(), etc). Ruby might behave similarly. I'm not sure that repeating the computation is a justifiable worry though: even floating point division is blazingly fast compared to anything ruby will try to do. Also, I'm not sure if it is possible to store a NaN or infinity in a document field, is it? I'm too lazy to check if Float.parseFloat("NaN") does the right thing in java. It came up for me because of a bug in a custom queryscorer that gamely attempted a division by zero. I've checked in a the fix that we discussed for python and ruby--I'll leave the issue open in case our resident rubyista has a better solution.
        Hide
        Erik Hatcher added a comment -

        I've looked around for constants for those too, and asked around to several Ruby experts, and sure enough no such constants exist for Infinity or NaN. But come on, Hoss, don't throw in the towel on Ruby just for that - there are plenty of other great features of Ruby to more than make up for this weird omission of those constants. There are methods on Float that allow you to ask if something is infinite or not a number.

        Hoss's point about outputting code is not that far-fetched and is actually along the lines of what I proposed in SOLR-358 - having a special Ruby response writer mode that output something interpretable by solr-ruby that took care of all data type / ordering issues. For now the divide-by-zero hack is fine I suppose.

        Show
        Erik Hatcher added a comment - I've looked around for constants for those too, and asked around to several Ruby experts, and sure enough no such constants exist for Infinity or NaN. But come on, Hoss, don't throw in the towel on Ruby just for that - there are plenty of other great features of Ruby to more than make up for this weird omission of those constants. There are methods on Float that allow you to ask if something is infinite or not a number. Hoss's point about outputting code is not that far-fetched and is actually along the lines of what I proposed in SOLR-358 - having a special Ruby response writer mode that output something interpretable by solr-ruby that took care of all data type / ordering issues. For now the divide-by-zero hack is fine I suppose.

          People

          • Assignee:
            Mike Klaas
            Reporter:
            Mike Klaas
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development