Solr
  1. Solr
  2. SOLR-1914

phps and json outputs illegal numeric values for NaN and +/- Infinity

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4
    • Fix Version/s: 3.1, 4.0-ALPHA
    • Component/s: clients - php
    • Labels:
      None
    • Environment:

      apache Solr 1.4, Jetty (out of the box). e-comerce application UTF-8

      Description

      (first of all Solr Rocks!)
      iam runnig an e-comerce application with dynamic filteset, depend on selected categories.
      everything works fine and stable but today i found out that ther is a strange case:

      i guess around 5000 strings in my app used as facet filter. There is one String "transluzentes Acryl" and even the english translation "Translucent acrylics" are not working.

      QUERY:
      http://ff01:8983/solr/core1/select/?q=*:*&fq=shop_id:10 AND language:2 AND category_1:"Badewannen" AND material:"transluzentes Acryl"&facet=on&facet.limit=400&facet.field=category_1&facet.field=category_2&facet.field=manu&facet.field=length&facet.field=width&facet.field=height&facet.field=color&facet.field=material&facet.field=delivery&facet.field=attributes&facet.mincount=1&facet.sort=lex&facet.method=fc&stats=true&stats.field=price&start=0&rows=50&sort=sort+desc&fl=name,model_nr,image,url,manu,delivery,in_stock,serie,price,uvp,nav_id,template,option,name,model_nr,image,url,manu,delivery,in_stock,serie,price,uvp,nav_id,template,option&wt=phps&V=1.0&version=2.2

      RESPONSE: as xml and php works fine! but json , phps ends up like this.
      .....s:2:"fq";s:79:"shop_id:10 AND language:2 AND category_1:"Badewannen" AND material:"transluzentes Acryl"";......
      see that double qoutes at the end.

      That happens only on this strings or Docs because even the english Docs fails. I though its may be a matter of encoding, so i added a transformer in the DIH which replaced these strings with "Hello World".
      Faces on "Hello Wolrd" fails. Other Facet Strings with white spaces, special chars and so on never fail.

      if i add brackets on the fq
      ...q=:&fq=shop_id:10 AND language:2 AND category_1:"Badewannen" AND material:"transluzentes Acryl"&facet=on&...
      it ends up with a double }}
      s:2:"fq";s:79:"shop_id:20 AND language:2 AND category_1:"Badewannen" AND material:"Hallo Welt"";}}

      1. ff01.xml
        3 kB
        Sascha Jovanoski
      2. SOLR-1949.patch
        2 kB
        Yonik Seeley

        Issue Links

          Activity

          Hide
          Yonik Seeley added a comment -

          It's not clear if this is a Solr bug, or just some bug in the e-commerce app.

          The double quote at the end looks fine... one quote is literally part of the string, the other quote is from the phps encoding. The first string with length (s:79) doesn't look like it matches that actual string value, but I assume maybe you edited the example by hand.

          Show
          Yonik Seeley added a comment - It's not clear if this is a Solr bug, or just some bug in the e-commerce app. The double quote at the end looks fine... one quote is literally part of the string, the other quote is from the phps encoding. The first string with length (s:79) doesn't look like it matches that actual string value, but I assume maybe you edited the example by hand.
          Hide
          Sascha Jovanoski added a comment -

          thx so far,

          i did some tests. i will attach the full request/response at the end.

          i guess if i do a request and get an response including 3 hits its fine so far.

          • &wt=phps php(5) unserialize it's FALSE
          • &wt=json php(5) json_decode it's FALSE
          • &wt=xml php(5) simplexml_load_string it's TRUE

          i cant imagine an app bug, so i did a standalone test.
          the doccontents should be valid UTF-8, database is complete UTF-8, indexing is by DIH mysql direct.
          also tried some characters encodings before

          i also can choose wt=xml but the facet-names in the result of simplexml_load_string ar gone only a numeric key and the count will left from the facets.

          Show
          Sascha Jovanoski added a comment - thx so far, i did some tests. i will attach the full request/response at the end. i guess if i do a request and get an response including 3 hits its fine so far. &wt=phps php(5) unserialize it's FALSE &wt=json php(5) json_decode it's FALSE &wt=xml php(5) simplexml_load_string it's TRUE i cant imagine an app bug, so i did a standalone test. the doccontents should be valid UTF-8, database is complete UTF-8, indexing is by DIH mysql direct. also tried some characters encodings before i also can choose wt=xml but the facet-names in the result of simplexml_load_string ar gone only a numeric key and the count will left from the facets.
          Hide
          Yonik Seeley added a comment -

          One issue I see is an NaN in the JSON response... unfortunately the JSON standard has no way of representing Nan, +/-Inf. We should encode this as a string instead.
          Anyone know if PHPS has a numeric representation for these values?

          Sascha, is this the only incorrect part of the JSON / PHPS output or do you see any other issues?

          Show
          Yonik Seeley added a comment - One issue I see is an NaN in the JSON response... unfortunately the JSON standard has no way of representing Nan, +/-Inf. We should encode this as a string instead. Anyone know if PHPS has a numeric representation for these values? Sascha, is this the only incorrect part of the JSON / PHPS output or do you see any other issues?
          Hide
          Sascha Jovanoski added a comment -

          hi Yonik,

          yes you are right! if i replace or escape this NaN, erverthing works fine and Solr still rocks.
          i was so focused on that strings and maybe wrong app encodings that it took me until today to see that unescaped NaN.

          i played around with that stats but i cant get this error simulate on other queries, i mean my problem case include three items with the same price.
          I did other request which response with items with the same price but i got a stddev of 0.0. that looks strange from outside the code.

          so i dont see any other issues in my app, thanks a lot !!!!!

          Show
          Sascha Jovanoski added a comment - hi Yonik, yes you are right! if i replace or escape this NaN, erverthing works fine and Solr still rocks. i was so focused on that strings and maybe wrong app encodings that it took me until today to see that unescaped NaN. i played around with that stats but i cant get this error simulate on other queries, i mean my problem case include three items with the same price. I did other request which response with items with the same price but i got a stddev of 0.0. that looks strange from outside the code. so i dont see any other issues in my app, thanks a lot !!!!!
          Hide
          Yonik Seeley added a comment -

          Looks like this was sort of fixed for Python & Ruby back in SOLR-449.
          I haven't had much luck finding a way to express them in PHP... I guess the default for all JSON subclasses should just write as a String.

          Show
          Yonik Seeley added a comment - Looks like this was sort of fixed for Python & Ruby back in SOLR-449 . I haven't had much luck finding a way to express them in PHP... I guess the default for all JSON subclasses should just write as a String.
          Hide
          Yonik Seeley added a comment -

          Here's a patch w/ test that writes the special values as a String, in the format the Java uses.
          "NaN","-Infinity","Infinity". I plan on committing shortly to 4.0 and 3.1

          Show
          Yonik Seeley added a comment - Here's a patch w/ test that writes the special values as a String, in the format the Java uses. "NaN","-Infinity","Infinity". I plan on committing shortly to 4.0 and 3.1
          Hide
          Hoss Man added a comment -

          Updating summary to clarify root of problem (and fix already committed)

          Show
          Hoss Man added a comment - Updating summary to clarify root of problem (and fix already committed)
          Hide
          Hoss Man added a comment -

          Sascha: as Yonik already noted, he fixed this in teh 3x (and trunk) branches – the fix does not exist in Solr 1.4.1, or on the (now abandoned) 1.5 branch.

          if you have other questions - please send them to the solr-user mailing list

          Show
          Hoss Man added a comment - Sascha: as Yonik already noted, he fixed this in teh 3x (and trunk) branches – the fix does not exist in Solr 1.4.1, or on the (now abandoned) 1.5 branch. if you have other questions - please send them to the solr-user mailing list
          Hide
          Grant Ingersoll added a comment -

          Bulk close for 3.1.0 release

          Show
          Grant Ingersoll added a comment - Bulk close for 3.1.0 release

            People

            • Assignee:
              Yonik Seeley
              Reporter:
              Sascha Jovanoski
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development