Solr
  1. Solr
  2. SOLR-1489

A UTF-8 character is output twice (Bug in Jetty)

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None
    • Environment:

      Jetty-6.1.3
      Jetty-6.1.21
      Jetty-7.0.0RC6

      Description

      A UTF-8 character is output twice under particular conditions.
      Attach the sample data.(error_utf8-example.xml)
      Registered only sample data, click the following URL.

      http://localhost:8983/solr/select?q=*%3A*&version=2.2&start=0&rows=10&omitHeader=true&fl=attr_json&wt=json

      Sample data is only "B", but response is "BB".
      When wt=phps, error occurs in PHP unsrialize() function.

      This bug is like a bug in Jetty.

      jettybugsample.war is the simplest one to reproduce the problem.
      Copy example/webapps, and start Jetty server, and click the following URL.

      http://localhost:8983/jettybugsample/filter/hoge

      Like earlier, B is output twice. Sysout only B once.
      I have tested this on Jetty 6.1.3 and 6.1.21, 7.0.0rc6.
      (When testing with 6.1.21or 7.0.0rc6, change "bufsize" from 128 to 512 in web.xml. )

      1. error_utf8-example.xml
        2 kB
        Jun Ohtani
      2. jetty-6.1.22.jar
        524 kB
        Jun Ohtani
      3. jettybugsample.war
        4 kB
        Jun Ohtani
      4. jetty-util-6.1.22.jar
        173 kB
        Jun Ohtani
      5. jsp-2.1.zip
        1.11 MB
        Jun Ohtani
      6. servlet-api-2.5-20081211.jar
        131 kB
        Jun Ohtani
      7. SOLR-1489.patch
        4 kB
        Koji Sekiguchi

        Issue Links

          Activity

          Hide
          Grant Ingersoll added a comment -

          Bulk close for 3.1.0 release

          Show
          Grant Ingersoll added a comment - Bulk close for 3.1.0 release
          Hide
          Koji Sekiguchi added a comment -

          Marking resolved as duplicate of SOLR-2381.

          Show
          Koji Sekiguchi added a comment - Marking resolved as duplicate of SOLR-2381 .
          Hide
          Jun Ohtani added a comment -

          Sekiguchi-san, I checked bufsize [510-512]. only saw B once. Maybe, it is OK.

          Show
          Jun Ohtani added a comment - Sekiguchi-san, I checked bufsize [510-512] . only saw B once. Maybe, it is OK.
          Hide
          Robert Muir added a comment -

          Just as a quick check, I used the data attached and clicked the URL and only saw B once.

          Show
          Robert Muir added a comment - Just as a quick check, I used the data attached and clicked the URL and only saw B once.
          Hide
          Koji Sekiguchi added a comment -

          Ohtani-san, now Solr 3x/4.0 have patched version of Jetty about handling UTF-8 characters. Can you try one of them with your data and see the result?

          Show
          Koji Sekiguchi added a comment - Ohtani-san, now Solr 3x/4.0 have patched version of Jetty about handling UTF-8 characters. Can you try one of them with your data and see the result?
          Hide
          Koji Sekiguchi added a comment -

          Attached patch fixes the above failure, but I got another failure (no expires header):

          Testcase: testCacheVetoHandler took 3.29 sec
          Testcase: testCacheVetoException took 1.395 sec
                  FAILED
          We got no Expires header
          junit.framework.AssertionFailedError: We got no Expires header
                  at org.apache.solr.servlet.CacheHeaderTest.checkVetoHeaders(CacheHeaderTest.java:73)
                  at org.apache.solr.servlet.CacheHeaderTest.testCacheVetoException(CacheHeaderTest.java:59)
          
          Testcase: testLastModified took 1.485 sec
          Testcase: testEtag took 1.577 sec
          Testcase: testCacheControl took 1.035 sec
          
          Show
          Koji Sekiguchi added a comment - Attached patch fixes the above failure, but I got another failure (no expires header): Testcase: testCacheVetoHandler took 3.29 sec Testcase: testCacheVetoException took 1.395 sec FAILED We got no Expires header junit.framework.AssertionFailedError: We got no Expires header at org.apache.solr.servlet.CacheHeaderTest.checkVetoHeaders(CacheHeaderTest.java:73) at org.apache.solr.servlet.CacheHeaderTest.testCacheVetoException(CacheHeaderTest.java:59) Testcase: testLastModified took 1.485 sec Testcase: testEtag took 1.577 sec Testcase: testCacheControl took 1.035 sec
          Hide
          Koji Sekiguchi added a comment -

          Thanks, Ohtani-san.

          Using these new jetty jars (6.1.22), I run ant test, but I got a failure:

          TEST-org.apache.solr.servlet.CacheHeaderTest.txt
          Testcase: testCacheVetoHandler took 2.469 sec
          Testcase: testCacheVetoException took 1.25 sec
          	FAILED
          null expected:<[no-cache, ]no-store> but was:<[must-revalidate,no-cache,]no-store>
          junit.framework.ComparisonFailure: null expected:<[no-cache, ]no-store> but was:<[must-revalidate,no-cache,]no-store>
          	at org.apache.solr.servlet.CacheHeaderTest.checkVetoHeaders(CacheHeaderTest.java:65)
          	at org.apache.solr.servlet.CacheHeaderTest.testCacheVetoException(CacheHeaderTest.java:59)
          
          Testcase: testLastModified took 1.188 sec
          Testcase: testEtag took 1.11 sec
          Testcase: testCacheControl took 1.391 sec
          

          According to SOLR-632, the cache header related test was failed when we used jetty-6.1.11, Lars filed https://jira.codehaus.org/browse/JETTY-646. Now the issue has been fixed, I thought jetty-6.1.22 should work. I've not looked into the details of cache header test, though.

          Show
          Koji Sekiguchi added a comment - Thanks, Ohtani-san. Using these new jetty jars (6.1.22), I run ant test, but I got a failure: TEST-org.apache.solr.servlet.CacheHeaderTest.txt Testcase: testCacheVetoHandler took 2.469 sec Testcase: testCacheVetoException took 1.25 sec FAILED null expected:<[no-cache, ]no-store> but was:<[must-revalidate,no-cache,]no-store> junit.framework.ComparisonFailure: null expected:<[no-cache, ]no-store> but was:<[must-revalidate,no-cache,]no-store> at org.apache.solr.servlet.CacheHeaderTest.checkVetoHeaders(CacheHeaderTest.java:65) at org.apache.solr.servlet.CacheHeaderTest.testCacheVetoException(CacheHeaderTest.java:59) Testcase: testLastModified took 1.188 sec Testcase: testEtag took 1.11 sec Testcase: testCacheControl took 1.391 sec According to SOLR-632 , the cache header related test was failed when we used jetty-6.1.11, Lars filed https://jira.codehaus.org/browse/JETTY-646 . Now the issue has been fixed, I thought jetty-6.1.22 should work. I've not looked into the details of cache header test, though.
          Hide
          Jun Ohtani added a comment -

          jsp-2.1 dir

          Show
          Jun Ohtani added a comment - jsp-2.1 dir
          Hide
          Jun Ohtani added a comment -

          Jetty-6.1.22's jar

          Show
          Jun Ohtani added a comment - Jetty-6.1.22's jar
          Hide
          Jun Ohtani added a comment -

          Sekiguchi-san, I test new jetty.
          This bug was not reproduced with jettybugsample.war.

          It doesn't test in error_utf8-example.xml. It is because the size of the buffer in new jetty has been changed.

          Show
          Jun Ohtani added a comment - Sekiguchi-san, I test new jetty. This bug was not reproduced with jettybugsample.war. It doesn't test in error_utf8-example.xml. It is because the size of the buffer in new jetty has been changed.
          Hide
          Koji Sekiguchi added a comment -

          Ok, http://jira.codehaus.org/browse/JETTY-1122 has been marked as fixed and jetty 6.1.22 released. Ohtani-san, can you test the new jetty with your test case to see the bug is gone? Thanks.

          Show
          Koji Sekiguchi added a comment - Ok, http://jira.codehaus.org/browse/JETTY-1122 has been marked as fixed and jetty 6.1.22 released. Ohtani-san, can you test the new jetty with your test case to see the bug is gone? Thanks.
          Hide
          Koji Sekiguchi added a comment -

          Good catch, Otani-san! I can reproduce the problem with the data and the filter you attached when running it on Jetty. And thank you for opening the JIRA ticket in Jetty.
          Now we are closing to releasing 1.4, I don't want this to be a blocker because this is not a Solr bug as you said. You can run Solr on arbitrary servlet containers other than Jetty if you'd like.
          I'd like to keep this opening, and watching http://jira.codehaus.org/browse/JETTY-1122 . Thanks.

          Show
          Koji Sekiguchi added a comment - Good catch, Otani-san! I can reproduce the problem with the data and the filter you attached when running it on Jetty. And thank you for opening the JIRA ticket in Jetty. Now we are closing to releasing 1.4, I don't want this to be a blocker because this is not a Solr bug as you said. You can run Solr on arbitrary servlet containers other than Jetty if you'd like. I'd like to keep this opening, and watching http://jira.codehaus.org/browse/JETTY-1122 . Thanks.
          Hide
          Jun Ohtani added a comment -
          Show
          Jun Ohtani added a comment - Report Jetty's JIRA. http://jira.codehaus.org/browse/JETTY-1122
          Hide
          Jun Ohtani added a comment -

          Attached sample data and Jetty bug sample program.

          Show
          Jun Ohtani added a comment - Attached sample data and Jetty bug sample program.

            People

            • Assignee:
              Koji Sekiguchi
              Reporter:
              Jun Ohtani
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development