Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-8578

Successful or not, requests are not always fully consumed by Solrj clients and we count on HttpClient or the JVM.

    Details

    • Type: Bug
    • Status: Reopened
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 6.0
    • Component/s: None
    • Labels:
      None

      Description

      Does not seem to happen with XML response parser.

      Not the largest deal because HttpClient appears to consume unread bytes from the stream for us, but something seems off.

      1. SOLR-8578.patch
        5 kB
        Mark Miller
      2. SOLR-8578.patch
        7 kB
        Mark Miller

        Issue Links

          Activity

          Hide
          markrmiller@gmail.com Mark Miller added a comment -

          Okay, I see now, we simply don't process the response, we just use the error code on success since it's {responseHeader={status=0,QTime=0}}.

          Following the logic in https://issues.apache.org/jira/browse/SOLR-8451?focusedCommentId=15110095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15110095, we should just nicely eat this up.

          Show
          markrmiller@gmail.com Mark Miller added a comment - Okay, I see now, we simply don't process the response, we just use the error code on success since it's {responseHeader={status=0,QTime=0}}. Following the logic in https://issues.apache.org/jira/browse/SOLR-8451?focusedCommentId=15110095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15110095 , we should just nicely eat this up.
          Hide
          markrmiller@gmail.com Mark Miller added a comment -

          Hmmm...and it looks like I thought it only affect Binary and not XML because XML does not specify a content type size and I was using the InputStream#available call. It affects all types.

          Show
          markrmiller@gmail.com Mark Miller added a comment - Hmmm...and it looks like I thought it only affect Binary and not XML because XML does not specify a content type size and I was using the InputStream#available call. It affects all types.
          Hide
          markrmiller@gmail.com Mark Miller added a comment -

          Seems only Binary will actually return a content length header rather than using chunked encoding.

          This seems to be because of some Writer nonsense. I think that we put a Writer around the outputstream. We can't and don't want to close it, that will close the outputstream. We can't do nothing or everything stays in the buffer. So we flush. I think that causes Jetty to use chunked encoding. I think what we would like to do is Writer#flushBuffer, but it's package protected. Binary just doesn't have this Writer issue.

          Show
          markrmiller@gmail.com Mark Miller added a comment - Seems only Binary will actually return a content length header rather than using chunked encoding. This seems to be because of some Writer nonsense. I think that we put a Writer around the outputstream. We can't and don't want to close it, that will close the outputstream. We can't do nothing or everything stays in the buffer. So we flush. I think that causes Jetty to use chunked encoding. I think what we would like to do is Writer#flushBuffer, but it's package protected. Binary just doesn't have this Writer issue.
          Hide
          markrmiller@gmail.com Mark Miller added a comment -

          We can fix this by passing a special outputstream to the Writer that will not flush.

          Show
          markrmiller@gmail.com Mark Miller added a comment - We can fix this by passing a special outputstream to the Writer that will not flush.
          Hide
          markrmiller@gmail.com Mark Miller added a comment -

          Created SOLR-8669 to fix the early flush in it's own issue.

          Show
          markrmiller@gmail.com Mark Miller added a comment - Created SOLR-8669 to fix the early flush in it's own issue.
          Hide
          markrmiller@gmail.com Mark Miller added a comment -

          Patch attached.

          Show
          markrmiller@gmail.com Mark Miller added a comment - Patch attached.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit a8bc427aac85d600e1abee28bb373f428c08c7ae in lucene-solr's branch refs/heads/master from markrmiller
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a8bc427 ]

          SOLR-8578: Successful or not, requests are not always fully consumed by Solrj clients and we count on HttpClient or the JVM.

          Show
          jira-bot ASF subversion and git services added a comment - Commit a8bc427aac85d600e1abee28bb373f428c08c7ae in lucene-solr's branch refs/heads/master from markrmiller [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a8bc427 ] SOLR-8578 : Successful or not, requests are not always fully consumed by Solrj clients and we count on HttpClient or the JVM.
          Hide
          markrmiller@gmail.com Mark Miller added a comment -

          Whoops, I need to move the CHANGES entry.

          Show
          markrmiller@gmail.com Mark Miller added a comment - Whoops, I need to move the CHANGES entry.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit a40118c9df0db1deaee9ea0d7e2ad399704ff5b3 in lucene-solr's branch refs/heads/master from markrmiller
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a40118c ]

          SOLR-8578: Fully consume proxy requests and move CHANGES entry to 6.0.

          Show
          jira-bot ASF subversion and git services added a comment - Commit a40118c9df0db1deaee9ea0d7e2ad399704ff5b3 in lucene-solr's branch refs/heads/master from markrmiller [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a40118c ] SOLR-8578 : Fully consume proxy requests and move CHANGES entry to 6.0.
          Hide
          anshumg Anshum Gupta added a comment -

          Reopening to back-port for 5.5.

          Show
          anshumg Anshum Gupta added a comment - Reopening to back-port for 5.5.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit dab4c81b03e51d9c8a11d60c41d916da50052dde in lucene-solr's branch refs/heads/branch_5x from markrmiller
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=dab4c81 ]

          SOLR-8578: Successful or not, requests are not always fully consumed by Solrj clients and we count on HttpClient or the JVM.

          Show
          jira-bot ASF subversion and git services added a comment - Commit dab4c81b03e51d9c8a11d60c41d916da50052dde in lucene-solr's branch refs/heads/branch_5x from markrmiller [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=dab4c81 ] SOLR-8578 : Successful or not, requests are not always fully consumed by Solrj clients and we count on HttpClient or the JVM.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 50ae1fff7c0a18576c59ec672eb3b6b6ad921781 in lucene-solr's branch refs/heads/branch_5_5 from markrmiller
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=50ae1ff ]

          SOLR-8578: Successful or not, requests are not always fully consumed by Solrj clients and we count on HttpClient or the JVM.

          Show
          jira-bot ASF subversion and git services added a comment - Commit 50ae1fff7c0a18576c59ec672eb3b6b6ad921781 in lucene-solr's branch refs/heads/branch_5_5 from markrmiller [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=50ae1ff ] SOLR-8578 : Successful or not, requests are not always fully consumed by Solrj clients and we count on HttpClient or the JVM.

            People

            • Assignee:
              markrmiller@gmail.com Mark Miller
              Reporter:
              markrmiller@gmail.com Mark Miller
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Development