Solr
  1. Solr
  2. SOLR-5590

SolrJ is still on httpcomponents/httpclient version 4.2.x, which has some problems

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.5.1
    • Fix Version/s: 4.7, 6.0
    • Component/s: None
    • Labels:
      None

      Description

      SolrJ depends on HttpClient 4.2.x right now, but HttpClient 4.2.x has issues that the ManifoldCF team encountered with handling of form data encoding - issues which are addressed in HttpClient 4.3.x. We developed a local patch, but Solr will eventually need to go to the new client. (ManifoldCF would plan to follow shortly thereafter).

      I tried to get Oleg (PMC chair of HttpComponents) to agree to port the fixed code to the 4.2.x stream but he did not want to do that. So I believe that that avenue is closed.

      See CONNECTORS-623 for a detailed description of the problem.

      1. SOLR-5590.patch
        0.8 kB
        Karl Wright

        Issue Links

          Activity

          Hide
          Karl Wright added a comment -

          Updating the httpclient version in the ivy versions file does not break compilation, at least on the 4x branch. Running all tests now.

          Show
          Karl Wright added a comment - Updating the httpclient version in the ivy versions file does not break compilation, at least on the 4x branch. Running all tests now.
          Hide
          Karl Wright added a comment -

          Tests pass too. Since they include a fair number of solr cloud tests, I believe this means it is probably safe to commit the update to the latest httpclient. Will attach a patch.

          Show
          Karl Wright added a comment - Tests pass too. Since they include a fair number of solr cloud tests, I believe this means it is probably safe to commit the update to the latest httpclient. Will attach a patch.
          Hide
          Karl Wright added a comment -

          Patch that updates to the latest httpclient.

          Note that this does NOT yet include any changes needed to specify UTF-8 encoding for multipart form elements; looking into how that is done now.

          Show
          Karl Wright added a comment - Patch that updates to the latest httpclient. Note that this does NOT yet include any changes needed to specify UTF-8 encoding for multipart form elements; looking into how that is done now.
          Hide
          Shawn Heisey added a comment -

          I checked the committer list for the project, and Karl Wright didn't show up there. I'm willing to work with Karl and get this change committed. I need a few hours before I can work on it, so if someone else has the time right now, feel free to take over.

          Another potential reason for someone else to take over, unless Karl already knows what to do: This change definitely means that SolrJ is using deprecated classes. I'd like to eliminate the deprecated classes, but that might require a completely new issue. I already tried once to use the new Builder classes in my own SolrJ code, but found that SolrJ didn't like it for some reason.

          Show
          Shawn Heisey added a comment - I checked the committer list for the project, and Karl Wright didn't show up there. I'm willing to work with Karl and get this change committed. I need a few hours before I can work on it, so if someone else has the time right now, feel free to take over. Another potential reason for someone else to take over, unless Karl already knows what to do: This change definitely means that SolrJ is using deprecated classes. I'd like to eliminate the deprecated classes, but that might require a completely new issue. I already tried once to use the new Builder classes in my own SolrJ code, but found that SolrJ didn't like it for some reason.
          Hide
          Karl Wright added a comment -

          Hi Shawn,

          Correct, I am not a committer for Solr, but I am involved in a downstream project (ManifoldCF) that has a Solr integration via SolrJ, so I have an interest in getting this stuff working.

          My suggestion, for now, is to create a separate ticket to work on the use of deprecated httpclient classes, but do this as a later step. For now, though, it would be great to just list them. I'll likely have a similar situation to deal with in ManifoldCF.

          Show
          Karl Wright added a comment - Hi Shawn, Correct, I am not a committer for Solr, but I am involved in a downstream project (ManifoldCF) that has a Solr integration via SolrJ, so I have an interest in getting this stuff working. My suggestion, for now, is to create a separate ticket to work on the use of deprecated httpclient classes, but do this as a later step. For now, though, it would be great to just list them. I'll likely have a similar situation to deal with in ManifoldCF.
          Hide
          Mark Miller added a comment -

          Thanks Karl! We should do this for 4.7.

          Show
          Mark Miller added a comment - Thanks Karl! We should do this for 4.7.
          Hide
          Shawn Heisey added a comment -

          Mark Miller, I take your comment to mean that this should not be backported to 4.6.1. I'm OK with that - httpclient is a pretty important component. The change seems pretty harmless, but users are a creative bunch, if there's even a remote chance for something to go wrong, someone will find it!

          Show
          Shawn Heisey added a comment - Mark Miller , I take your comment to mean that this should not be backported to 4.6.1. I'm OK with that - httpclient is a pretty important component. The change seems pretty harmless, but users are a creative bunch, if there's even a remote chance for something to go wrong, someone will find it!
          Hide
          Shawn Heisey added a comment -

          Tests and precommit passed on trunk. Committing there. Will merge to 4x and verify.

          Show
          Shawn Heisey added a comment - Tests and precommit passed on trunk. Committing there. Will merge to 4x and verify.
          Hide
          ASF subversion and git services added a comment -

          Commit 1555004 from Shawn Heisey in branch 'dev/trunk'
          [ https://svn.apache.org/r1555004 ]

          SOLR-5590: Upgrade HttpClient/HttpComponents to 4.3.x.

          Show
          ASF subversion and git services added a comment - Commit 1555004 from Shawn Heisey in branch 'dev/trunk' [ https://svn.apache.org/r1555004 ] SOLR-5590 : Upgrade HttpClient/HttpComponents to 4.3.x.
          Hide
          Shawn Heisey added a comment -

          It only took a couple of test runs on trunk to get the tests to pass, but I had to run it a bunch of times on branch_4x. It was a semi-repeatable failure that you can see with the following commandline (tested on Linux):

          ant test Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=testDistribSearch

          The same failure happens on a clean 4x checkout with similar frequency, so I don't think this change has caused any new problems. The precommit passed on the first try. I'll go ahead and commit.

          Show
          Shawn Heisey added a comment - It only took a couple of test runs on trunk to get the tests to pass, but I had to run it a bunch of times on branch_4x. It was a semi-repeatable failure that you can see with the following commandline (tested on Linux): ant test Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=testDistribSearch The same failure happens on a clean 4x checkout with similar frequency, so I don't think this change has caused any new problems. The precommit passed on the first try. I'll go ahead and commit.
          Hide
          ASF subversion and git services added a comment -

          Commit 1555022 from Shawn Heisey in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1555022 ]

          SOLR-5590: Upgrade HttpClient/HttpComponents to 4.3.x. (merge trunk r1555004)

          Show
          ASF subversion and git services added a comment - Commit 1555022 from Shawn Heisey in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1555022 ] SOLR-5590 : Upgrade HttpClient/HttpComponents to 4.3.x. (merge trunk r1555004)
          Hide
          Shawn Heisey added a comment -

          Thank you for the patch, Karl!

          Show
          Shawn Heisey added a comment - Thank you for the patch, Karl!
          Hide
          Karl Wright added a comment -

          You're welcome.

          Show
          Karl Wright added a comment - You're welcome.
          Hide
          Mark Miller added a comment -

          It was a semi-repeatable failure

          Shawn Heisey, can you open a JIRA issue for that if there is not one? Or point to a failed jenkins run for that? Anything someone sees too often locally, I'm happy to help harden.

          Show
          Mark Miller added a comment - It was a semi-repeatable failure Shawn Heisey , can you open a JIRA issue for that if there is not one? Or point to a failed jenkins run for that? Anything someone sees too often locally, I'm happy to help harden.

            People

            • Assignee:
              Shawn Heisey
              Reporter:
              Karl Wright
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development