Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-12798

Structural changes in SolrJ since version 7.0.0 have effectively disabled multipart post

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 7.4
    • Fix Version/s: None
    • Component/s: SolrJ
    • Labels:
      None

      Description

      Project ManifoldCF uses SolrJ to post documents to Solr. When upgrading from SolrJ 7.0.x to SolrJ 7.4, we encountered significant structural changes to SolrJ's HttpSolrClient class that seemingly disable any use of multipart post. This is critical because ManifoldCF's documents often contain metadata in excess of 4K that therefore cannot be stuffed into a URL.

      The changes in question seem to have been performed by Paul Noble on 10/31/2017, with the introduction of the RequestWriter mechanism. Basically, if a request has a RequestWriter, it is used exclusively to write the request, and that overrides the stream mechanism completely. I haven't chased it back to a specific ticket.

      ManifoldCF's usage of SolrJ involves the creation of ContentStreamUpdateRequests for all posts meant for Solr Cell, and the creation of UpdateRequests for posts not meant for Solr Cell (as well as for delete and commit requests). For our release cycle that is taking place right now, we're shipping a modified version of HttpSolrClient that ignores the RequestWriter when dealing with ContentStreamUpdateRequests. We apparently cannot use multipart for all requests because on the Solr side we get "pfountz Should not get here!" errors on the Solr side when we do, which generate HTTP error code 500 responses. That should not happen either, in my opinion.

        Attachments

        1. solr-update-request.txt
          36 kB
          Julien Massiera
        2. HOT Balloon Trip_Ultra HD.jpg
          1.18 MB
          Julien Massiera
        3. SOLR-12798-approach.patch
          6 kB
          Karl Wright
        4. no params in url.png
          80 kB
          Mikhail Khludnev
        5. SOLR-12798-reproducer.patch
          2 kB
          Mikhail Khludnev
        6. SOLR-12798-workaround.patch
          3 kB
          Mikhail Khludnev
        7. SOLR-12798.patch
          6 kB
          Mikhail Khludnev
        8. SOLR-12798.patch
          8 kB
          Noble Paul
        9. SOLR-12798.patch
          13 kB
          Noble Paul

          Issue Links

            Activity

              People

              • Assignee:
                kwright@metacarta.com Karl Wright
                Reporter:
                kwright@metacarta.com Karl Wright
              • Votes:
                1 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated: