Uploaded image for project: 'ManifoldCF'
  1. ManifoldCF
  2. CONNECTORS-1438

MIME type with ";" causes SolrServerException in Solr connector

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: ManifoldCF 2.7
    • Fix Version/s: ManifoldCF 2.8
    • Component/s: None
    • Labels:
      None

      Description

      When running job for Solr connection,if target web site include MIME type with ";" (e.g. "text/html; charset=UTF-8") SolrServerException occurs.

      Here is stack trace.

      Exception tossed: Unhandled SolrServerException: java.lang.IllegalArgumentException: MIME type may not contain reserved characters
      org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unhandled SolrServerException: java.lang.IllegalArgumentException: MIME type may not contain reserved characters
              at org.apache.manifoldcf.agents.output.solr.HttpPoster.handleSolrServerException(HttpPoster.java:385)
              at org.apache.manifoldcf.agents.output.solr.HttpPoster.indexPost(HttpPoster.java:636)
              at org.apache.manifoldcf.agents.output.solr.SolrConnector.addOrReplaceDocumentWithException(SolrConnector.java:587)
              at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
              at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$OutputAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3407)
              at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
              at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
              at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
              at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
              at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
              at org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.processDocument(WebcrawlerConnector.java:1431)
              at org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.processDocuments(WebcrawlerConnector.java:752)
              at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
      Caused by: org.apache.solr.client.solrj.SolrServerException: java.lang.IllegalArgumentException: MIME type may not contain reserved characters
              at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:473)
              at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
              at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1292)
              at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1062)
              at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1004)
              at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
              at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
              at org.apache.manifoldcf.agents.output.solr.HttpPoster$IngestThread.run(HttpPoster.java:923)
      Caused by: java.lang.IllegalArgumentException: MIME type may not contain reserved characters
              at org.apache.http.util.Args.check(Args.java:36)
              at org.apache.http.entity.ContentType.create(ContentType.java:206)
              at org.apache.http.entity.ContentType.create(ContentType.java:218)
              at org.apache.http.entity.mime.content.InputStreamBody.<init>(InputStreamBody.java:58)
              at org.apache.manifoldcf.agents.output.solr.ModifiedHttpSolrClient.createMethod(ModifiedHttpSolrClient.java:200)
              at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:260)
              at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
              at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
              ... 7 more
      

        Attachments

        1. CONNECTORS-1438.patch
          0.8 kB
          Karl Wright

          Activity

            People

            • Assignee:
              kwright@metacarta.com Karl Wright
              Reporter:
              Kenta KASAHARA Kenta Kasahara
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: