[CONNECTORS-1438] MIME type with ";" causes SolrServerException in Solr connector

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: ManifoldCF 2.7
• Fix Version/s: ManifoldCF 2.8
• Assignee: Karl Wright
• Reporter: Kenta Kasahara
• Created: 29/Jun/17 00:47
• Updated: 29/Jun/17 01:43
• Resolved: 29/Jun/17 01:43

Description

When running job for Solr connection,if target web site include MIME type with ";" (e.g. "text/html; charset=UTF-8") SolrServerException occurs.

Here is stack trace.

Exception tossed: Unhandled SolrServerException: java.lang.IllegalArgumentException: MIME type may not contain reserved characters
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unhandled SolrServerException: java.lang.IllegalArgumentException: MIME type may not contain reserved characters
        at org.apache.manifoldcf.agents.output.solr.HttpPoster.handleSolrServerException(HttpPoster.java:385)
        at org.apache.manifoldcf.agents.output.solr.HttpPoster.indexPost(HttpPoster.java:636)
        at org.apache.manifoldcf.agents.output.solr.SolrConnector.addOrReplaceDocumentWithException(SolrConnector.java:587)
        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$OutputAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3407)
        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
        at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
        at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
        at org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.processDocument(WebcrawlerConnector.java:1431)
        at org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.processDocuments(WebcrawlerConnector.java:752)
        at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
Caused by: org.apache.solr.client.solrj.SolrServerException: java.lang.IllegalArgumentException: MIME type may not contain reserved characters
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:473)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1292)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1062)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1004)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
        at org.apache.manifoldcf.agents.output.solr.HttpPoster$IngestThread.run(HttpPoster.java:923)
Caused by: java.lang.IllegalArgumentException: MIME type may not contain reserved characters
        at org.apache.http.util.Args.check(Args.java:36)
        at org.apache.http.entity.ContentType.create(ContentType.java:206)
        at org.apache.http.entity.ContentType.create(ContentType.java:218)
        at org.apache.http.entity.mime.content.InputStreamBody.<init>(InputStreamBody.java:58)
        at org.apache.manifoldcf.agents.output.solr.ModifiedHttpSolrClient.createMethod(ModifiedHttpSolrClient.java:200)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:260)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
        ... 7 more