[CONNECTORS-1620] Accept Sitemaps with content type application/xml

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: ManifoldCF 2.14
• Component/s: Web connector
• Assignee: Markus Schuch
• Reporter: Markus Schuch
• Created: 22/Aug/19 14:57
• Updated: 22/Jan/22 06:34
• Resolved: 22/Aug/19 15:33

Description

Given an Output Connection, that does not accepts the MIME type application/xml for ingestion, it is currently not possible to crawl a sitemap.xml, when the webserver returns application/xml as content type for the sitemap.

The sitemap is discarded before the links are extracted, because the mime type application/xml is not listed in the interestingMimeTypeArray.

Issue Links

• relates to: CONNECTORS-255 Consider adding SiteMap support to RSS connector