Major Description
Because SitemapProcessor uses the HTTP protocol plugin, which limits the size of a response to http.content.limit (64KB by default), it can only handle sitemaps smaller than that size.
I don't believe that is the intent of the users by setting http.content.limit - they want to limit document size, not sitemap size. The spec specifically says that sitemaps can be up to 50MB.
Attachments
Issue Links
- relates to
-
NUTCH-2666 Increase default value for http.content.limit / ftp.content.limit / file.content.limit
-
- Closed
-