[NUTCH-1039] Fetcher fails for pages without content-length header

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Cannot Reproduce
• Affects Version/s: 1.4
• Fix Version/s: 1.7
• Assignee: Markus Jelsma
• Reporter: Markus Jelsma
• Created: 11/Jul/11 12:52
• Updated: 22/May/13 03:53
• Resolved: 03/May/13 20:36

Description

Fetcher fails:
2011-07-11 14:45:34,764 ERROR http.Http - org.apache.nutch.protocol.http.api.HttpException: bad content length:
2011-07-11 14:45:34,765 ERROR http.Http - at org.apache.nutch.protocol.http.HttpResponse.readPlainContent(HttpResponse.java:218)
2011-07-11 14:45:34,765 ERROR http.Http - at org.apache.nutch.protocol.http.HttpResponse.<init>(HttpResponse.java:158)
2011-07-11 14:45:34,765 ERROR http.Http - at org.apache.nutch.protocol.http.Http.getResponse(Http.java:64)
2011-07-11 14:45:34,765 ERROR http.Http - at org.apache.nutch.protocol.http.api.HttpBase.getProtocolOutput(HttpBase.java:138)
2011-07-11 14:45:34,765 ERROR http.Http - at org.apache.nutch.parse.ParserChecker.main(ParserChecker.java:79)

Both fetcher and indexing filter checker fail sometimes. I'm unsure whether this is something in Nutch or whether the remote server only returns content-length incidentally.

Issue Links

• is related to: NUTCH-1096 Empty (not null) ContentLength results in failure of fetch