[NUTCH-374] when http.content.limit be set to -1 and Response.CONTENT_ENCODING is gzip or x-gzip , it can not fetch any thing.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8, 0.8.1
• Fix Version/s: 0.9.0
• Assignee: Piotr Kosiorowski
• Reporter: King Kong
• Created: 27/Sep/06 17:32
• Updated: 18/Apr/07 15:44
• Resolved: 30/Sep/06 19:38

Description

I set "http.content.limit" to -1 to not truncate content being fetched.
However , if response used gzip or x-gzip , then it was not able to uncompress.

I found the problem is in HttpBase.processGzipEncoded (plugin lib-http)
...
byte[] content = GZIPUtils.unzipBestEffort(compressed, getMaxContent());
...
because it is not deal with -1 to no limit , so must modify code to solve it;

byte[] content;
if (getMaxContent()>=0)

{ content = GZIPUtils.unzipBestEffort(compressed, getMaxContent()); }

else

{ content = GZIPUtils.unzipBestEffort(compressed); }