[NUTCH-564] External parser supports encoding attribute

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Fix Version/s: nutchgora
• Component/s: indexer

Environment

All

• Patch Info: Patch Available
• Assignee: Chris A. Mattmann
• Reporter: Antony Bowesman
• Created: 03/Oct/07 21:42
• Updated: 01/Apr/11 15:07
• Resolved: 08/Aug/10 19:04

Description

When an external component generates text, which is returned to the external parser, it always converts the text using the default character set. (os.toString()). For example, the returned text may be utf-8, but will not be converted to a String correctly.

I added the attribute <encoding> to the <implementation> XML in plugin.xml and this is then used to convert the text.

I have tested my original fix on my local 0.9 and include a patch, but have also made an untested patch for trunk.