[NUTCH-625] Non-ascii character broken in dumped content for mixed encoding (utf-8 and multi-byte)

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 1.0.0
• Assignee: Unassigned
• Reporter: Vinci
• Created: 30/Mar/08 12:05
• Updated: 22/May/13 03:54
• Resolved: 12/Jan/13 20:05

Description

If the crawl db contains both utf-8 non-ascii character and non-utf-8 non-ascii character(i.e. multi-byte character), the dumped contents by readseg utility will have garbled character appear in all of the non-utf8 non-ascii text, and those texts are unable to repair by encoding reload.

At the same time, the utf-8 text is normal, only the non-utf8 text broken.

Any possible solution available for repairing the broken text?