[PIG-63] PigStorage does not properly handle UTF8 data

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.1.0
• Assignee: Benjamin Reed
• Reporter: Olga Natkovich
• Created: 15/Jan/08 00:36
• Updated: 03/Jul/08 18:12
• Resolved: 24/Jan/08 22:03

Description

From Ben:

I just checked the code and the problem seems to be PigStorage. getNext() uses
readLine() which does not handle UTF8 correctly. putNext() also uses default encoder rather than UTF8 explicitly.

Internally and in BinStorage UTF8 appears to be handled correctly.