[PIG-1271] Provide a more flexible data format to load complex field (bag/tuple/map) in PigStorage

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.7.0
• Labels: gsoc2013
• Assignee: Daniel Dai
• Reporter: Daniel Dai
• Created: 02/Mar/10 19:57
• Updated: 21/Mar/13 01:13

Description

With PIG-613, we are able to load txt files containing complex data type (map/bag/tuple) according to schema. However, the format of complex data field is very strict. User have to use pre-determined special characters to mark the beginning and end of each field, and those special characters can not be used in the content. The goals of this issue are:

1. Provide a way for user to escape special characters
2. Make it easy for users to customize Utf8StorageConverter when they have their own data format

This is a candidate project for Google summer of code 2013. More information about the program can be found at https://cwiki.apache.org/confluence/display/PIG/GSoc2013