[PIG-960] Using Hadoop's optimized LineRecordReader for reading Tuples in PigStorage

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.6.0
• Component/s: impl
• Patch Info: Patch Available
• Assignee: Ankit Modi
• Reporter: Ankit Modi
• Created: 15/Sep/09 01:07
• Updated: 24/Mar/10 22:15
• Resolved: 02/Oct/09 17:00

Description

PigStorage's reading of Tuples ( lines ) can be optimized using Hadoop's LineRecordReader.

This can help in following areas

• Improving performance reading of Tuples (lines) in PigStorage
• Any future improvements in line reading done in Hadoop's LineRecordReader is automatically carried over to Pig

Issues that are handled by this patch

• BZip uses internal buffers and positioning for determining the number of bytes read. Hence buffering done by LineRecordReader has to be turned off
• Current implementation of LocalSeekableInputStream does not implement available method. This method has to be implemented.