[TEZ-1264] Support for limiting output records in OnFileSortedOutput

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Rohini Palaniswamy
• Reporter: Rohini Palaniswamy
• Created: 07/Jul/14 22:27
• Updated: 09/Dec/14 03:37

Description

When we are limiting on unsorted output, we can stop after reaching the count in the Processor. But if limiting has to be done on sorted output in map phase it is not possible as sorting is done by OnFileSortedOutput. If limiting was supported as part of the output, then we can limit records before writing to each part file after Partitioner is applied.

Issue Links

• is required by: PIG-4049 Improve performance of Limit following an Orderby on Tez