[PIG-3557] Implement optimizations for LIMIT

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: tez-branch
• Fix Version/s: tez-branch
• Component/s: tez
• Assignee: Alex Bain
• Reporter: Alex Bain
• Created: 01/Nov/13 18:43
• Updated: 21/Nov/14 05:59
• Resolved: 16/Jan/14 01:31

Description

Implement optimizations for LIMIT when other parts of Pig-on-Tez are more mature. Some of the optimizations mentioned by Daniel include:

1. If the previous stage using 1 reduce, no need to add one more vertex
2. If the limitplan is null (ie, not the "limited order by" case), we might not need a shuffle edge, a pass through edge should be enough if possible
3. Similar to PIG-1270, we can push limit to InputHandler
4. We also need to think through the "limited order by" case once "order by" is implemented