[PIG-3839] Umbrella jira for Pig on Tez Performance Improvements

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: tez
• Assignee: Unassigned
• Reporter: Rohini Palaniswamy
• Created: 27/Mar/14 19:38
• Updated: 24/May/16 22:31

Description

Separating out performance improvements from PIG-3446 which is the main jira for Pig on Tez.

Issue Links

• is related to: PIG-3446 Umbrella jira for Pig on Tez

Sub-Tasks

• Improve performance of replicate-join: Open
• Improve performance of union: Closed
• Use unsorted shuffle in Orderby, Skewed Join to improve performance in Tez: Open
• Use shared edge with no multiquery: Open
• Implement automatic reducer parallelism: Closed
• Sort avoidance for group by and join: Open
• Dynamically switch to replicate join: Open
• Integrate YSmart into Pig on tez: Open
• Optimize join followed by order by using same key: Open
• Simplify plan of Limit on Tez: Open
• UnionOptimizer in Tez should optimize the case of replicated join: Open
• Handle two outputs from split going to same input in MultiQueryOptimizer: Resolved
• Improve parallelism of union and join: Resolved
• Use unsorted shuffle in Union: Closed
• Improve performance of Limit following an Orderby on Tez: Open
• Limit reduce task should start as soon as one map task finishes: Closed
• Broadcast the index file in case of POMergeCoGroup and POMergeJoin: Open
• Rework Hash based aggregation for Tez: Open
• 1-1 edge vertices should use same jvm opts: Closed
• Size estimation should be done in sampler instead of sample aggregator: Open
• Enhance Tez AM size estimation: Resolved
• Better multi-query planning in case of multiple edges: Closed
• Eliminate identity vertex for order by and skewed join right after LOAD: Closed
• Optimize multi-query plan for diamond shape edges: Open
• Eliminate duplicate split calculation for Order by and Skewed Join: Open