[TEZ-388] Pig on Tez

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Rohini Palaniswamy
• Created: 23/Aug/13 22:41
• Updated: 03/Jul/14 03:41

Description

Umbrella jira to track Pig on Tez issues.

Issue Links

• relates to: PIG-3446 Umbrella jira for Pig on Tez
• requires: TEZ-391 SharedEdge - Support for passing same output from a vertex as input to two different vertices
• requires: TEZ-793 Staging directories accumulate and pollute the home directory
• requires: TEZ-606 Fix Tez to work with kerberos security
• requires: TEZ-711 Fix memory leak when not reading from inputs due to caching
• requires: TEZ-1107 Support increase of parallelism of vertex in case of custom partitioner
• requires: TEZ-789 Setting intermediate output compression does not work
• requires: TEZ-668 Allow a Processor to trigger an Input as to when to fetch/prep data.
• requires: TEZ-800 One-one edge with parallelism -1 fails if source vertex parallelism is not -1 as well
• requires: TEZ-984 Improve usability and debuggability
• requires: TEZ-1153 Allow object caching for more than one vertex
• requires: TEZ-661 Implement a non-sorted partitioned output
• requires: TEZ-678 Support for union operations via VertexGroup abstraction
• requires: TEZ-802 Determination of Task Placement for 1-1 Edges

Sub-Tasks

• SharedEdge - Support for passing same output from a vertex as input to two different vertices: Open
• Better scheduling for uneven DAGs: Patch Available
• MRInputSplitDistributor should set parallelism like MRInputAMSplitGenerator: Open
• Make some statistics available for the Input: Open