[PIG-4120] Broadcast the index file in case of POMergeCoGroup and POMergeJoin

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.18.0
• Component/s: tez
• Hadoop Flags: Reviewed
• Assignee: Satish Saley
• Reporter: Rohini Palaniswamy
• Created: 13/Aug/14 06:31
• Updated: 03/Oct/17 17:32
• Resolved: 03/Oct/17 17:32

Description

Currently merge join and merge cogroup use two DAGs - the first DAG creates the index file in hdfs and second DAG does the merge join. Similar to replicate join, we can broadcast the index file and cache it and use it in merge join and merge cogroup. This will give better performance and also eliminate need for the second DAG.

Issue Links

• links to: Review Board