[HIVE-8699] Enable support for common map join [Spark Branch]

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Spark
• Assignee: Unassigned
• Reporter: Xuefu Zhang
• Created: 02/Nov/14 14:59
• Updated: 04/Nov/14 20:00

Description

This JIRA is to track issues related to common map-join support in Spark, including logical and physical optimizations. HIVE-8616 provided initialial processing, mainly represented by SparkMapJoinOptimizer. We need to continue the work to make map join work from end to end, including enhancement needed for SparkMapJoinOptimizer and subsequent physical optimization SparkMapJoinResolver.

Issue Links

• is part of: HIVE-7292 Hive on Spark
• is related to: HIVE-8616 convert joinOp to MapJoinOp and generate MapWorks only [Spark Branch]
• is related to: HIVE-7613 Research optimization of auto convert join to map join [Spark branch]

Sub-Tasks

• Broadcast files for small tables via SparkContext.addFile() and SparkFiles.get() [Spark Branch]: Open
• Enable non-staged mapjoin [Spark Branch]: Open
• Union input to a join operator poses problem when converting to map join [Spark Branch]: Open