[TAJO-1766] Improve the performance of cross join

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.11.0
• Component/s: distributed query plan
• Assignee: Jihoon Son
• Reporter: Jihoon Son
• Created: 16/Aug/15 07:07
• Updated: 25/Aug/15 10:45
• Resolved: 25/Aug/15 09:24

Description

Cross join is one of the very heavy operations. Furthermore, this operator is performed by a single worker in the current implementation. (Please see the implementation of HashPartitioner. If partitionKeyIds is empty, getPartition() always returns a single value.)

One possible alternative is executing cross join with broadcast join. That is, outer table (smaller one) is always broadcasted, and join is performed by the machine who stores a part of inner table.

To do so, a new session variable is required to set the broadcast threshold for cross join.