[TAJO-2177] In BroadcastJoinRule, the total volume of broadcast tables should be checked before stages are merged

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.11.3
• Fix Version/s: 0.12.0, 0.11.4
• Component/s: Planner/Optimizer
• Assignee: Jihoon Son
• Reporter: Jihoon Son
• Created: 15/Jul/16 12:39
• Updated: 19/Jul/16 15:00
• Resolved: 19/Jul/16 14:16

Description

Currently, one of broadcast candidates may be marked as not being broadcastable because the total volume of the merged stage exceeds the broadcast join threshold.
This is done in BroadcastJoinRule::checkTotalSizeOfBroadcastableRelations(), but it causes a bug because it doesn't split the merged stage into separate ones again.
This routine should be improved to check the total volume of broadcast candidates before merging stages.

Issue Links

• links to: GitHub Pull Request #1040