Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-2177

In BroadcastJoinRule, the total volume of broadcast tables should be checked before stages are merged

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.3
    • Fix Version/s: 0.12.0, 0.11.4
    • Component/s: Planner/Optimizer
    • Labels:
      None

      Description

      Currently, one of broadcast candidates may be marked as not being broadcastable because the total volume of the merged stage exceeds the broadcast join threshold.
      This is done in BroadcastJoinRule::checkTotalSizeOfBroadcastableRelations(), but it causes a bug because it doesn't split the merged stage into separate ones again.
      This routine should be improved to check the total volume of broadcast candidates before merging stages.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jihoonson Jihoon Son
                Reporter:
                jihoonson Jihoon Son
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: