Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4400

Aggregate runtime filters locally

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: None
    • Component/s: Backend
    • Labels:

      Description

      At the moment, runtime filters are sent from each fragment instance directly to the coordinator for aggregation (ORing) at the coordinator.

      With multi-threaded execution, we will have an order of magnitude more fragment instances per node, at which point the coordinator would become a bottleneck during the aggregation process. To avoid that, we need to aggregate the local instances' runtime filters at each node before sending the filter off to the coordinator.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tarmstrong Tim Armstrong
                Reporter:
                marcelk Marcel Kinard
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: