Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3902

Multi-threaded query execution

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Epic
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • Impala 2.6.0
    • Impala 4.0.0
    • Backend
    • Impala Multi-threaded query execution

    Description

      Currently, a single query fragment is run in a quasi-single threaded manner on a node: the scanners are run in multiple threads, but all other operators (joins, aggregation) are run in the main thread.

      The goal is to add multi-threaded execution on a single node by running multiple fragment instances (each of which runs in a single thread).

      Attachments

        Issue Links

        is a parent of

        Sub-task - The sub-task of the issue IMPALA-3567 Factor out join build-side

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved
        is depended upon by

        Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. IMPALA-9807 Multi-threaded query execution - Further Enhancements

        • Major - Major loss of function.
        • Open

        Umbrella - An overarching type made of sub-tasks IMPALA-8965 Cleanup after switch to multi-threaded query execution

        • Major - Major loss of function.
        • Open
        is duplicated by

        Improvement - An improvement or enhancement to an existing feature or task. IMPALA-2849 Investigate on improving the parallelism on a single node

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved
        is related to

        Improvement - An improvement or enhancement to an existing feature or task. IMPALA-753 Parallelize HBase scan

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved
        relates to

        Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. IMPALA-10486 Multithreading upgrade path for large clusters

        • Major - Major loss of function.
        • Open
        There are no Sub-Tasks for this issue.
         
        		 IMPALA-4080 Share codegen work between fragment instances Improvement Resolved Bikramjeet Vig
        Actions
         
        		 IMPALA-4224 Add backend support for join build sinks in parallel plans Improvement Resolved Tim Armstrong
        Actions
         
        		 IMPALA-4400 Aggregate runtime filters locally Improvement Resolved Tim Armstrong
        Actions
         
        		 IMPALA-6267 MT Scanners do not check runtime filters per-file before processing each split Bug Resolved Tim Armstrong
        Actions
         
        		 IMPALA-8125 Limit number of files generated by insert Improvement Resolved Bikramjeet Vig
        Actions
         
        		 IMPALA-9000 Fix all the TODO-MT comments Improvement Resolved Tim Armstrong
        Actions
         
        		 IMPALA-9099 Allow setting mt_dop manually for queries with joins New Feature Resolved Tim Armstrong
        Actions
         
        		 IMPALA-9126 Cleanly separate build and probe state in hash join node Improvement Resolved Tim Armstrong
        Actions
         
        		 IMPALA-9127 Clean up probe-side state machine in hash join Improvement Resolved Tim Armstrong
        Actions
         
        		 IMPALA-9156 Share broadcast join builds between fragments Improvement Resolved Tim Armstrong
        Actions
         
        		 IMPALA-9176 Make access to null-aware partition from PartitionedHashJoinNode read-only Improvement Resolved Tim Armstrong
        Actions
         
        		 IMPALA-9197 Hash table lookup should be read-only Improvement Resolved Tim Armstrong
        Actions
         
        		 IMPALA-9302 Multithreaded scanners don't check for filter effectiveness Improvement Resolved Tim Armstrong
        Actions
         
        		 IMPALA-9317 Improve number of instances estimate for scans in planner Improvement Resolved Tim Armstrong
        Actions
         
        		 IMPALA-9318 Guard rail for mt_dop value Improvement Resolved Joe McDonnell
        Actions
         
        		 IMPALA-9422 Improve join builder profiles Improvement Resolved Bikramjeet Vig
        Actions
         
        		 IMPALA-9586 Update query option docs to account for interactions with mt_dop Improvement Resolved Tim Armstrong
        Actions
         
        		 IMPALA-9782 KuduPartitionExpr is not thread-safe Bug Resolved Tim Armstrong
        Actions
         
        		 IMPALA-10343 control_service_queue_mem_limit default is too low for large clusters Improvement Resolved Tim Armstrong
        Actions
         
        		 IMPALA-10351 Enable mt_dop for DML Improvement Resolved Tim Armstrong
        Actions

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            tarmstrong Tim Armstrong
            marcelk Marcel Kinard
            Votes:
            4 Vote for this issue
            Watchers:
            35 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment