Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3902 Multi-threaded query execution
  3. IMPALA-4270

Planner should disallow queries with mt_dop > 0 that are not executable.

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.6.0, Impala 2.7.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Frontend
    • Labels:
      None

      Description

      We can currently execute queries with mt_dop > 0 as long as there are no distributed joins in the plan. The planner should detect such unsupported cases and fail queries with an appropriate error message.

      For HDFS scans that do not yet have an optimized MT implementation in the backend we should fall back to the old scan node with num_scanner_threads=1.

      This change will help to enable some of our tests with mt_dop > 0.

        Activity

        Hide
        alex.behm Alexander Behm added a comment -

        commit 04802535661979c50e5d06ef04e62eee677b901e
        Author: Alex Behm <alex.behm@cloudera.com>
        Date: Mon Oct 10 11:03:43 2016 -0700

        IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0.

        MT_DOP > 0 is only supported for plans without distributed joins
        or table sinks. Adds validation to fail unsupported queries
        gracefully in planning.

        For scans in queries that are executable with MT_DOP > 0 we either
        use the optimized MT scan node BE implementation (only Parquet), or
        we use the conventional scan node with num_scanner_threads=1.

        TODO: Still need to add end-to-end tests.

        Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d
        Reviewed-on: http://gerrit.cloudera.org:8080/4677
        Reviewed-by: Alex Behm <alex.behm@cloudera.com>
        Tested-by: Internal Jenkins

        Show
        alex.behm Alexander Behm added a comment - commit 04802535661979c50e5d06ef04e62eee677b901e Author: Alex Behm <alex.behm@cloudera.com> Date: Mon Oct 10 11:03:43 2016 -0700 IMPALA-4270 : Gracefully fail unsupported queries with mt_dop > 0. MT_DOP > 0 is only supported for plans without distributed joins or table sinks. Adds validation to fail unsupported queries gracefully in planning. For scans in queries that are executable with MT_DOP > 0 we either use the optimized MT scan node BE implementation (only Parquet), or we use the conventional scan node with num_scanner_threads=1. TODO: Still need to add end-to-end tests. Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Reviewed-on: http://gerrit.cloudera.org:8080/4677 Reviewed-by: Alex Behm <alex.behm@cloudera.com> Tested-by: Internal Jenkins

          People

          • Assignee:
            alex.behm Alexander Behm
            Reporter:
            alex.behm Alexander Behm
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development