Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Frontend
    • Labels:

      Description

      • COMPUTE STATS on Parquet tables should be run with MT_DOP=4 by default.
      • COMPUTE STATS on non-Parquet tables will run without MT_DOP.
      • Users can always override the behavior by setting MT_DOP manually. Setting MT_DOP to 0 means a statement will be run in the conventional execution mode (without intra-node paralellism based on multiple fragment instances)

        Activity

        Hide
        jrussell John Russell added a comment -

        Made the relevent doc change in https://gerrit.cloudera.org/#/c/5652/

        Show
        jrussell John Russell added a comment - Made the relevent doc change in https://gerrit.cloudera.org/#/c/5652/
        Hide
        alex.behm Alexander Behm added a comment -

        commit 7efa08316ecb8f73d1c968ed602d11d40c714a1f
        Author: Alex Behm <alex.behm@cloudera.com>
        Date: Thu Dec 1 13:58:19 2016 -0800

        IMPALA-4572: Run COMPUTE STATS on Parquet tables with MT_DOP=4.

        COMPUTE STATS on Parquet tables is run with MT_DOP=4 by default.
        COMPUTE STATS on non-Parquet tables will run without MT_DOP.

        Users can always override the behavior by setting MT_DOP manually.
        Setting MT_DOP to 0 means a statement will be run in the
        conventional execution mode (without intra-node paralellism based
        on multiple fragment instances). Users can set a higher MT_DOP
        even for Parquet tables.

        Testing: Added a new test that checks the effective MT_DOP.
        Locally ran test_mt_dop.py, test_scanners.py, test_nested_types.py,
        test_compute_stats.py, and test_cancellation.py.

        Change-Id: I2be3c7c9f3004e9a759224a2e5756eb6e4efa359
        Reviewed-on: http://gerrit.cloudera.org:8080/5315
        Reviewed-by: Alex Behm <alex.behm@cloudera.com>
        Tested-by: Internal Jenkins

        Show
        alex.behm Alexander Behm added a comment - commit 7efa08316ecb8f73d1c968ed602d11d40c714a1f Author: Alex Behm <alex.behm@cloudera.com> Date: Thu Dec 1 13:58:19 2016 -0800 IMPALA-4572 : Run COMPUTE STATS on Parquet tables with MT_DOP=4. COMPUTE STATS on Parquet tables is run with MT_DOP=4 by default. COMPUTE STATS on non-Parquet tables will run without MT_DOP. Users can always override the behavior by setting MT_DOP manually. Setting MT_DOP to 0 means a statement will be run in the conventional execution mode (without intra-node paralellism based on multiple fragment instances). Users can set a higher MT_DOP even for Parquet tables. Testing: Added a new test that checks the effective MT_DOP. Locally ran test_mt_dop.py, test_scanners.py, test_nested_types.py, test_compute_stats.py, and test_cancellation.py. Change-Id: I2be3c7c9f3004e9a759224a2e5756eb6e4efa359 Reviewed-on: http://gerrit.cloudera.org:8080/5315 Reviewed-by: Alex Behm <alex.behm@cloudera.com> Tested-by: Internal Jenkins

          People

          • Assignee:
            alex.behm Alexander Behm
            Reporter:
            alex.behm Alexander Behm
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development