Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4638

Queries with mt_dop > 0 skip admission control

    XMLWordPrintableJSON

Details

    Description

      With mt_dop > 0 queries skip admission control

      Query (id=c34b4445491d7984:4514bfc000000000):
        Summary:
          Session ID: 7743a91e02573fae:a8f116736e980bad
          Session Type: BEESWAX
          Start Time: 2016-12-09 14:26:54.031855000
          End Time: 2016-12-09 14:27:00.713773000
          Query Type: QUERY
          Query State: FINISHED
          Query Status: OK
          Impala Version: impalad version 2.8.0-SNAPSHOT RELEASE (build a5176818172103693fa124d69982963b04b1098e)
          User: systest@HALXG.CLOUDERA.COM
          Connected User: systest@HALXG.CLOUDERA.COM
          Delegated User: 
          Network Address: 10.17.234.19:60064
          Default Db: tpch_300_parquet
          Sql Statement: select count(*) from lineitem group by l_orderkey having count(*) > 10000
          Coordinator: vd1309.halxg.cloudera.com:22000
          Query Options (non default): NUM_SCANNER_THREADS=1,MT_DOP=8
          Plan: 
      ----------------
      Estimated Per-Host Requirements: Memory=7.98GB VCores=2
      WARNING: Spilling is disabled for this query as a safety guard.
      Reason: Query option disable_unsafe_spills is set, at least one table
      is missing relevant stats, and no plan hints were given.
      
      PLAN-ROOT SINK
      |
      04:EXCHANGE [UNPARTITIONED]
      |  hosts=7 per-host-mem=unavailable
      |  tuple-ids=1 row-size=16B cardinality=43790950
      |
      03:AGGREGATE [FINALIZE]
      |  output: count:merge(*)
      |  group by: l_orderkey
      |  having: count(*) > 10000
      |  hosts=7 per-host-mem=735.02MB
      |  tuple-ids=1 row-size=16B cardinality=43790950
      |
      02:EXCHANGE [HASH(l_orderkey)]
      |  hosts=7 per-host-mem=0B
      |  tuple-ids=1 row-size=16B cardinality=437909504
      |
      01:AGGREGATE [STREAMING]
      |  output: count(*)
      |  group by: l_orderkey
      |  hosts=7 per-host-mem=7.18GB
      |  tuple-ids=1 row-size=16B cardinality=437909504
      |
      00:SCAN HDFS [tpch_300_parquet.lineitem, RANDOM]
         partitions=1/1 files=259 size=63.71GB
         table stats: 1799989091 rows total
         column stats: all
         hosts=7 per-host-mem=88.00MB
         tuple-ids=0 row-size=8B cardinality=1799989091
      ----------------
          Estimated Per-Host Mem: 8570202713
          Estimated Per-Host VCores: 2
          Request Pool: root.systest
          ExecSummary: 
      Operator       #Hosts   Avg Time   Max Time    #Rows  Est. #Rows   Peak Mem  Est. Peak Mem  Detail                    
      ----------------------------------------------------------------------------------------------------------------------
      04:EXCHANGE         1    7.533us    7.533us        0      43.79M          0        -1.00 B  UNPARTITIONED             
      03:AGGREGATE       56    2s643ms    2s861ms        0      43.79M  402.06 MB      735.02 MB  FINALIZE                  
      02:EXCHANGE        56  276.883ms  301.636ms  450.04M     437.91M          0              0  HASH(l_orderkey)          
      01:AGGREGATE       56    2s189ms    2s519ms  450.04M     437.91M  521.33 MB        7.18 GB  STREAMING                 
      00:SCAN HDFS       56  506.329ms  649.212ms    1.80B       1.80B   12.47 MB       88.00 MB  tpch_300_parquet.lineitem 
          Planner Timeline: 2.319ms
             - Analysis finished: 674.915us (674.915us)
             - Equivalence classes computed: 766.195us (91.280us)
             - Single node plan created: 1.193ms (427.359us)
             - Runtime filters computed: 1.222ms (29.215us)
             - Distributed plan created: 1.372ms (149.684us)
             - Lineage info computed: 1.413ms (41.209us)
             - Parallel plans created: 1.465ms (51.646us)
             - Planning finished: 2.319ms (854.253us)
          Query Timeline: 6s705ms
             - Query submitted: 70.897us (70.897us)
             - Planning finished: 3.788ms (3.717ms)
             - Ready to start 113 fragment instances: 5.457ms (1.669ms)
             - All 113 fragment instances started: 157.998ms (152.540ms)
             - Rows available: 6s539ms (6s381ms)
             - First row fetched: 6s627ms (87.972ms)
             - Unregister query: 6s681ms (54.351ms)
           - ComputeScanRangeAssignmentTimer: 445.101us
      

      Attachments

        Activity

          People

            alex.behm Alexander Behm
            mmokhtar Mostafa Mokhtar
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: