Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5200

Profile timers not updated during long-running sort

    XMLWordPrintableJSON

    Details

      Description

      If you have a query plan with a long-running sort operation (e.g. minutes), the profile timers are not updated to reflect the time spent in the sort until the sort starts returning rows.

      E.g. this is a summary from a sort query that was running for a few hours . The summary was misleading and the "heat map" plan in the debug web UI is misleading - it showed the join as the "hot" operator. It would be ideal if we could somehow at least periodically update the time spent in the operator.

      Operator              #Hosts   Avg Time   Max Time    #Rows  Est. #Rows   Peak Mem  Est. Peak Mem  Detail                   
      ----------------------------------------------------------------------------------------------------------------------------
      05:MERGING-EXCHANGE        1    0.000ns    0.000ns        0     635.58M          0        -1.00 B  UNPARTITIONED            
      03:SORT                    1    0.000ns    0.000ns        0     635.58M   47.86 GB      800.00 MB                           
      02:HASH JOIN               1    4s859ms    4s859ms  771.02M     635.58M  162.11 MB       16.03 MB  INNER JOIN, BROADCAST    
      |--04:EXCHANGE             1   38.988ms   38.988ms  247.20K     247.20K          0              0  BROADCAST                
      |  01:SCAN HDFS            1    8s089ms    8s089ms  247.20K     247.20K    3.79 MB       32.00 MB  product b     
      00:SCAN HDFS               1  209.997ms  209.997ms   15.09M     635.58M  185.27 MB      176.00 MB  sales a
      

      http://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Impala-join-query-running-slow

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                tarmstrong Tim Armstrong
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: