Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4511

Regression : Reported Fragment total time is incorrect

    XMLWordPrintableJSON

Details

    Description

      A recent change seems to have broken the fragment reported timings

          Averaged Fragment F00:(Total: 43.874ms, non-child: 0.000ns, % non-child: 0.00%)
            split sizes:  min: 7.88 GB, max: 7.95 GB, avg: 7.92 GB, stddev: 21.60 MB
            completion times: min:19s509ms  max:19s883ms  mean: 19s720ms  stddev:112.439ms
            execution rates: min:407.03 MB/sec  max:416.52 MB/sec  mean:411.07 MB/sec  stddev:2.89 MB/sec
            num instances: 15
             - AverageThreadTokens: 11.32 
             - BloomFilterBytes: 17.00 MB (17825792)
             - PeakMemoryUsage: 123.02 MB (128992933)
             - PerHostPeakMemUsage: 844.39 MB (885405464)
             - RowsProduced: 6.75M (6745672)
             - TotalCpuTime: 3m25s
             - TotalNetworkReceiveTime: 0.000ns
             - TotalNetworkSendTime: 17s164ms
             - TotalStorageWaitTime: 394.254ms
            Fragment Instance Lifecycle Timings:
               - ExecTime: 19s670ms
                 - ExecTreeExecTime: 1s034ms
               - OpenTime: 17.251ms
                 - ExecTreeOpenTime: 142.269us
               - PrepareTime: 26.600ms
                 - ExecTreePrepareTime: 20.760ms
      

      Before the regression

          Averaged Fragment F00:(Total: 29s527ms, non-child: 3s362ms, % non-child: 11.39%)
            split sizes:  min: 11.46 GB, max: 12.01 GB, avg: 11.77 GB, stddev: 124.72 MB
            completion times: min:17s281ms  max:46s307ms  mean: 30s414ms  stddev:7s903ms
            execution rates: min:255.29 MB/sec  max:699.00 MB/sec  mean:426.25 MB/sec  stddev:121.13 MB/sec
            num instances: 32
             - AverageThreadTokens: 3.86 
             - BloomFilterBytes: 27.00 MB (28311552)
             - PeakMemoryUsage: 219.11 MB (229755142)
             - PerHostPeakMemUsage: 1.48 GB (1588976755)
             - PrepareTime: 171.122ms
             - RowsProduced: 7.59M (7590805)
             - TotalCpuTime: 1m42s
             - TotalNetworkReceiveTime: 16s432ms
             - TotalNetworkSendTime: 236.902ms
             - TotalStorageWaitTime: 499.184ms
      

      Attachments

        1. TPCH-Q3.txt
          1.46 MB
          Mostafa Mokhtar
        2. TableSinkProfile.txt
          66 kB
          Mostafa Mokhtar

        Activity

          People

            henryr Henry Robinson
            mmokhtar Mostafa Mokhtar
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: