Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4511

Regression : Reported Fragment total time is incorrect

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      A recent change seems to have broken the fragment reported timings

          Averaged Fragment F00:(Total: 43.874ms, non-child: 0.000ns, % non-child: 0.00%)
            split sizes:  min: 7.88 GB, max: 7.95 GB, avg: 7.92 GB, stddev: 21.60 MB
            completion times: min:19s509ms  max:19s883ms  mean: 19s720ms  stddev:112.439ms
            execution rates: min:407.03 MB/sec  max:416.52 MB/sec  mean:411.07 MB/sec  stddev:2.89 MB/sec
            num instances: 15
             - AverageThreadTokens: 11.32 
             - BloomFilterBytes: 17.00 MB (17825792)
             - PeakMemoryUsage: 123.02 MB (128992933)
             - PerHostPeakMemUsage: 844.39 MB (885405464)
             - RowsProduced: 6.75M (6745672)
             - TotalCpuTime: 3m25s
             - TotalNetworkReceiveTime: 0.000ns
             - TotalNetworkSendTime: 17s164ms
             - TotalStorageWaitTime: 394.254ms
            Fragment Instance Lifecycle Timings:
               - ExecTime: 19s670ms
                 - ExecTreeExecTime: 1s034ms
               - OpenTime: 17.251ms
                 - ExecTreeOpenTime: 142.269us
               - PrepareTime: 26.600ms
                 - ExecTreePrepareTime: 20.760ms
      

      Before the regression

          Averaged Fragment F00:(Total: 29s527ms, non-child: 3s362ms, % non-child: 11.39%)
            split sizes:  min: 11.46 GB, max: 12.01 GB, avg: 11.77 GB, stddev: 124.72 MB
            completion times: min:17s281ms  max:46s307ms  mean: 30s414ms  stddev:7s903ms
            execution rates: min:255.29 MB/sec  max:699.00 MB/sec  mean:426.25 MB/sec  stddev:121.13 MB/sec
            num instances: 32
             - AverageThreadTokens: 3.86 
             - BloomFilterBytes: 27.00 MB (28311552)
             - PeakMemoryUsage: 219.11 MB (229755142)
             - PerHostPeakMemUsage: 1.48 GB (1588976755)
             - PrepareTime: 171.122ms
             - RowsProduced: 7.59M (7590805)
             - TotalCpuTime: 1m42s
             - TotalNetworkReceiveTime: 16s432ms
             - TotalNetworkSendTime: 236.902ms
             - TotalStorageWaitTime: 499.184ms
      

      Attachments

        1. TPCH-Q3.txt
          1.46 MB
          Mostafa Mokhtar
        2. TableSinkProfile.txt
          66 kB
          Mostafa Mokhtar

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            henryr Henry Robinson
            mmokhtar Mostafa Mokhtar
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment