Query (id=2145dc34def319e3:82987cee00000000): Summary: Session ID: e74416b9e7a12f10:2a8ba0a01789c3b7 Session Type: BEESWAX Start Time: 2018-05-13 19:03:33.173707000 End Time: 2018-05-13 19:03:56.833026000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.12.0-cdh5.15.0 RELEASE (build 2f9498d5c2f980aa7ff9505c56654c8e59e026ca) User: mmokhtar Connected User: mmokhtar Delegated User: Network Address: ::ffff:10.17.234.27:45692 Default Db: tpch_300_parquet Sql Statement: select * FROM ( SELECT o.*, ROW_NUMBER() OVER(ORDER BY evt_ts DESC) AS rn FROM ( SELECT l_orderkey,l_partkey,l_linenumber,l_quantity, cast (l_shipdate as string) evt_ts FROM lineitem WHERE l_shipdate BETWEEN '1992-01-01 00:00:00' AND '1992-01-15 00:00:00' ) o ) r WHERE rn BETWEEN 1 AND 101 ORDER BY rn Coordinator: vd1317.foo.com:22000 Query Options (set by configuration): EXPLAIN_LEVEL=2 Query Options (set by configuration and planner): EXPLAIN_LEVEL=2,MT_DOP=0 Plan: ---------------- Max Per-Host Resource Reservation: Memory=22.00MB Per-Host Resource Estimates: Memory=606.00MB F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=36.00MB mem-reservation=16.00MB PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B | 04:SORT | order by: rn ASC | mem-estimate=32.00MB mem-reservation=12.00MB spill-buffer=2.00MB | tuple-ids=4 row-size=56B cardinality=17999891 | 03:SELECT | predicates: row_number() <= 101, row_number() >= 1 | mem-estimate=0B mem-reservation=0B | tuple-ids=7,6 row-size=56B cardinality=17999891 | 02:ANALYTIC | functions: row_number() | order by: CAST(l_shipdate AS STRING) DESC | window: ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW | mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB | tuple-ids=7,6 row-size=56B cardinality=179998909 | 05:MERGING-EXCHANGE [UNPARTITIONED] | order by: CAST(l_shipdate AS STRING) DESC | mem-estimate=0B mem-reservation=0B | tuple-ids=7 row-size=48B cardinality=179998909 | F00:PLAN FRAGMENT [RANDOM] hosts=7 instances=7 Per-Host Resources: mem-estimate=570.00MB mem-reservation=6.00MB 01:SORT | order by: CAST(l_shipdate AS STRING) DESC | mem-estimate=130.00MB mem-reservation=6.00MB spill-buffer=2.00MB | tuple-ids=7 row-size=48B cardinality=179998909 | 00:SCAN HDFS [tpch_300_parquet.lineitem, RANDOM] partitions=1/1 files=266 size=64.56GB predicates: l_shipdate <= TIMESTAMP '1992-01-15 00:00:00', l_shipdate >= TIMESTAMP '1992-01-01 00:00:00' stored statistics: table: rows=1799989091 size=64.56GB columns: all extrapolated-rows=disabled parquet statistics predicates: l_shipdate <= TIMESTAMP '1992-01-15 00:00:00', l_shipdate >= TIMESTAMP '1992-01-01 00:00:00' parquet dictionary predicates: l_shipdate <= TIMESTAMP '1992-01-15 00:00:00', l_shipdate >= TIMESTAMP '1992-01-01 00:00:00' mem-estimate=440.00MB mem-reservation=0B tuple-ids=0 row-size=48B cardinality=179998909 ---------------- Estimated Per-Host Mem: 635437056 Per Host Min Reservation: vd1317.foo.com:22000(22.00 MB) vd1318.foo.com:22000(6.00 MB) vd1319.foo.com:22000(6.00 MB) vd1321.foo.com:22000(6.00 MB) vd1322.foo.com:22000(6.00 MB) vd1323.foo.com:22000(6.00 MB) vd1324.foo.com:22000(6.00 MB) Request Pool: root.mmokhtar Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ----------------------------------------------------------------------------------------------------------------------------- 04:SORT 1 2.711ms 2.711ms 101 18.00M 20.02 MB 32.00 MB 03:SELECT 1 2.053ms 2.053ms 101 18.00M 8.02 MB 0 02:ANALYTIC 1 85.512ms 85.512ms 648.97K 180.00M 13.14 MB 4.00 MB 05:MERGING-EXCHANGE 1 23s306ms 23s306ms 648.97K 180.00M 10.46 MB 0 UNPARTITIONED 01:SORT 7 6s751ms 15s638ms 648.97K 180.00M 10.07 MB 130.00 MB 00:SCAN HDFS 7 245.039ms 380.887ms 648.97K 180.00M 123.88 MB 440.00 MB tpch_300_parquet.lineitem Errors: Query Compilation: 11.973ms - Metadata of all 1 tables cached: 780.752us (780.752us) - Analysis finished: 6.057ms (5.276ms) - Value transfer graph computed: 6.231ms (174.014us) - Single node plan created: 9.307ms (3.076ms) - Runtime filters computed: 9.469ms (162.837us) - Distributed plan created: 9.539ms (69.632us) - Lineage info computed: 9.752ms (213.006us) - Planning finished: 11.973ms (2.220ms) Query Timeline: 23s663ms - Query submitted: 59.154us (59.154us) - Planning finished: 14.623ms (14.564ms) - Submit for admission: 15.872ms (1.249ms) - Completed admission: 15.974ms (101.329us) - Ready to start on 7 backends: 16.320ms (346.230us) - All 7 execution backends (8 fragment instances) started: 17.626ms (1.306ms) - Rows available: 23s575ms (23s557ms) - First row fetched: 23s655ms (79.938ms) - Last row fetched: 23s655ms (728.013us) - Released admission control resources: 23s656ms (427.373us) - Unregister query: 23s659ms (3.091ms) - ComputeScanRangeAssignmentTimer: 541.386us ImpalaServer: - ClientFetchWaitTimer: 82.068ms - RowMaterializationTimer: 2.097ms Execution Profile 2145dc34def319e3:82987cee00000000:(Total: 23s561ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- Backend startup latencies: Count: 7, min / max: 1ms / 1ms, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 1ms Per Node Peak Memory Usage: vd1317.foo.com:22000(34.50 MB) vd1318.foo.com:22000(132.72 MB) vd1319.foo.com:22000(67.10 MB) vd1321.foo.com:22000(67.10 MB) vd1322.foo.com:22000(7.74 MB) vd1323.foo.com:22000(67.19 MB) vd1324.foo.com:22000(132.72 MB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns - NumBackends: 7 (7) - NumFragmentInstances: 8 (8) - NumFragments: 2 (2) Averaged Fragment F01:(Total: 23s637ms, non-child: 83.598ms, % non-child: 0.35%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:23s638ms max:23s638ms mean: 23s638ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 34.50 MB (36180016) - PeakReservation: 16.00 MB (16777216) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 34.50 MB (36180016) - RowsProduced: 101 (101) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 14 (14) - TotalThreadsTotalWallClockTime: 23s636ms - TotalThreadsSysTime: 25.997ms - TotalThreadsUserTime: 7s603ms - TotalThreadsVoluntaryContextSwitches: 5 (5) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 80.239ms - ExecTreeExecTime: 22.091us - OpenTime: 23s556ms - ExecTreeOpenTime: 23s396ms - PrepareTime: 768.183us - ExecTreePrepareTime: 187.817us PLAN_ROOT_SINK: - PeakMemoryUsage: 0 SORT_NODE (id=4):(Total: 23s396ms, non-child: 2.711ms, % non-child: 0.01%) - InMemorySortTime: 15.181us - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 20.02 MB (20987904) - RowsReturned: 101 (101) - RowsReturnedRate: 4.00 /sec - SortDataSize: 7.50 KB (7676) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) Buffer pool: - AllocTime: 27.030us - CumulativeAllocationBytes: 6.00 MB (6291456) - CumulativeAllocations: 3 (3) - PeakReservation: 12.00 MB (12582912) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 6.00 MB (6291456) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns SELECT_NODE (id=3):(Total: 23s393ms, non-child: 2.053ms, % non-child: 0.01%) - PeakMemoryUsage: 8.02 MB (8404992) - RowsReturned: 101 (101) - RowsReturnedRate: 4.00 /sec ANALYTIC_EVAL_NODE (id=2):(Total: 23s391ms, non-child: 85.512ms, % non-child: 0.37%) - EvaluationTime: 82.336ms - PeakMemoryUsage: 13.14 MB (13780992) - RowsReturned: 648.97K (648967) - RowsReturnedRate: 27.74 K/sec Buffer pool: - AllocTime: 94.428us - CumulativeAllocationBytes: 32.00 MB (33554432) - CumulativeAllocations: 16 (16) - PeakReservation: 4.00 MB (4194304) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 4.00 MB (4194304) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=5):(Total: 23s306ms, non-child: 23s306ms, % non-child: 100.00%) - ConvertRowBatchTime: 0.000ns - PeakMemoryUsage: 10.46 MB (10970713) - RowsReturned: 648.97K (648967) - RowsReturnedRate: 27.84 K/sec Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns DataStreamReceiver: - BytesReceived: 13.77 MB (14434967) - DeserializeRowBatchTimer: 34.797ms - FirstBatchArrivalWaitTime: 15s926ms - MergeGetNext: 7s378ms - MergeGetNextBatch: 1.231ms - PeakMemoryUsage: 10.34 MB (10843737) - SendersBlockedTimer: 13s699ms - SendersBlockedTotalTimer(*): 37s573ms CodeGen:(Total: 157.500ms, non-child: 157.500ms, % non-child: 100.00%) - CodegenTime: 3.521ms - CompileTime: 32.089ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039864) - NumFunctions: 140 (140) - NumInstructions: 1.98K (1981) - OptimizationTime: 104.572ms - PeakMemoryUsage: 990.50 KB (1014272) - PrepareTime: 20.188ms Coordinator Fragment F01: Instance 2145dc34def319e3:82987cee00000000 (host=vd1317.foo.com:22000):(Total: 23s637ms, non-child: 83.598ms, % non-child: 0.35%) Fragment Instance Lifecycle Event Timeline: 23s637ms - Prepare Finished: 1.057ms (1.057ms) - Open Finished: 23s557ms (23s556ms) - First Batch Produced: 23s557ms (19.787us) - First Batch Sent: 23s637ms (80.206ms) - ExecInternal Finished: 23s637ms (8.734us) MemoryUsage(500.000ms): 12.00 KB, 20.84 KB, 20.84 KB, 20.84 KB, 20.84 KB, 20.84 KB, 20.84 KB, 20.84 KB, 20.84 KB, 20.84 KB, 20.84 KB, 20.84 KB, 20.84 KB, 20.84 KB, 10.03 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.19 MB, 10.19 MB, 10.19 MB, 10.19 MB, 10.19 MB, 10.19 MB, 10.19 MB, 26.47 MB, 27.47 MB, 28.48 MB, 29.51 MB, 29.53 MB, 30.51 MB, 31.56 MB, 31.50 MB, 32.54 MB, 33.51 MB, 32.47 MB, 33.57 MB, 31.17 MB, 22.89 MB, 21.54 MB, 12.02 MB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 34.50 MB (36180016) - PeakReservation: 16.00 MB (16777216) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 34.50 MB (36180016) - RowsProduced: 101 (101) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 14 (14) - TotalThreadsTotalWallClockTime: 23s636ms - TotalThreadsSysTime: 25.997ms - TotalThreadsUserTime: 7s603ms - TotalThreadsVoluntaryContextSwitches: 5 (5) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 80.239ms - ExecTreeExecTime: 22.091us - OpenTime: 23s556ms - ExecTreeOpenTime: 23s396ms - PrepareTime: 768.183us - ExecTreePrepareTime: 187.817us PLAN_ROOT_SINK: - PeakMemoryUsage: 0 SORT_NODE (id=4):(Total: 23s396ms, non-child: 2.711ms, % non-child: 0.01%) SortType: Total ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 101 (101) ; Min: 101 (101) ; Max: 101 (101) ; Number of samples: 1) - InMemorySortTime: 15.181us - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 20.02 MB (20987904) - RowsReturned: 101 (101) - RowsReturnedRate: 4.00 /sec - SortDataSize: 7.50 KB (7676) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) Buffer pool: - AllocTime: 27.030us - CumulativeAllocationBytes: 6.00 MB (6291456) - CumulativeAllocations: 3 (3) - PeakReservation: 12.00 MB (12582912) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 6.00 MB (6291456) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns SELECT_NODE (id=3):(Total: 23s393ms, non-child: 2.053ms, % non-child: 0.01%) ExecOption: Codegen Enabled - PeakMemoryUsage: 8.02 MB (8404992) - RowsReturned: 101 (101) - RowsReturnedRate: 4.00 /sec ANALYTIC_EVAL_NODE (id=2):(Total: 23s391ms, non-child: 85.512ms, % non-child: 0.37%) - EvaluationTime: 82.336ms - PeakMemoryUsage: 13.14 MB (13780992) - RowsReturned: 648.97K (648967) - RowsReturnedRate: 27.74 K/sec Buffer pool: - AllocTime: 94.428us - CumulativeAllocationBytes: 32.00 MB (33554432) - CumulativeAllocations: 16 (16) - PeakReservation: 4.00 MB (4194304) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 4.00 MB (4194304) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=5):(Total: 23s306ms, non-child: 23s306ms, % non-child: 100.00%) ExecOption: Codegen Enabled - ConvertRowBatchTime: 0.000ns - PeakMemoryUsage: 10.46 MB (10970713) - RowsReturned: 648.97K (648967) - RowsReturnedRate: 27.84 K/sec Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns DataStreamReceiver: BytesReceived(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3.93 MB, 3.97 MB, 3.97 MB, 3.97 MB, 3.97 MB, 3.97 MB, 3.97 MB, 3.97 MB, 3.97 MB, 3.97 MB, 3.97 MB, 4.04 MB, 4.04 MB, 4.04 MB, 4.04 MB, 4.04 MB, 4.04 MB, 4.04 MB, 4.12 MB, 5.05 MB, 5.94 MB, 6.84 MB, 7.77 MB, 8.67 MB, 9.57 MB, 10.53 MB, 11.42 MB, 12.35 MB, 12.84 MB, 13.77 MB, 13.77 MB, 13.77 MB, 13.77 MB - BytesReceived: 13.77 MB (14434967) - DeserializeRowBatchTimer: 34.797ms - FirstBatchArrivalWaitTime: 15s926ms - MergeGetNext: 7s378ms - MergeGetNextBatch: 1.231ms - PeakMemoryUsage: 10.34 MB (10843737) - SendersBlockedTimer: 13s699ms - SendersBlockedTotalTimer(*): 37s573ms CodeGen:(Total: 157.500ms, non-child: 157.500ms, % non-child: 100.00%) - CodegenTime: 3.521ms - CompileTime: 32.089ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039864) - NumFunctions: 140 (140) - NumInstructions: 1.98K (1981) - OptimizationTime: 104.572ms - PeakMemoryUsage: 990.50 KB (1014272) - PrepareTime: 20.188ms Averaged Fragment F00:(Total: 12s492ms, non-child: 5s378ms, % non-child: 43.05%) split sizes: min: 9.06 GB, max: 9.40 GB, avg: 9.22 GB, stddev: 98.87 MB completion times: min:112.441ms max:21s434ms mean: 12s493ms stddev:9s144ms execution rates: min:432.70 MB/sec max:82.28 GB/sec mean:22.54 GB/sec stddev:34.72 GB/sec num instances: 7 - AverageThreadTokens: 0.88 - BloomFilterBytes: 0 - PeakMemoryUsage: 68.88 MB (72227790) - PeakReservation: 7.14 MB (7489828) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 71.31 MB (74771187) - RowsProduced: 92.71K (92709) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 5s378ms - TotalStorageWaitTime: 45.022ms - TotalThreadsInvoluntaryContextSwitches: 10 (10) - TotalThreadsTotalWallClockTime: 13s026ms - TotalThreadsSysTime: 18.938ms - TotalThreadsUserTime: 7s195ms - TotalThreadsVoluntaryContextSwitches: 381 (381) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 5s391ms - ExecTreeExecTime: 516.364us - OpenTime: 7s100ms - ExecTreeOpenTime: 6s995ms - PrepareTime: 636.105us - ExecTreePrepareTime: 250.855us DataStreamSender (dst_id=5):(Total: 14.296ms, non-child: 14.296ms, % non-child: 100.00%) - BytesSent: 1.97 MB (2062138) - NetworkThroughput(*): 740.77 MB/sec - OverallThroughput: 105.82 MB/sec - PeakMemoryUsage: 2.61 KB (2672) - RowsReturned: 92.71K (92709) - SerializeBatchTime: 11.974ms - TransmitDataRPCTime: 2.210ms - UncompressedRowBatchSize: 5.04 MB (5284445) SORT_NODE (id=1):(Total: 6s996ms, non-child: 6s751ms, % non-child: 96.50%) - InMemorySortTime: 6s742ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 7.20 MB (7550098) - RowsReturned: 92.71K (92709) - RowsReturnedRate: 9.57 K/sec - SortDataSize: 4.33 MB (4542769) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) Buffer pool: - AllocTime: 30.030us - CumulativeAllocationBytes: 6.00 MB (6291456) - CumulativeAllocations: 3 (3) - PeakReservation: 7.14 MB (7489828) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 6.00 MB (6291456) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns HDFS_SCAN_NODE (id=0):(Total: 245.039ms, non-child: 245.039ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.29 - AverageScannerThreadConcurrency: 0.71 - BytesRead: 81.59 MB (85552883) - BytesReadDataNodeCache: 0 - BytesReadLocal: 81.59 MB (85552883) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 81.59 MB (85552883) - CachedFileHandlesHitCount: 51 (51) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 114.872ms - MaxCompressedTextFileLength: 0 - NumColumns: 5 (5) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 8 (8) - NumRowGroups: 38 (38) - NumScannerThreadsStarted: 38 (38) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 37 (37) - PeakMemoryUsage: 61.88 MB (64885481) - PerReadThreadRawHdfsThroughput: 1.26 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 242.199ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 6.87M (6868414) - RowsReturned: 92.71K (92709) - RowsReturnedRate: 275.51 K/sec - ScanRangesComplete: 38 (38) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 534.904ms - MaterializeTupleTime(*): 211.225ms - ScannerThreadsSysTime: 9.403ms - ScannerThreadsUserTime: 333.325ms - ScannerThreadsVoluntaryContextSwitches: 214 (214) - TotalRawHdfsOpenFileTime(*): 281.230us - TotalRawHdfsReadTime(*): 51.311ms - TotalReadThroughput: 93.98 MB/sec CodeGen:(Total: 103.191ms, non-child: 103.191ms, % non-child: 100.00%) - CodegenTime: 2.882ms - CompileTime: 16.414ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039864) - NumFunctions: 123 (123) - NumInstructions: 1.53K (1535) - OptimizationTime: 71.018ms - PeakMemoryUsage: 767.50 KB (785920) - PrepareTime: 15.258ms Fragment F00: Instance 2145dc34def319e3:82987cee00000004 (host=vd1318.foo.com:22000):(Total: 21s433ms, non-child: 9s047ms, % non-child: 42.21%) Hdfs split stats (:<# splits>/): 1:5/1.24 GB 2:3/760.00 MB 7:3/575.89 MB 3:3/591.53 MB 4:6/1.48 GB 6:7/1.73 GB 0:6/1.48 GB 5:5/1.24 GB Fragment Instance Lifecycle Event Timeline: 21s434ms - Prepare Finished: 793.045us (793.045us) - Open Finished: 12s350ms (12s349ms) - First Batch Produced: 12s350ms (12.642us) - First Batch Sent: 12s350ms (135.578us) - ExecInternal Finished: 21s434ms (9s083ms) MemoryUsage(500.000ms): 102.26 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 8.08 MB, 8.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB ThreadUsage(500.000ms): 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.05 - BloomFilterBytes: 0 - PeakMemoryUsage: 132.72 MB (139166411) - PeakReservation: 10.00 MB (10485760) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 132.72 MB (139166411) - RowsProduced: 185.53K (185527) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 9s050ms - TotalStorageWaitTime: 41.821ms - TotalThreadsInvoluntaryContextSwitches: 17 (17) - TotalThreadsTotalWallClockTime: 22s201ms - TotalThreadsSysTime: 19.267ms - TotalThreadsUserTime: 12s695ms - TotalThreadsVoluntaryContextSwitches: 496 (496) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 9s083ms - ExecTreeExecTime: 1.510ms - OpenTime: 12s349ms - ExecTreeOpenTime: 12s255ms - PrepareTime: 569.201us - ExecTreePrepareTime: 211.109us DataStreamSender (dst_id=5):(Total: 36.330ms, non-child: 36.330ms, % non-child: 100.00%) - BytesSent: 3.93 MB (4124889) - NetworkThroughput(*): 622.95 MB/sec - OverallThroughput: 108.28 MB/sec - PeakMemoryUsage: 2.61 KB (2672) - RowsReturned: 185.53K (185527) - SerializeBatchTime: 29.866ms - TransmitDataRPCTime: 6.314ms - UncompressedRowBatchSize: 10.09 MB (10575039) SORT_NODE (id=1):(Total: 12s256ms, non-child: 11s941ms, % non-child: 97.43%) SortType: Total ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 185.53K (185527) ; Min: 185.53K (185527) ; Max: 185.53K (185527) ; Number of samples: 1) - InMemorySortTime: 11s924ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 10.07 MB (10563584) - RowsReturned: 185.53K (185527) - RowsReturnedRate: 15.14 K/sec - SortDataSize: 8.67 MB (9090823) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) Buffer pool: - AllocTime: 31.984us - CumulativeAllocationBytes: 10.00 MB (10485760) - CumulativeAllocations: 5 (5) - PeakReservation: 10.00 MB (10485760) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 10.00 MB (10485760) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns HDFS_SCAN_NODE (id=0):(Total: 315.242ms, non-child: 315.242ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:5/1.24 GB 2:3/760.00 MB 7:3/575.89 MB 3:3/591.53 MB 4:6/1.48 GB 6:7/1.73 GB 0:6/1.48 GB 5:5/1.24 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 38 out of 38 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% File Formats: PARQUET/NONE:36 PARQUET/SNAPPY:2 BytesRead(500.000ms): 159.47 MB - FooterProcessingTime: (Avg: 649.269us ; Min: 292.164us ; Max: 1.028ms ; Number of samples: 38) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 2.00 - BytesRead: 159.47 MB (167217737) - BytesReadDataNodeCache: 0 - BytesReadLocal: 159.47 MB (167217737) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 159.47 MB (167217737) - CachedFileHandlesHitCount: 64 (64) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 209.103ms - MaxCompressedTextFileLength: 0 - NumColumns: 5 (5) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 8 (8) - NumRowGroups: 38 (38) - NumScannerThreadsStarted: 38 (38) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 36 (36) - PeakMemoryUsage: 122.65 MB (128608888) - PerReadThreadRawHdfsThroughput: 1.74 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 312.049ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 13.74M (13737052) - RowsReturned: 185.53K (185527) - RowsReturnedRate: 588.52 K/sec - ScanRangesComplete: 38 (38) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 767.972ms - MaterializeTupleTime(*): 407.974ms - ScannerThreadsSysTime: 10.270ms - ScannerThreadsUserTime: 632.078ms - ScannerThreadsVoluntaryContextSwitches: 239 (239) - TotalRawHdfsOpenFileTime(*): 276.866us - TotalRawHdfsReadTime(*): 89.375ms - TotalReadThroughput: 318.94 MB/sec CodeGen:(Total: 93.217ms, non-child: 93.217ms, % non-child: 100.00%) - CodegenTime: 2.635ms - CompileTime: 14.272ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039864) - NumFunctions: 123 (123) - NumInstructions: 1.53K (1535) - OptimizationTime: 65.024ms - PeakMemoryUsage: 767.50 KB (785920) - PrepareTime: 13.491ms Instance 2145dc34def319e3:82987cee00000005 (host=vd1324.foo.com:22000):(Total: 21s258ms, non-child: 5s144ms, % non-child: 24.20%) Hdfs split stats (:<# splits>/): 7:2/506.69 MB 2:7/1.73 GB 6:2/506.67 MB 1:9/2.23 GB 4:5/1.24 GB 0:1/253.32 MB 3:5/1.24 GB 5:7/1.73 GB Fragment Instance Lifecycle Event Timeline: 21s259ms - Prepare Finished: 823.197us (823.197us) - Open Finished: 16s087ms (16s086ms) - First Batch Produced: 16s087ms (16.226us) - First Batch Sent: 16s087ms (292.514us) - ExecInternal Finished: 21s259ms (5s171ms) MemoryUsage(500.000ms): 778.11 KB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 10.08 MB, 8.08 MB, 8.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB ThreadUsage(500.000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 132.72 MB (139166410) - PeakReservation: 10.00 MB (10485760) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 132.72 MB (139166410) - RowsProduced: 185.91K (185905) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 5s146ms - TotalStorageWaitTime: 48.109ms - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 22s083ms - TotalThreadsSysTime: 36.026ms - TotalThreadsUserTime: 16s420ms - TotalThreadsVoluntaryContextSwitches: 494 (494) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 5s171ms - ExecTreeExecTime: 918.899us - OpenTime: 16s086ms - ExecTreeOpenTime: 15s980ms - PrepareTime: 553.935us - ExecTreePrepareTime: 233.001us DataStreamSender (dst_id=5):(Total: 27.131ms, non-child: 27.131ms, % non-child: 100.00%) - BytesSent: 3.94 MB (4134345) - NetworkThroughput(*): 931.43 MB/sec - OverallThroughput: 145.32 MB/sec - PeakMemoryUsage: 2.61 KB (2672) - RowsReturned: 185.91K (185905) - SerializeBatchTime: 22.768ms - TransmitDataRPCTime: 4.233ms - UncompressedRowBatchSize: 10.11 MB (10596585) SORT_NODE (id=1):(Total: 15s981ms, non-child: 15s638ms, % non-child: 97.86%) SortType: Total ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 185.91K (185905) ; Min: 185.91K (185905) ; Max: 185.91K (185905) ; Number of samples: 1) - InMemorySortTime: 15s619ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 10.07 MB (10563584) - RowsReturned: 185.91K (185905) - RowsReturnedRate: 11.63 K/sec - SortDataSize: 8.69 MB (9109345) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) Buffer pool: - AllocTime: 49.780us - CumulativeAllocationBytes: 10.00 MB (10485760) - CumulativeAllocations: 5 (5) - PeakReservation: 10.00 MB (10485760) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 10.00 MB (10485760) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns HDFS_SCAN_NODE (id=0):(Total: 342.532ms, non-child: 342.532ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:2/506.69 MB 2:7/1.73 GB 6:2/506.67 MB 1:9/2.23 GB 4:5/1.24 GB 0:1/253.32 MB 3:5/1.24 GB 5:7/1.73 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 38 out of 38 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% File Formats: PARQUET/NONE:36 PARQUET/SNAPPY:2 BytesRead(500.000ms): 0 - FooterProcessingTime: (Avg: 640.416us ; Min: 439.688us ; Max: 843.296us ; Number of samples: 38) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 159.46 MB (167209012) - BytesReadDataNodeCache: 0 - BytesReadLocal: 159.46 MB (167209012) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 159.46 MB (167209012) - CachedFileHandlesHitCount: 64 (64) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 229.819ms - MaxCompressedTextFileLength: 0 - NumColumns: 5 (5) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 8 (8) - NumRowGroups: 38 (38) - NumScannerThreadsStarted: 38 (38) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 36 (36) - PeakMemoryUsage: 123.88 MB (129892983) - PerReadThreadRawHdfsThroughput: 1.60 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 339.475ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 13.74M (13736408) - RowsReturned: 185.91K (185905) - RowsReturnedRate: 542.74 K/sec - ScanRangesComplete: 38 (38) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 825.831ms - MaterializeTupleTime(*): 427.153ms - ScannerThreadsSysTime: 11.702ms - ScannerThreadsUserTime: 671.189ms - ScannerThreadsVoluntaryContextSwitches: 231 (231) - TotalRawHdfsOpenFileTime(*): 342.335us - TotalRawHdfsReadTime(*): 97.633ms - TotalReadThroughput: 0.00 /sec CodeGen:(Total: 105.406ms, non-child: 105.406ms, % non-child: 100.00%) - CodegenTime: 2.741ms - CompileTime: 17.917ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039864) - NumFunctions: 123 (123) - NumInstructions: 1.53K (1535) - OptimizationTime: 71.983ms - PeakMemoryUsage: 767.50 KB (785920) - PrepareTime: 14.965ms Instance 2145dc34def319e3:82987cee00000006 (host=vd1321.foo.com:22000):(Total: 20s919ms, non-child: 13s667ms, % non-child: 65.33%) Hdfs split stats (:<# splits>/): 0:6/1.48 GB 3:3/759.89 MB 7:6/1.48 GB 5:4/1013.36 MB 1:3/759.90 MB 6:7/1.73 GB 2:2/341.40 MB 4:7/1.73 GB Fragment Instance Lifecycle Event Timeline: 20s919ms - Prepare Finished: 1.079ms (1.079ms) - Open Finished: 7s240ms (7s239ms) - First Batch Produced: 7s240ms (11.434us) - First Batch Sent: 7s240ms (169.105us) - ExecInternal Finished: 20s919ms (13s679ms) MemoryUsage(500.000ms): 59.01 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB ThreadUsage(500.000ms): 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.02 - BloomFilterBytes: 0 - PeakMemoryUsage: 67.10 MB (70357381) - PeakReservation: 6.00 MB (6291456) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 67.10 MB (70357381) - RowsProduced: 92.28K (92275) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 13s667ms - TotalStorageWaitTime: 85.299ms - TotalThreadsInvoluntaryContextSwitches: 19 (19) - TotalThreadsTotalWallClockTime: 21s755ms - TotalThreadsSysTime: 24.480ms - TotalThreadsUserTime: 7s235ms - TotalThreadsVoluntaryContextSwitches: 376 (376) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 13s679ms - ExecTreeExecTime: 450.394us - OpenTime: 7s239ms - ExecTreeOpenTime: 7s133ms - PrepareTime: 802.236us - ExecTreePrepareTime: 327.712us DataStreamSender (dst_id=5):(Total: 13.110ms, non-child: 13.110ms, % non-child: 100.00%) - BytesSent: 1.96 MB (2053261) - NetworkThroughput(*): 1.05 GB/sec - OverallThroughput: 149.36 MB/sec - PeakMemoryUsage: 2.61 KB (2672) - RowsReturned: 92.28K (92275) - SerializeBatchTime: 11.179ms - TransmitDataRPCTime: 1.818ms - UncompressedRowBatchSize: 5.02 MB (5259675) SORT_NODE (id=1):(Total: 7s134ms, non-child: 6s753ms, % non-child: 94.66%) SortType: Total ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 92.28K (92275) ; Min: 92.28K (92275) ; Max: 92.28K (92275) ; Number of samples: 1) - InMemorySortTime: 6s744ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 6.07 MB (6369280) - RowsReturned: 92.28K (92275) - RowsReturnedRate: 12.93 K/sec - SortDataSize: 4.31 MB (4521475) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) Buffer pool: - AllocTime: 47.785us - CumulativeAllocationBytes: 6.00 MB (6291456) - CumulativeAllocations: 3 (3) - PeakReservation: 6.00 MB (6291456) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 6.00 MB (6291456) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns HDFS_SCAN_NODE (id=0):(Total: 380.887ms, non-child: 380.887ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:6/1.48 GB 3:3/759.89 MB 7:6/1.48 GB 5:4/1013.36 MB 1:3/759.90 MB 6:7/1.73 GB 2:2/341.40 MB 4:7/1.73 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 38 out of 38 Hdfs Read Thread Concurrency Bucket: 0:0% 1:100% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% File Formats: PARQUET/NONE:37 PARQUET/SNAPPY:1 BytesRead(500.000ms): 43.12 MB - FooterProcessingTime: (Avg: 2.050ms ; Min: 395.095us ; Max: 13.670ms ; Number of samples: 38) - AverageHdfsReadThreadConcurrency: 1.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 81.59 MB (85553427) - BytesReadDataNodeCache: 0 - BytesReadLocal: 81.59 MB (85553427) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 81.59 MB (85553427) - CachedFileHandlesHitCount: 51 (51) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 124.248ms - MaxCompressedTextFileLength: 0 - NumColumns: 5 (5) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 8 (8) - NumRowGroups: 38 (38) - NumScannerThreadsStarted: 38 (38) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 37 (37) - PeakMemoryUsage: 61.08 MB (64044338) - PerReadThreadRawHdfsThroughput: 1.60 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 377.571ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 6.87M (6868463) - RowsReturned: 92.28K (92275) - RowsReturnedRate: 242.26 K/sec - ScanRangesComplete: 38 (38) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 836.988ms - MaterializeTupleTime(*): 243.634ms - ScannerThreadsSysTime: 13.336ms - ScannerThreadsUserTime: 373.324ms - ScannerThreadsVoluntaryContextSwitches: 214 (214) - TotalRawHdfsOpenFileTime(*): 287.542us - TotalRawHdfsReadTime(*): 49.668ms - TotalReadThroughput: 86.24 MB/sec CodeGen:(Total: 104.229ms, non-child: 104.229ms, % non-child: 100.00%) - CodegenTime: 2.742ms - CompileTime: 16.997ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039864) - NumFunctions: 123 (123) - NumInstructions: 1.53K (1535) - OptimizationTime: 70.602ms - PeakMemoryUsage: 767.50 KB (785920) - PrepareTime: 16.053ms Instance 2145dc34def319e3:82987cee00000007 (host=vd1319.foo.com:22000):(Total: 16s818ms, non-child: 9s772ms, % non-child: 58.11%) Hdfs split stats (:<# splits>/): 2:1/253.37 MB 5:2/506.72 MB 0:4/1013.36 MB 3:6/1.48 GB 4:5/1.24 GB 6:3/760.01 MB 1:6/1.48 GB 7:11/2.55 GB Fragment Instance Lifecycle Event Timeline: 16s818ms - Prepare Finished: 725.281us (725.281us) - Open Finished: 7s035ms (7s035ms) - First Batch Produced: 7s035ms (11.093us) - First Batch Sent: 7s036ms (137.309us) - ExecInternal Finished: 16s818ms (9s782ms) MemoryUsage(500.000ms): 67.10 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB ThreadUsage(500.000ms): 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.03 - BloomFilterBytes: 0 - PeakMemoryUsage: 67.10 MB (70357222) - PeakReservation: 6.00 MB (6291456) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 67.10 MB (70357222) - RowsProduced: 92.91K (92905) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 9s772ms - TotalStorageWaitTime: 67.943ms - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 17s447ms - TotalThreadsSysTime: 15.216ms - TotalThreadsUserTime: 7s031ms - TotalThreadsVoluntaryContextSwitches: 363 (363) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 9s782ms - ExecTreeExecTime: 304.307us - OpenTime: 7s035ms - ExecTreeOpenTime: 6s945ms - PrepareTime: 535.096us - ExecTreePrepareTime: 205.061us DataStreamSender (dst_id=5):(Total: 11.152ms, non-child: 11.152ms, % non-child: 100.00%) - BytesSent: 1.97 MB (2067059) - NetworkThroughput(*): 1.13 GB/sec - OverallThroughput: 176.76 MB/sec - PeakMemoryUsage: 2.61 KB (2672) - RowsReturned: 92.91K (92905) - SerializeBatchTime: 9.348ms - TransmitDataRPCTime: 1.705ms - UncompressedRowBatchSize: 5.05 MB (5295585) SORT_NODE (id=1):(Total: 6s946ms, non-child: 6s610ms, % non-child: 95.17%) SortType: Total ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 92.91K (92905) ; Min: 92.91K (92905) ; Max: 92.91K (92905) ; Number of samples: 1) - InMemorySortTime: 6s602ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 6.07 MB (6369280) - RowsReturned: 92.91K (92905) - RowsReturnedRate: 13.38 K/sec - SortDataSize: 4.34 MB (4552345) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) Buffer pool: - AllocTime: 23.702us - CumulativeAllocationBytes: 6.00 MB (6291456) - CumulativeAllocations: 3 (3) - PeakReservation: 6.00 MB (6291456) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 6.00 MB (6291456) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns HDFS_SCAN_NODE (id=0):(Total: 335.570ms, non-child: 335.570ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:1/253.37 MB 5:2/506.72 MB 0:4/1013.36 MB 3:6/1.48 GB 4:5/1.24 GB 6:3/760.01 MB 1:6/1.48 GB 7:11/2.55 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 38 out of 38 Hdfs Read Thread Concurrency Bucket: 0:0% 1:100% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% File Formats: PARQUET/NONE:37 PARQUET/SNAPPY:1 BytesRead(500.000ms): 67.12 MB - FooterProcessingTime: (Avg: 1.555ms ; Min: 368.016us ; Max: 13.406ms ; Number of samples: 38) - AverageHdfsReadThreadConcurrency: 1.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 81.59 MB (85551853) - BytesReadDataNodeCache: 0 - BytesReadLocal: 81.59 MB (85551853) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 81.59 MB (85551853) - CachedFileHandlesHitCount: 51 (51) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 122.133ms - MaxCompressedTextFileLength: 0 - NumColumns: 5 (5) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 8 (8) - NumRowGroups: 38 (38) - NumScannerThreadsStarted: 38 (38) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 37 (37) - PeakMemoryUsage: 61.08 MB (64044179) - PerReadThreadRawHdfsThroughput: 1.59 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 333.293ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 6.87M (6868423) - RowsReturned: 92.91K (92905) - RowsReturnedRate: 276.86 K/sec - ScanRangesComplete: 38 (38) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 630.065ms - MaterializeTupleTime(*): 200.320ms - ScannerThreadsSysTime: 6.225ms - ScannerThreadsUserTime: 328.415ms - ScannerThreadsVoluntaryContextSwitches: 192 (192) - TotalRawHdfsOpenFileTime(*): 203.588us - TotalRawHdfsReadTime(*): 50.054ms - TotalReadThroughput: 134.24 MB/sec CodeGen:(Total: 88.672ms, non-child: 88.672ms, % non-child: 100.00%) - CodegenTime: 2.543ms - CompileTime: 13.939ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039864) - NumFunctions: 123 (123) - NumInstructions: 1.53K (1535) - OptimizationTime: 61.459ms - PeakMemoryUsage: 767.50 KB (785920) - PrepareTime: 12.870ms Instance 2145dc34def319e3:82987cee00000002 (host=vd1323.foo.com:22000):(Total: 6s777ms, non-child: 10.591ms, % non-child: 0.16%) Hdfs split stats (:<# splits>/): 1:3/760.07 MB 2:7/1.73 GB 0:3/331.17 MB 3:4/1013.24 MB 5:7/1.73 GB 7:8/1.98 GB 4:3/760.12 MB 6:4/1013.41 MB Fragment Instance Lifecycle Event Timeline: 6s777ms - Prepare Finished: 796.511us (796.511us) - Open Finished: 6s756ms (6s755ms) - First Batch Produced: 6s756ms (15.446us) - First Batch Sent: 6s756ms (187.438us) - ExecInternal Finished: 6s777ms (21.293ms) MemoryUsage(500.000ms): 67.01 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB, 6.08 MB ThreadUsage(500.000ms): 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.07 - BloomFilterBytes: 0 - PeakMemoryUsage: 67.19 MB (70451713) - PeakReservation: 6.00 MB (6291456) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 67.19 MB (70451713) - RowsProduced: 92.36K (92355) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 10.117ms - TotalStorageWaitTime: 34.616ms - TotalThreadsInvoluntaryContextSwitches: 20 (20) - TotalThreadsTotalWallClockTime: 7s263ms - TotalThreadsSysTime: 18.966ms - TotalThreadsUserTime: 6s754ms - TotalThreadsVoluntaryContextSwitches: 388 (388) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 21.501ms - ExecTreeExecTime: 427.684us - OpenTime: 6s755ms - ExecTreeOpenTime: 6s647ms - PrepareTime: 560.056us - ExecTreePrepareTime: 221.840us DataStreamSender (dst_id=5):(Total: 12.175ms, non-child: 12.175ms, % non-child: 100.00%) - BytesSent: 1.96 MB (2055413) - NetworkThroughput(*): 1.37 GB/sec - OverallThroughput: 160.99 MB/sec - PeakMemoryUsage: 2.61 KB (2672) - RowsReturned: 92.36K (92355) - SerializeBatchTime: 10.658ms - TransmitDataRPCTime: 1.401ms - UncompressedRowBatchSize: 5.02 MB (5264235) SORT_NODE (id=1):(Total: 6s648ms, non-child: 6s316ms, % non-child: 95.01%) SortType: Total ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 92.36K (92355) ; Min: 92.36K (92355) ; Max: 92.36K (92355) ; Number of samples: 1) - InMemorySortTime: 6s306ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 6.07 MB (6369280) - RowsReturned: 92.36K (92355) - RowsReturnedRate: 13.89 K/sec - SortDataSize: 4.32 MB (4525395) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) Buffer pool: - AllocTime: 37.056us - CumulativeAllocationBytes: 6.00 MB (6291456) - CumulativeAllocations: 3 (3) - PeakReservation: 6.00 MB (6291456) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 6.00 MB (6291456) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns HDFS_SCAN_NODE (id=0):(Total: 332.005ms, non-child: 332.005ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:3/760.07 MB 2:7/1.73 GB 0:3/331.17 MB 3:4/1013.24 MB 5:7/1.73 GB 7:8/1.98 GB 4:3/760.12 MB 6:4/1013.41 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 39 out of 39 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% File Formats: PARQUET/NONE:38 PARQUET/SNAPPY:1 BytesRead(500.000ms): 59.22 MB - FooterProcessingTime: (Avg: 674.377us ; Min: 361.898us ; Max: 1.178ms ; Number of samples: 39) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 81.69 MB (85658158) - BytesReadDataNodeCache: 0 - BytesReadLocal: 81.69 MB (85658158) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 81.69 MB (85658158) - CachedFileHandlesHitCount: 52 (52) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 118.799ms - MaxCompressedTextFileLength: 0 - NumColumns: 5 (5) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 8 (8) - NumRowGroups: 39 (39) - NumScannerThreadsStarted: 39 (39) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 38 (38) - PeakMemoryUsage: 61.17 MB (64138670) - PerReadThreadRawHdfsThroughput: 1.45 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 328.980ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 6.87M (6868554) - RowsReturned: 92.36K (92355) - RowsReturnedRate: 278.17 K/sec - ScanRangesComplete: 39 (39) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 487.185ms - MaterializeTupleTime(*): 199.492ms - ScannerThreadsSysTime: 11.609ms - ScannerThreadsUserTime: 324.250ms - ScannerThreadsVoluntaryContextSwitches: 218 (218) - TotalRawHdfsOpenFileTime(*): 449.248us - TotalRawHdfsReadTime(*): 55.164ms - TotalReadThroughput: 118.44 MB/sec CodeGen:(Total: 106.292ms, non-child: 106.292ms, % non-child: 100.00%) - CodegenTime: 3.026ms - CompileTime: 17.103ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039864) - NumFunctions: 123 (123) - NumInstructions: 1.53K (1535) - OptimizationTime: 73.456ms - PeakMemoryUsage: 767.50 KB (785920) - PrepareTime: 15.221ms Instance 2145dc34def319e3:82987cee00000001 (host=vd1317.foo.com:22000):(Total: 125.994ms, non-child: 1.855ms, % non-child: 1.47%) Hdfs split stats (:<# splits>/): 3:4/1013.32 MB 2:4/1013.52 MB 7:5/1.24 GB 1:1/253.38 MB 4:6/1.48 GB 5:5/1.24 GB 0:7/1.73 GB 6:5/1.24 GB Fragment Instance Lifecycle Event Timeline: 126.258ms - Prepare Finished: 1.078ms (1.078ms) - Open Finished: 126.188ms (125.110ms) - First Batch Produced: 126.190ms (2.037us) - First Batch Sent: 126.191ms (662.000ns) - ExecInternal Finished: 126.258ms (66.939us) - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 7.61 MB (7980115) - PeakReservation: 6.00 MB (6291456) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 24.59 MB (25783891) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 160.000ns - TotalStorageWaitTime: 15.736ms - TotalThreadsInvoluntaryContextSwitches: 1 (1) - TotalThreadsTotalWallClockTime: 199.320ms - TotalThreadsSysTime: 8.037ms - TotalThreadsUserTime: 121.429ms - TotalThreadsVoluntaryContextSwitches: 278 (278) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 71.898us - ExecTreeExecTime: 1.250us - OpenTime: 125.107ms - ExecTreeOpenTime: 3.489ms - PrepareTime: 770.534us - ExecTreePrepareTime: 294.290us DataStreamSender (dst_id=5):(Total: 86.961us, non-child: 86.961us, % non-child: 100.00%) - BytesSent: 0 - NetworkThroughput(*): 0.00 /sec - OverallThroughput: 0.00 /sec - PeakMemoryUsage: 2.61 KB (2672) - RowsReturned: 0 (0) - SerializeBatchTime: 0.000ns - TransmitDataRPCTime: 0.000ns - UncompressedRowBatchSize: 0 SORT_NODE (id=1):(Total: 3.783ms, non-child: 22.180us, % non-child: 0.59%) SortType: Total ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 0 (0) ; Min: 0 (0) ; Max: 0 (0) ; Number of samples: 1) - InMemorySortTime: 431.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 6.02 MB (6307840) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) Buffer pool: - AllocTime: 10.258us - CumulativeAllocationBytes: 2.00 MB (2097152) - CumulativeAllocations: 1 (1) - PeakReservation: 6.00 MB (6291456) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 2.00 MB (2097152) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns HDFS_SCAN_NODE (id=0):(Total: 3.761ms, non-child: 3.761ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:4/1013.32 MB 2:4/1013.52 MB 7:5/1.24 GB 1:1/253.38 MB 4:6/1.48 GB 5:5/1.24 GB 0:7/1.73 GB 6:5/1.24 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 37 out of 37 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% File Formats: PARQUET/NONE:37 - FooterProcessingTime: (Avg: 493.288us ; Min: 306.701us ; Max: 767.643us ; Number of samples: 37) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 3.61 MB (3788800) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.61 MB (3788800) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.61 MB (3788800) - CachedFileHandlesHitCount: 37 (37) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 5 (5) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 8 (8) - NumRowGroups: 37 (37) - NumScannerThreadsStarted: 37 (37) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 37 (37) - PeakMemoryUsage: 1.59 MB (1667072) - PerReadThreadRawHdfsThroughput: 495.08 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1.443ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 37 (37) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 74.199ms - MaterializeTupleTime(*): 2.329us - ScannerThreadsSysTime: 5.058ms - ScannerThreadsUserTime: 1.115ms - ScannerThreadsVoluntaryContextSwitches: 203 (203) - TotalRawHdfsOpenFileTime(*): 247.720us - TotalRawHdfsReadTime(*): 7.298ms - TotalReadThroughput: 0.00 /sec CodeGen:(Total: 120.268ms, non-child: 120.268ms, % non-child: 100.00%) - CodegenTime: 3.412ms - CompileTime: 18.047ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039864) - NumFunctions: 123 (123) - NumInstructions: 1.53K (1535) - OptimizationTime: 82.610ms - PeakMemoryUsage: 767.50 KB (785920) - PrepareTime: 19.054ms Instance 2145dc34def319e3:82987cee00000003 (host=vd1322.foo.com:22000):(Total: 111.521ms, non-child: 1.666ms, % non-child: 1.49%) Hdfs split stats (:<# splits>/): 0:2/506.78 MB 1:2/506.70 MB 2:5/1.24 GB 7:6/1.48 GB 5:8/1.98 GB 6:5/1.24 GB 3:4/1013.41 MB 4:6/1.33 GB Fragment Instance Lifecycle Event Timeline: 111.828ms - Prepare Finished: 995.778us (995.778us) - Open Finished: 111.711ms (110.715ms) - First Batch Produced: 111.713ms (1.741us) - First Batch Sent: 111.714ms (713.000ns) - ExecInternal Finished: 111.828ms (114.761us) MemoryUsage(500.000ms): 778.11 KB ThreadUsage(500.000ms): 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 7.74 MB (8115283) - PeakReservation: 6.00 MB (6291456) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 7.74 MB (8115283) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 146.000ns - TotalStorageWaitTime: 21.630ms - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 232.845ms - TotalThreadsSysTime: 10.580ms - TotalThreadsUserTime: 107.477ms - TotalThreadsVoluntaryContextSwitches: 277 (277) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 103.782us - ExecTreeExecTime: 1.267us - OpenTime: 110.728ms - ExecTreeOpenTime: 5.249ms - PrepareTime: 661.679us - ExecTreePrepareTime: 262.976us DataStreamSender (dst_id=5):(Total: 90.710us, non-child: 90.710us, % non-child: 100.00%) - BytesSent: 0 - NetworkThroughput(*): 0.00 /sec - OverallThroughput: 0.00 /sec - PeakMemoryUsage: 2.61 KB (2672) - RowsReturned: 0 (0) - SerializeBatchTime: 0.000ns - TransmitDataRPCTime: 0.000ns - UncompressedRowBatchSize: 0 SORT_NODE (id=1):(Total: 5.511ms, non-child: 232.739us, % non-child: 4.22%) SortType: Total ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 0 (0) ; Min: 0 (0) ; Max: 0 (0) ; Number of samples: 1) - InMemorySortTime: 560.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 6.02 MB (6307840) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) Buffer pool: - AllocTime: 9.649us - CumulativeAllocationBytes: 2.00 MB (2097152) - CumulativeAllocations: 1 (1) - PeakReservation: 6.00 MB (6291456) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 2.00 MB (2097152) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns HDFS_SCAN_NODE (id=0):(Total: 5.278ms, non-child: 5.278ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:2/506.78 MB 1:2/506.70 MB 2:5/1.24 GB 7:6/1.48 GB 5:8/1.98 GB 6:5/1.24 GB 3:4/1013.41 MB 4:6/1.33 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 38 out of 38 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% File Formats: PARQUET/NONE:38 BytesRead(500.000ms): 0 - FooterProcessingTime: (Avg: 666.065us ; Min: 476.677us ; Max: 1.132ms ; Number of samples: 38) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 3.71 MB (3891200) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.71 MB (3891200) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.71 MB (3891200) - CachedFileHandlesHitCount: 38 (38) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 5 (5) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 8 (8) - NumRowGroups: 38 (38) - NumScannerThreadsStarted: 38 (38) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 38 (38) - PeakMemoryUsage: 1.72 MB (1802240) - PerReadThreadRawHdfsThroughput: 371.70 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 2.583ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 38 (38) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 122.093ms - MaterializeTupleTime(*): 3.319us - ScannerThreadsSysTime: 7.625ms - ScannerThreadsUserTime: 2.904ms - ScannerThreadsVoluntaryContextSwitches: 203 (203) - TotalRawHdfsOpenFileTime(*): 161.315us - TotalRawHdfsReadTime(*): 9.983ms - TotalReadThroughput: 0.00 /sec CodeGen:(Total: 104.252ms, non-child: 104.252ms, % non-child: 100.00%) - CodegenTime: 3.077ms - CompileTime: 16.626ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039864) - NumFunctions: 123 (123) - NumInstructions: 1.53K (1535) - OptimizationTime: 71.993ms - PeakMemoryUsage: 767.50 KB (785920) - PrepareTime: 15.148ms