Query (id=954ce6d0fc8ed210:567f911e00000000): Summary: Session ID: 314eb11f8f411c1e:72175dbc55ef8f97 Session Type: BEESWAX Start Time: 2018-03-26 16:34:33.477021000 End Time: 2018-03-26 16:35:04.108783000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 3.0.0-SNAPSHOT RELEASE (build 1d0b618cac905c2e3af5cdb41d440b56b76f2b01) User: mmokhtar Connected User: mmokhtar Delegated User: Network Address: ::ffff:127.0.0.1:48357 Default Db: tpch_10_parquet Sql Statement: select PERCENTILE_CONT(0.99) WITHIN GROUP (ORDER BY l_orderkey) AS rank FROM lineitem Coordinator: mostafa-01:22000 Query Options (set by configuration): DISABLE_CODEGEN_ROWS_THRESHOLD=1 Query Options (set by configuration and planner): MT_DOP=0,DISABLE_CODEGEN_ROWS_THRESHOLD=1 Plan: ---------------- Max Per-Host Resource Reservation: Memory=14.00MB Per-Host Resource Estimates: Memory=138.00MB F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=18.00MB mem-reservation=8.00MB PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B | 04:AGGREGATE [FINALIZE] | output: _percentile_cont_interpolation(l_orderkey, row_number() - 1 - count(l_orderkey) - 1 * 0.99) | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB | tuple-ids=4 row-size=8B cardinality=1 | 03:ANALYTIC | functions: count(l_orderkey) | mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB | tuple-ids=8,6,7 row-size=24B cardinality=59986052 | 02:ANALYTIC | functions: row_number() | order by: l_orderkey ASC | window: ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW | mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB | tuple-ids=8,6 row-size=16B cardinality=59986052 | 05:MERGING-EXCHANGE [UNPARTITIONED] | order by: l_orderkey ASC NULLS LAST | mem-estimate=0B mem-reservation=0B | tuple-ids=8 row-size=8B cardinality=59986052 | F00:PLAN FRAGMENT [RANDOM] hosts=1 instances=1 Per-Host Resources: mem-estimate=120.00MB mem-reservation=6.00MB 01:SORT | order by: l_orderkey ASC NULLS LAST | mem-estimate=32.00MB mem-reservation=6.00MB spill-buffer=2.00MB | tuple-ids=8 row-size=8B cardinality=59986052 | 00:SCAN HDFS [tpch_10_parquet.lineitem, RANDOM] partitions=1/1 files=15 size=2.05GB stored statistics: table: rows=59986052 size=2.05GB columns: all extrapolated-rows=disabled mem-estimate=88.00MB mem-reservation=0B tuple-ids=0 row-size=8B cardinality=59986052 ---------------- Estimated Per-Host Mem: 144703488 Per Host Min Reservation: mostafa-01:22000(14.00 MB) Request Pool: default-pool Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail --------------------------------------------------------------------------------------------------------------------------- 04:AGGREGATE 1 6s346ms 6s346ms 1 1 36.00 KB 10.00 MB FINALIZE 03:ANALYTIC 1 3s119ms 3s119ms 59.99M 59.99M 1.01 GB 4.00 MB 02:ANALYTIC 1 4s959ms 4s959ms 59.99M 59.99M 14.05 MB 4.00 MB 05:MERGING-EXCHANGE 1 892.850ms 892.850ms 59.99M 59.99M 14.19 MB 0 UNPARTITIONED 01:SORT 1 15s053ms 15s053ms 59.99M 59.99M 516.02 MB 32.00 MB 00:SCAN HDFS 1 105.044ms 105.044ms 59.99M 59.99M 68.16 MB 88.00 MB tpch_10_parquet.lineitem Errors: Query Compilation: 5.692ms - Metadata of all 1 tables cached: 278.060us (278.060us) - Analysis finished: 2.444ms (2.166ms) - Value transfer graph computed: 2.669ms (224.436us) - Single node plan created: 4.031ms (1.362ms) - Runtime filters computed: 4.125ms (93.713us) - Distributed plan created: 4.167ms (41.906us) - Planning finished: 5.692ms (1.525ms) Query Timeline: 30s632ms - Query submitted: 45.519us (45.519us) - Planning finished: 7.140ms (7.095ms) - Submit for admission: 7.255ms (115.177us) - Completed admission: 7.278ms (22.717us) - Ready to start on 1 backends: 7.365ms (86.716us) - All 1 execution backends (2 fragment instances) started: 8.050ms (685.551us) - Rows available: 30s351ms (30s343ms) - First row fetched: 30s630ms (278.830ms) - Last row fetched: 30s630ms (103.897us) - Released admission control resources: 30s630ms (527.831us) - Unregister query: 30s631ms (902.043us) - ComputeScanRangeAssignmentTimer: 21.667us ImpalaServer: - ClientFetchWaitTimer: 279.372ms - RowMaterializationTimer: 980.218us Execution Profile 954ce6d0fc8ed210:567f911e00000000:(Total: 30s344ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- Backend startup latencies: Count: 1, min / max: 1ms / 1ms, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 1ms Per Node Peak Memory Usage: mostafa-01:22000(1.03 GB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns - NumBackends: 1 (1) - NumFragmentInstances: 2 (2) - NumFragments: 2 (2) Averaged Fragment F01:(Total: 30s622ms, non-child: 282.352ms, % non-child: 0.92%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:30s622ms max:30s622ms mean: 30s622ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.03 GB (1109756644) - PeakReservation: 1.01 GB (1084227584) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 1.03 GB (1109756644) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 2.23K (2226) - TotalThreadsTotalWallClockTime: 30s621ms - TotalThreadsSysTime: 407.080ms - TotalThreadsUserTime: 15s002ms - TotalThreadsVoluntaryContextSwitches: 16 (16) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 278.911ms - ExecTreeExecTime: 3.299us - OpenTime: 30s342ms - ExecTreeOpenTime: 30s224ms - PrepareTime: 379.641us - ExecTreePrepareTime: 120.758us PLAN_ROOT_SINK: - PeakMemoryUsage: 0 AGGREGATION_NODE (id=4):(Total: 30s225ms, non-child: 6s346ms, % non-child: 21.00%) - BuildTime: 6s333ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 36.00 KB (36864) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) ANALYTIC_EVAL_NODE (id=3):(Total: 23s878ms, non-child: 3s119ms, % non-child: 13.06%) - EvaluationTime: 3s041ms - PeakMemoryUsage: 1.01 GB (1089507328) - RowsReturned: 59.99M (59986052) - RowsReturnedRate: 2.51 M/sec ANALYTIC_EVAL_NODE (id=2):(Total: 20s759ms, non-child: 4s959ms, % non-child: 23.89%) - EvaluationTime: 4s928ms - PeakMemoryUsage: 14.05 MB (14737408) - RowsReturned: 59.99M (59986052) - RowsReturnedRate: 2.89 M/sec EXCHANGE_NODE (id=5):(Total: 15s800ms, non-child: 15s800ms, % non-child: 100.00%) - ConvertRowBatchTime: 0.000ns - MergeGetNext: 883.827ms - MergeGetNextBatch: 112.349ms - PeakMemoryUsage: 14.19 MB (14874617) - RowsReturned: 59.99M (59986052) - RowsReturnedRate: 3.80 M/sec Buffer pool: - AllocTime: 50.679ms - CumulativeAllocationBytes: 1.34 GB (1440374784) - CumulativeAllocations: 117.39K (117390) - PeakReservation: 14.18 MB (14868480) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 14.18 MB (14868480) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: - FirstBatchArrivalWaitTime: 14s907ms - TotalBytesReceived: 972.52 MB (1019762884) - TotalGetBatchTime: 15s013ms - DataArrivalTimer: 14s907ms SenderSide: - DeserializeRowBatchTime: 496.205ms - NumBatchesArrived: 58.70K (58695) - NumBatchesDeferred: 55.97K (55965) - NumBatchesEnqueued: 58.70K (58695) - NumBatchesReceived: 58.70K (58695) - NumEarlySenders: 0 (0) - NumEosReceived: 1 (1) Buffer pool: - AllocTime: 10.732ms - CumulativeAllocationBytes: 516.00 MB (541065216) - CumulativeAllocations: 258 (258) - PeakReservation: 4.00 MB (4194304) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 4.00 MB (4194304) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Buffer pool: - AllocTime: 7.447ms - CumulativeAllocationBytes: 1.01 GB (1080033280) - CumulativeAllocations: 515 (515) - PeakReservation: 1.01 GB (1080033280) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 1.01 GB (1080033280) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns CodeGen:(Total: 114.769ms, non-child: 114.769ms, % non-child: 100.00%) - CodegenTime: 20.115ms - CompileTime: 15.832ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2041744) - NumFunctions: 222 (222) - NumInstructions: 8.17K (8167) - OptimizationTime: 74.142ms - PeakMemoryUsage: 3.99 MB (4181504) - PrepareTime: 24.315ms Coordinator Fragment F01: Instance 954ce6d0fc8ed210:567f911e00000000 (host=mostafa-01:22000):(Total: 30s622ms, non-child: 282.352ms, % non-child: 0.92%) Fragment Instance Lifecycle Event Timeline: 30s622ms - Prepare Finished: 510.042us (510.042us) - Open Finished: 30s343ms (30s342ms) - First Batch Produced: 30s343ms (3.762us) - First Batch Sent: 30s622ms (278.887ms) - ExecInternal Finished: 30s622ms (18.386us) MemoryUsage(500.000ms): 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 22.72 KB, 47.41 MB, 106.28 MB, 171.29 MB, 235.28 MB, 297.28 MB, 360.28 MB, 422.29 MB, 487.28 MB, 551.29 MB, 613.28 MB, 674.28 MB, 738.29 MB, 799.28 MB, 857.29 MB, 917.28 MB, 980.29 MB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.03 GB (1109756644) - PeakReservation: 1.01 GB (1084227584) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 1.03 GB (1109756644) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 2.23K (2226) - TotalThreadsTotalWallClockTime: 30s621ms - TotalThreadsSysTime: 407.080ms - TotalThreadsUserTime: 15s002ms - TotalThreadsVoluntaryContextSwitches: 16 (16) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 278.911ms - ExecTreeExecTime: 3.299us - OpenTime: 30s342ms - ExecTreeOpenTime: 30s224ms - PrepareTime: 379.641us - ExecTreePrepareTime: 120.758us PLAN_ROOT_SINK: - PeakMemoryUsage: 0 AGGREGATION_NODE (id=4):(Total: 30s225ms, non-child: 6s346ms, % non-child: 21.00%) ExecOption: Codegen Enabled - BuildTime: 6s333ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 36.00 KB (36864) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) ANALYTIC_EVAL_NODE (id=3):(Total: 23s878ms, non-child: 3s119ms, % non-child: 13.06%) - EvaluationTime: 3s041ms - PeakMemoryUsage: 1.01 GB (1089507328) - RowsReturned: 59.99M (59986052) - RowsReturnedRate: 2.51 M/sec ANALYTIC_EVAL_NODE (id=2):(Total: 20s759ms, non-child: 4s959ms, % non-child: 23.89%) - EvaluationTime: 4s928ms - PeakMemoryUsage: 14.05 MB (14737408) - RowsReturned: 59.99M (59986052) - RowsReturnedRate: 2.89 M/sec EXCHANGE_NODE (id=5):(Total: 15s800ms, non-child: 892.850ms, % non-child: 5.65%) ExecOption: Codegen Enabled - ConvertRowBatchTime: 0.000ns - MergeGetNext: 883.827ms - MergeGetNextBatch: 112.349ms - PeakMemoryUsage: 14.19 MB (14874617) - RowsReturned: 59.99M (59986052) - RowsReturnedRate: 3.80 M/sec Buffer pool: - AllocTime: 50.679ms - CumulativeAllocationBytes: 1.34 GB (1440374784) - CumulativeAllocations: 117.39K (117390) - PeakReservation: 14.18 MB (14868480) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 14.18 MB (14868480) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 35.63 MB, 89.23 MB, 145.86 MB, 205.68 MB, 264.86 MB, 324.03 MB, 383.51 MB, 443.49 MB, 503.14 MB, 562.66 MB, 621.88 MB, 681.14 MB, 739.92 MB, 797.54 MB, 854.99 MB, 910.64 MB, 968.68 MB, 972.52 MB, 972.52 MB, 972.52 MB, 972.52 MB, 972.52 MB, 972.52 MB, 972.52 MB, 972.52 MB, 972.52 MB, 972.52 MB, 972.52 MB, 972.52 MB, 972.52 MB, 972.52 MB - FirstBatchArrivalWaitTime: 14s907ms - TotalBytesReceived: 972.52 MB (1019762884) - TotalGetBatchTime: 15s013ms - DataArrivalTimer: 14s907ms SenderSide: - DeserializeRowBatchTime: 496.205ms - NumBatchesArrived: 58.70K (58695) - NumBatchesDeferred: 55.97K (55965) - NumBatchesEnqueued: 58.70K (58695) - NumBatchesReceived: 58.70K (58695) - NumEarlySenders: 0 (0) - NumEosReceived: 1 (1) Buffer pool: - AllocTime: 10.732ms - CumulativeAllocationBytes: 516.00 MB (541065216) - CumulativeAllocations: 258 (258) - PeakReservation: 4.00 MB (4194304) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 4.00 MB (4194304) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Buffer pool: - AllocTime: 7.447ms - CumulativeAllocationBytes: 1.01 GB (1080033280) - CumulativeAllocations: 515 (515) - PeakReservation: 1.01 GB (1080033280) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 1.01 GB (1080033280) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns CodeGen:(Total: 114.769ms, non-child: 114.769ms, % non-child: 100.00%) - CodegenTime: 20.115ms - CompileTime: 15.832ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2041744) - NumFunctions: 222 (222) - NumInstructions: 8.17K (8167) - OptimizationTime: 74.142ms - PeakMemoryUsage: 3.99 MB (4181504) - PrepareTime: 24.315ms Averaged Fragment F00:(Total: 23s322ms, non-child: 6s603ms, % non-child: 28.31%) split sizes: min: 2.05 GB, max: 2.05 GB, avg: 2.05 GB, stddev: 0 completion times: min:23s322ms max:23s322ms mean: 23s322ms stddev:0.000ns execution rates: min:90.13 MB/sec max:90.13 MB/sec mean:90.13 MB/sec stddev:0.94 B/sec num instances: 1 - AverageThreadTokens: 1.32 - BloomFilterBytes: 0 - PeakMemoryUsage: 528.42 MB (554090058) - PeakReservation: 516.00 MB (541065216) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 1.03 GB (1105599579) - RowsProduced: 59.99M (59986052) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6s313ms - TotalStorageWaitTime: 176.266ms - TotalThreadsInvoluntaryContextSwitches: 4.29K (4295) - TotalThreadsTotalWallClockTime: 31s025ms - TotalThreadsSysTime: 367.349ms - TotalThreadsUserTime: 17s488ms - TotalThreadsVoluntaryContextSwitches: 116.77K (116767) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 8s296ms - ExecTreeExecTime: 181.665ms - OpenTime: 15s025ms - ExecTreeOpenTime: 14s992ms - PrepareTime: 546.356us - ExecTreePrepareTime: 133.181us KrpcDataStreamSender (dst_id=5):(Total: 1s527ms, non-child: 1s527ms, % non-child: 100.00%) - BytesSent: 327.04 MB (342922842) - EosSent: 1 (1) - OverallThroughput: 214.07 MB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 59.99M (59986052) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 1s523ms - UncompressedRowBatchSize: 972.52 MB (1019762884) SORT_NODE (id=1):(Total: 15s158ms, non-child: 15s053ms, % non-child: 99.31%) - InMemorySortTime: 13s916ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 516.02 MB (541082624) - RowsReturned: 59.99M (59986052) - RowsReturnedRate: 3.96 M/sec - SortDataSize: 514.86 MB (539874468) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) Buffer pool: - AllocTime: 3.662ms - CumulativeAllocationBytes: 516.00 MB (541065216) - CumulativeAllocations: 258 (258) - PeakReservation: 516.00 MB (541065216) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 516.00 MB (541065216) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns HDFS_SCAN_NODE (id=0):(Total: 105.044ms, non-child: 105.044ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 7.50 - BytesRead: 93.01 MB (97524845) - BytesReadDataNodeCache: 0 - BytesReadLocal: 93.01 MB (97524845) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 93.01 MB (97524845) - CachedFileHandlesHitCount: 30 (30) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 230.566ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 15 (15) - NumScannerThreadsStarted: 8 (8) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 68.16 MB (71470511) - PerReadThreadRawHdfsThroughput: 585.87 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 17.136ms - RowBatchQueuePutWaitTime: 6s961ms - RowsRead: 59.99M (59986052) - RowsReturned: 59.99M (59986052) - RowsReturnedRate: 571.06 M/sec - ScanRangesComplete: 15 (15) - ScannerThreadsInvoluntaryContextSwitches: 175 (175) - ScannerThreadsTotalWallClockTime: 7s704ms - MaterializeTupleTime(*): 240.483ms - ScannerThreadsSysTime: 82.084ms - ScannerThreadsUserTime: 614.891ms - ScannerThreadsVoluntaryContextSwitches: 58.31K (58315) - TotalRawHdfsOpenFileTime(*): 100.216us - TotalRawHdfsReadTime(*): 158.749ms - TotalReadThroughput: 93.01 MB/sec CodeGen:(Total: 32.283ms, non-child: 32.283ms, % non-child: 100.00%) - CodegenTime: 1.270ms - CompileTime: 5.175ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2041744) - NumFunctions: 35 (35) - NumInstructions: 553 (553) - OptimizationTime: 10.062ms - PeakMemoryUsage: 276.50 KB (283136) - PrepareTime: 16.578ms Fragment F00: Instance 954ce6d0fc8ed210:567f911e00000001 (host=mostafa-01:22000):(Total: 23s322ms, non-child: 6s603ms, % non-child: 28.31%) Hdfs split stats (:<# splits>/): 0:15/2.05 GB Fragment Instance Lifecycle Event Timeline: 23s322ms - Prepare Finished: 750.462us (750.462us) - Open Finished: 15s025ms (15s025ms) - First Batch Produced: 15s025ms (6.354us) - First Batch Sent: 15s025ms (51.974us) - ExecInternal Finished: 23s322ms (8s296ms) MemoryUsage(500.000ms): 185.97 MB, 407.52 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 516.03 MB, 498.03 MB, 470.03 MB, 440.03 MB, 408.03 MB, 376.03 MB, 346.03 MB, 314.03 MB, 282.03 MB, 250.03 MB, 220.03 MB, 188.03 MB, 156.03 MB, 126.03 MB, 94.03 MB, 64.03 MB, 34.03 MB, 6.03 MB ThreadUsage(500.000ms): 9, 8, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.32 - BloomFilterBytes: 0 - PeakMemoryUsage: 528.42 MB (554090058) - PeakReservation: 516.00 MB (541065216) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 1.03 GB (1105599579) - RowsProduced: 59.99M (59986052) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6s313ms - TotalStorageWaitTime: 176.266ms - TotalThreadsInvoluntaryContextSwitches: 4.29K (4295) - TotalThreadsTotalWallClockTime: 31s025ms - TotalThreadsSysTime: 367.349ms - TotalThreadsUserTime: 17s488ms - TotalThreadsVoluntaryContextSwitches: 116.77K (116767) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 8s296ms - ExecTreeExecTime: 181.665ms - OpenTime: 15s025ms - ExecTreeOpenTime: 14s992ms - PrepareTime: 546.356us - ExecTreePrepareTime: 133.181us KrpcDataStreamSender (dst_id=5):(Total: 1s527ms, non-child: 1s527ms, % non-child: 100.00%) - BytesSent: 327.04 MB (342922842) - EosSent: 1 (1) - OverallThroughput: 214.07 MB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 59.99M (59986052) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 1s523ms - UncompressedRowBatchSize: 972.52 MB (1019762884) SORT_NODE (id=1):(Total: 15s158ms, non-child: 15s053ms, % non-child: 99.31%) SortType: Total ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 59.99M (59986052) ; Min: 59.99M (59986052) ; Max: 59.99M (59986052) ; Number of samples: 1) - InMemorySortTime: 13s916ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 516.02 MB (541082624) - RowsReturned: 59.99M (59986052) - RowsReturnedRate: 3.96 M/sec - SortDataSize: 514.86 MB (539874468) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) Buffer pool: - AllocTime: 3.662ms - CumulativeAllocationBytes: 516.00 MB (541065216) - CumulativeAllocations: 258 (258) - PeakReservation: 516.00 MB (541065216) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 516.00 MB (541065216) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns HDFS_SCAN_NODE (id=0):(Total: 105.044ms, non-child: 105.044ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:15/2.05 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 15 out of 15 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:15 BytesRead(500.000ms): 50.68 MB, 93.01 MB - FooterProcessingTime: (Avg: 5.295ms ; Min: 843.239us ; Max: 19.959ms ; Number of samples: 15) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 7.50 - BytesRead: 93.01 MB (97524845) - BytesReadDataNodeCache: 0 - BytesReadLocal: 93.01 MB (97524845) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 93.01 MB (97524845) - CachedFileHandlesHitCount: 30 (30) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 230.566ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 15 (15) - NumScannerThreadsStarted: 8 (8) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 68.16 MB (71470511) - PerReadThreadRawHdfsThroughput: 585.87 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 17.136ms - RowBatchQueuePutWaitTime: 6s961ms - RowsRead: 59.99M (59986052) - RowsReturned: 59.99M (59986052) - RowsReturnedRate: 571.06 M/sec - ScanRangesComplete: 15 (15) - ScannerThreadsInvoluntaryContextSwitches: 175 (175) - ScannerThreadsTotalWallClockTime: 7s704ms - MaterializeTupleTime(*): 240.483ms - ScannerThreadsSysTime: 82.084ms - ScannerThreadsUserTime: 614.891ms - ScannerThreadsVoluntaryContextSwitches: 58.31K (58315) - TotalRawHdfsOpenFileTime(*): 100.216us - TotalRawHdfsReadTime(*): 158.749ms - TotalReadThroughput: 93.01 MB/sec CodeGen:(Total: 32.283ms, non-child: 32.283ms, % non-child: 100.00%) - CodegenTime: 1.270ms - CompileTime: 5.175ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2041744) - NumFunctions: 35 (35) - NumInstructions: 553 (553) - OptimizationTime: 10.062ms - PeakMemoryUsage: 276.50 KB (283136) - PrepareTime: 16.578ms