Query (id=3b4691e0b8ed45f6:6edea28100000000): Summary: Session ID: 554d5258e09aae31:ef3c050acfcad4aa Session Type: BEESWAX Start Time: 2018-01-03 15:01:55.158523000 End Time: 2018-01-03 15:02:07.751324000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.9.0-cdh5.12.0 RELEASE (build 03c6ddbdcec39238be4f5b14a300d5c4f576097e) User: mmokhtar Connected User: mmokhtar Delegated User: Network Address: ::ffff:10.20.122.102:49720 Default Db: test_db Sql Statement: select count(*),max(hour + nb_display) from bi_arbitrage_full Coordinator: d2402.foo.com:22000 Query Options (non default): Plan: ---------------- Per-Host Resource Reservation: Memory=0B Per-Host Resource Estimates: Memory=42.00MB WARNING: The following tables are missing relevant table and/or column statistics. test_db.bi_arbitrage_full F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B | 03:AGGREGATE [FINALIZE] | output: count:merge(*), max:merge(hour + nb_display) | mem-estimate=10.00MB mem-reservation=0B | tuple-ids=1 row-size=16B cardinality=1 | 02:EXCHANGE [UNPARTITIONED] | mem-estimate=0B mem-reservation=0B | tuple-ids=1 row-size=16B cardinality=1 | F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=10 01:AGGREGATE | output: count(*), max(hour + nb_display) | mem-estimate=10.00MB mem-reservation=0B | tuple-ids=1 row-size=16B cardinality=1 | 00:SCAN HDFS [ test_db.bi_arbitrage_full, RANDOM] partitions=186/186 files=136550 size=14.97GB table stats: unavailable column stats: unavailable mem-estimate=32.00MB mem-reservation=0B tuple-ids=0 row-size=8B cardinality=unavailable ---------------- Estimated Per-Host Mem: 44040192 Per-Host Memory Reservation: 0 Tables Missing Stats: test_db.bi_arbitrage_full Request Pool: root.mmokhtar Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ------------------------------------------------------------------------------------------------------------------- 03:AGGREGATE 1 385.681us 385.681us 1 1 56.00 KB 10.00 MB FINALIZE 02:EXCHANGE 1 144.608us 144.608us 10 1 0 0 UNPARTITIONED 01:AGGREGATE 10 1s310ms 1s382ms 10 1 185.25 KB 10.00 MB 00:SCAN HDFS 10 9s890ms 10s291ms 1.89B -1 27.77 MB 32.00 MB test_db.bi_arbitrage_full Errors: Planner Timeline: 182.410ms - Analysis finished: 6.026ms (6.026ms) - Equivalence classes computed: 6.423ms (396.947us) - Single node plan created: 173.305ms (166.882ms) - Runtime filters computed: 173.357ms (51.141us) - Distributed plan created: 174.227ms (869.873us) - Lineage info computed: 174.404ms (177.413us) - Planning finished: 182.410ms (8.006ms) Query Timeline: 12s596ms - Query submitted: 37.906us (37.906us) - Planning finished: 552.943ms (552.906ms) - Submit for admission: 884.391ms (331.447ms) - Completed admission: 884.785ms (394.049us) - Ready to start on 10 backends: 928.705ms (43.919ms) - All 10 execution backends (11 fragment instances) started: 962.752ms (34.047ms) - Rows available: 12s572ms (11s609ms) - First row fetched: 12s584ms (12.501ms) - Unregister query: 12s592ms (8.130ms) - ComputeScanRangeAssignmentTimer: 214.484ms ImpalaServer: - ClientFetchWaitTimer: 13.942ms - RowMaterializationTimer: 6.669ms Execution Profile 3b4691e0b8ed45f6:6edea28100000000:(Total: 11s639ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- Backend startup latencies: Count: 10, min / max: 17ms / 29ms, 25th %-ile: 18ms, 50th %-ile: 19ms, 75th %-ile: 21ms, 90th %-ile: 23ms, 95th %-ile: 29ms, 99.9th %-ile: 29ms Per Node Peak Memory Usage: d2402.foo.com:22000(20.10 MB) d2403.foo.com:22000(25.85 MB) d2404.foo.com:22000(18.59 MB) d2405.foo.com:22000(27.81 MB) d2406.foo.com:22000(19.82 MB) d2407.foo.com:22000(22.71 MB) d2408.foo.com:22000(20.83 MB) d2409.foo.com:22000(22.64 MB) d2410.foo.com:22000(22.35 MB) d2411.foo.com:22000(27.39 MB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Averaged Fragment F01:(Total: 11s622ms, non-child: 13.022ms, % non-child: 0.11%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:11s624ms max:11s624ms mean: 11s624ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 64.02 KB (65552) - PerHostPeakMemUsage: 20.10 MB (21075787) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 11s587ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 2 (2) - TotalThreadsTotalWallClockTime: 11s600ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 589.000us - TotalThreadsVoluntaryContextSwitches: 13 (13) Fragment Instance Lifecycle Timings: - ExecTime: 12.604ms - ExecTreeExecTime: 18.142us - OpenTime: 11s587ms - ExecTreeOpenTime: 11s587ms - PrepareTime: 22.342ms - ExecTreePrepareTime: 86.012us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 216.00 GB (231928233984) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 21.884ms, non-child: 21.884ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077532) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 21.244ms AGGREGATION_NODE (id=3):(Total: 11s587ms, non-child: 385.681us, % non-child: 0.00%) - BuildTime: 30.785us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 56.00 KB (57344) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=2):(Total: 11s587ms, non-child: 11s587ms, % non-child: 100.00%) - BytesReceived: 250.00 B (250) - ConvertRowBatchTime: 29.957us - DeserializeRowBatchTimer: 107.267us - FirstBatchArrivalWaitTime: 10s243ms - PeakMemoryUsage: 0 - RowsReturned: 10 (10) - RowsReturnedRate: 0 - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Coordinator Fragment F01: Instance 3b4691e0b8ed45f6:6edea28100000000 (host=d2402.foo.com:22000):(Total: 11s622ms, non-child: 13.022ms, % non-child: 0.11%) MemoryUsage(500.000ms): 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 28.01 KB, 32.01 KB, 48.01 KB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 64.02 KB (65552) - PerHostPeakMemUsage: 20.10 MB (21075787) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 11s587ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 2 (2) - TotalThreadsTotalWallClockTime: 11s600ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 589.000us - TotalThreadsVoluntaryContextSwitches: 13 (13) Fragment Instance Lifecycle Timings: - ExecTime: 12.604ms - ExecTreeExecTime: 18.142us - OpenTime: 11s587ms - ExecTreeOpenTime: 11s587ms - PrepareTime: 22.342ms - ExecTreePrepareTime: 86.012us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 216.00 GB (231928233984) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 21.884ms, non-child: 21.884ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077532) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 21.244ms AGGREGATION_NODE (id=3):(Total: 11s587ms, non-child: 385.681us, % non-child: 0.00%) - BuildTime: 30.785us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 56.00 KB (57344) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=2):(Total: 11s587ms, non-child: 144.608us, % non-child: 0.00%) BytesReceived(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 25.00 B, 50.00 B, 150.00 B - BytesReceived: 250.00 B (250) - ConvertRowBatchTime: 29.957us - DeserializeRowBatchTimer: 107.267us - FirstBatchArrivalWaitTime: 10s243ms - PeakMemoryUsage: 0 - RowsReturned: 10 (10) - RowsReturnedRate: 0 - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 11s256ms, non-child: 2.757ms, % non-child: 0.02%) split sizes: min: 1.50 GB, max: 1.50 GB, avg: 1.50 GB, stddev: 1.13 MB completion times: min:10s275ms max:11s611ms mean: 11s257ms stddev:391.582ms execution rates: min:131.95 MB/sec max:149.11 MB/sec mean:136.35 MB/sec stddev:4.99 MB/sec num instances: 10 - AverageThreadTokens: 24.90 - BloomFilterBytes: 0 - PeakMemoryUsage: 22.81 MB (23913305) - PerHostPeakMemUsage: 22.81 MB (23915763) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 248.444us - TotalStorageWaitTime: 4m19s - TotalThreadsInvoluntaryContextSwitches: 1.47K (1467) - TotalThreadsTotalWallClockTime: 4m38s - TotalThreadsSysTime: 1s068ms - TotalThreadsUserTime: 8s236ms - TotalThreadsVoluntaryContextSwitches: 209.43K (209429) Fragment Instance Lifecycle Timings: - ExecTime: 399.760us - ExecTreeExecTime: 6.871us - OpenTime: 11s196ms - ExecTreeOpenTime: 11s164ms - PrepareTime: 59.510ms - ExecTreePrepareTime: 36.589ms DataStreamSender (dst_id=2):(Total: 54.429us, non-child: 54.429us, % non-child: 100.00%) - BytesSent: 25.00 B (25) - NetworkThroughput(*): 941.04 KB/sec - OverallThroughput: 470.14 KB/sec - PeakMemoryUsage: 7.52 KB (7704) - RowsReturned: 1 (1) - SerializeBatchTime: 13.251us - TransmitDataRPCTime: 28.342us - UncompressedRowBatchSize: 25.00 B (25) CodeGen:(Total: 52.390ms, non-child: 52.390ms, % non-child: 100.00%) - CodegenTime: 1.885ms - CompileTime: 10.753ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077532) - NumFunctions: 22 (22) - NumInstructions: 387 (387) - OptimizationTime: 20.489ms - PeakMemoryUsage: 193.50 KB (198144) - PrepareTime: 20.588ms AGGREGATION_NODE (id=1):(Total: 11s201ms, non-child: 1s310ms, % non-child: 11.70%) - BuildTime: 1s196ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 178.95 KB (183244) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 9s890ms, non-child: 9s890ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.33 - AverageScannerThreadConcurrency: 24.00 - BytesRead: 1.32 GB (1413061946) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 336.297ms - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 13.65K (13655) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 22.77 MB (23873430) - PerReadThreadRawHdfsThroughput: 369.34 MB/sec - RemoteScanRanges: 40.97K (40965) - RowBatchQueueGetWaitTime: 9s263ms - RowBatchQueuePutWaitTime: 12.168ms - RowsRead: 189.00M (188999351) - RowsReturned: 189.00M (188999351) - RowsReturnedRate: 19.16 M/sec - ScanRangesComplete: 13.65K (13655) - ScannerThreadsInvoluntaryContextSwitches: 1.11K (1113) - ScannerThreadsTotalWallClockTime: 4m27s - MaterializeTupleTime(*): 3s176ms - ScannerThreadsSysTime: 798.659ms - ScannerThreadsUserTime: 6s142ms - ScannerThreadsVoluntaryContextSwitches: 78.67K (78670) - TotalRawHdfsReadTime(*): 3s653ms - TotalReadThroughput: 116.36 MB/sec BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 216.00 GB (231928233984) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns Fragment F00: Instance 3b4691e0b8ed45f6:6edea28100000003 (host=d2402.foo.com:22000):(Total: 11s609ms, non-child: 3.014ms, % non-child: 0.03%) Hdfs split stats (:<# splits>/): -1:14594/1.50 GB MemoryUsage(500.000ms): 4.79 MB, 367.24 KB, 9.14 MB, 8.95 MB, 4.95 MB, 4.53 MB, 4.67 MB, 311.22 KB, 323.23 KB, 4.60 MB, 5.14 MB, 9.02 MB, 4.65 MB, 4.77 MB, 360.23 KB, 9.02 MB, 363.27 KB, 357.24 KB, 263.28 KB, 4.76 MB, 249.27 KB, 4.43 MB, 4.57 MB ThreadUsage(500.000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25 - AverageThreadTokens: 25.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 20.08 MB (21059403) - PerHostPeakMemUsage: 20.10 MB (21075787) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 218.676us - TotalStorageWaitTime: 4m27s - TotalThreadsInvoluntaryContextSwitches: 1.41K (1412) - TotalThreadsTotalWallClockTime: 4m47s - TotalThreadsSysTime: 1s062ms - TotalThreadsUserTime: 8s305ms - TotalThreadsVoluntaryContextSwitches: 218.89K (218887) Fragment Instance Lifecycle Timings: - ExecTime: 343.251us - ExecTreeExecTime: 5.986us - OpenTime: 11s543ms - ExecTreeOpenTime: 11s508ms - PrepareTime: 65.218ms - ExecTreePrepareTime: 41.880ms DataStreamSender (dst_id=2):(Total: 67.388us, non-child: 67.388us, % non-child: 100.00%) - BytesSent: 25.00 B (25) - NetworkThroughput(*): 656.52 KB/sec - OverallThroughput: 362.29 KB/sec - PeakMemoryUsage: 7.52 KB (7704) - RowsReturned: 1 (1) - SerializeBatchTime: 11.938us - TransmitDataRPCTime: 37.187us - UncompressedRowBatchSize: 25.00 B (25) CodeGen:(Total: 56.195ms, non-child: 56.195ms, % non-child: 100.00%) - CodegenTime: 2.353ms - CompileTime: 11.129ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077532) - NumFunctions: 22 (22) - NumInstructions: 387 (387) - OptimizationTime: 23.819ms - PeakMemoryUsage: 193.50 KB (198144) - PrepareTime: 20.691ms AGGREGATION_NODE (id=1):(Total: 11s550ms, non-child: 1s259ms, % non-child: 10.90%) ExecOption: Codegen Enabled - BuildTime: 1s153ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 176.25 KB (180480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 10s291ms, non-child: 10s291ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): -1:14594/1.50 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 14594 out of 14594 Hdfs Read Thread Concurrency Bucket: 0:69.57% 1:21.74% 2:8.696% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0% File Formats: PARQUET/SNAPPY:29188 BytesRead(500.000ms): 30.91 MB, 85.26 MB, 146.54 MB, 198.99 MB, 261.91 MB, 328.68 MB, 368.44 MB, 418.58 MB, 488.95 MB, 549.17 MB, 610.23 MB, 666.86 MB, 703.67 MB, 789.04 MB, 842.82 MB, 893.61 MB, 955.12 MB, 1.01 GB, 1.08 GB, 1.13 GB, 1.18 GB, 1.24 GB, 1.30 GB - FooterProcessingTime: (Avg: 8.923ms ; Min: 1.251ms ; Max: 82.095ms ; Number of samples: 14594) - AverageHdfsReadThreadConcurrency: 0.39 - AverageScannerThreadConcurrency: 24.00 - BytesRead: 1.33 GB (1424815870) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 325.704ms - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 14.59K (14594) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 20.04 MB (21017275) - PerReadThreadRawHdfsThroughput: 383.14 MB/sec - RemoteScanRanges: 43.78K (43782) - RowBatchQueueGetWaitTime: 9s652ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 182.94M (182937778) - RowsReturned: 182.94M (182937778) - RowsReturnedRate: 17.78 M/sec - ScanRangesComplete: 14.59K (14594) - ScannerThreadsInvoluntaryContextSwitches: 1.02K (1020) - ScannerThreadsTotalWallClockTime: 4m35s - MaterializeTupleTime(*): 3s165ms - ScannerThreadsSysTime: 774.031ms - ScannerThreadsUserTime: 6s264ms - ScannerThreadsVoluntaryContextSwitches: 84.68K (84679) - TotalRawHdfsReadTime(*): 3s546ms - TotalReadThroughput: 115.90 MB/sec Instance 3b4691e0b8ed45f6:6edea28100000008 (host=d2404.foo.com:22000):(Total: 11s568ms, non-child: 2.672ms, % non-child: 0.02%) Hdfs split stats (:<# splits>/): -1:14363/1.50 GB MemoryUsage(500.000ms): 463.25 KB, 13.70 MB, 9.00 MB, 4.94 MB, 454.24 KB, 4.64 MB, 4.91 MB, 4.88 MB, 4.94 MB, 4.51 MB, 4.49 MB, 422.27 KB, 381.26 KB, 386.23 KB, 220.25 KB, 4.84 MB, 178.25 KB, 4.97 MB, 4.82 MB, 4.85 MB, 343.25 KB, 9.18 MB, 390.24 KB ThreadUsage(500.000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25 - AverageThreadTokens: 25.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 18.59 MB (19489575) - PerHostPeakMemUsage: 18.59 MB (19489575) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 217.331us - TotalStorageWaitTime: 4m27s - TotalThreadsInvoluntaryContextSwitches: 1.55K (1552) - TotalThreadsTotalWallClockTime: 4m46s - TotalThreadsSysTime: 1s162ms - TotalThreadsUserTime: 8s147ms - TotalThreadsVoluntaryContextSwitches: 216.77K (216771) Fragment Instance Lifecycle Timings: - ExecTime: 359.949us - ExecTreeExecTime: 10.639us - OpenTime: 11s509ms - ExecTreeOpenTime: 11s477ms - PrepareTime: 59.479ms - ExecTreePrepareTime: 37.183ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 216.00 GB (231928233984) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 56.494us, non-child: 56.494us, % non-child: 100.00%) - BytesSent: 25.00 B (25) - NetworkThroughput(*): 1.00 MB/sec - OverallThroughput: 432.15 KB/sec - PeakMemoryUsage: 7.52 KB (7704) - RowsReturned: 1 (1) - SerializeBatchTime: 21.373us - TransmitDataRPCTime: 23.813us - UncompressedRowBatchSize: 25.00 B (25) CodeGen:(Total: 51.108ms, non-child: 51.108ms, % non-child: 100.00%) - CodegenTime: 1.700ms - CompileTime: 10.189ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077532) - NumFunctions: 22 (22) - NumInstructions: 387 (387) - OptimizationTime: 20.118ms - PeakMemoryUsage: 193.50 KB (198144) - PrepareTime: 20.248ms AGGREGATION_NODE (id=1):(Total: 11s515ms, non-child: 1s249ms, % non-child: 10.85%) ExecOption: Codegen Enabled - BuildTime: 1s137ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 185.25 KB (189696) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 10s265ms, non-child: 10s265ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): -1:14363/1.50 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 14363 out of 14363 Hdfs Read Thread Concurrency Bucket: 0:73.91% 1:21.74% 2:4.348% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0% File Formats: PARQUET/SNAPPY:28726 BytesRead(500.000ms): 876.61 KB, 65.41 MB, 109.78 MB, 174.83 MB, 221.56 MB, 280.66 MB, 366.96 MB, 402.85 MB, 456.32 MB, 501.63 MB, 559.20 MB, 629.27 MB, 684.08 MB, 739.61 MB, 816.25 MB, 875.51 MB, 935.56 MB, 986.59 MB, 1.01 GB, 1.08 GB, 1.14 GB, 1.20 GB, 1.27 GB - FooterProcessingTime: (Avg: 8.976ms ; Min: 1.620ms ; Max: 85.413ms ; Number of samples: 14363) - AverageHdfsReadThreadConcurrency: 0.35 - AverageScannerThreadConcurrency: 24.00 - BytesRead: 1.32 GB (1421616203) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 330.533ms - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 14.36K (14363) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 18.55 MB (19447447) - PerReadThreadRawHdfsThroughput: 364.38 MB/sec - RemoteScanRanges: 43.09K (43089) - RowBatchQueueGetWaitTime: 9s653ms - RowBatchQueuePutWaitTime: 4.485ms - RowsRead: 184.56M (184556009) - RowsReturned: 184.56M (184556009) - RowsReturnedRate: 17.98 M/sec - ScanRangesComplete: 14.36K (14363) - ScannerThreadsInvoluntaryContextSwitches: 1.19K (1188) - ScannerThreadsTotalWallClockTime: 4m34s - MaterializeTupleTime(*): 3s122ms - ScannerThreadsSysTime: 884.756ms - ScannerThreadsUserTime: 6s121ms - ScannerThreadsVoluntaryContextSwitches: 80.53K (80527) - TotalRawHdfsReadTime(*): 3s720ms - TotalReadThroughput: 112.85 MB/sec Instance 3b4691e0b8ed45f6:6edea2810000000a (host=d2405.foo.com:22000):(Total: 11s533ms, non-child: 3.017ms, % non-child: 0.03%) Hdfs split stats (:<# splits>/): -1:13776/1.50 GB MemoryUsage(500.000ms): 194.27 KB, 344.25 KB, 367.21 KB, 241.27 KB, 204.23 KB, 4.52 MB, 304.27 KB, 4.91 MB, 4.57 MB, 8.71 MB, 192.23 KB, 15.81 MB, 317.26 KB, 273.26 KB, 4.76 MB, 9.34 MB, 304.26 KB, 4.50 MB, 4.46 MB, 458.25 KB, 303.27 KB, 305.26 KB, 232.26 KB ThreadUsage(500.000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25 - AverageThreadTokens: 25.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 27.80 MB (29151537) - PerHostPeakMemUsage: 27.81 MB (29159729) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 246.424us - TotalStorageWaitTime: 4m26s - TotalThreadsInvoluntaryContextSwitches: 1.41K (1408) - TotalThreadsTotalWallClockTime: 4m45s - TotalThreadsSysTime: 985.679ms - TotalThreadsUserTime: 8s071ms - TotalThreadsVoluntaryContextSwitches: 203.95K (203947) Fragment Instance Lifecycle Timings: - ExecTime: 362.312us - ExecTreeExecTime: 6.780us - OpenTime: 11s472ms - ExecTreeOpenTime: 11s441ms - PrepareTime: 59.801ms - ExecTreePrepareTime: 36.331ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 216.00 GB (231928233984) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 44.495us, non-child: 44.495us, % non-child: 100.00%) - BytesSent: 25.00 B (25) - NetworkThroughput(*): 1.07 MB/sec - OverallThroughput: 548.69 KB/sec - PeakMemoryUsage: 7.52 KB (7704) - RowsReturned: 1 (1) - SerializeBatchTime: 10.850us - TransmitDataRPCTime: 22.340us - UncompressedRowBatchSize: 25.00 B (25) CodeGen:(Total: 51.925ms, non-child: 51.925ms, % non-child: 100.00%) - CodegenTime: 2.424ms - CompileTime: 10.342ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077532) - NumFunctions: 22 (22) - NumInstructions: 387 (387) - OptimizationTime: 20.228ms - PeakMemoryUsage: 193.50 KB (198144) - PrepareTime: 20.799ms AGGREGATION_NODE (id=1):(Total: 11s478ms, non-child: 1s284ms, % non-child: 11.19%) ExecOption: Codegen Enabled - BuildTime: 1s174ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 185.25 KB (189696) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 10s193ms, non-child: 10s193ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): -1:13776/1.50 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 13776 out of 13776 Hdfs Read Thread Concurrency Bucket: 0:73.91% 1:21.74% 2:4.348% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0% File Formats: PARQUET/SNAPPY:27552 BytesRead(500.000ms): 2.64 MB, 56.35 MB, 115.28 MB, 160.34 MB, 217.60 MB, 275.57 MB, 335.76 MB, 402.03 MB, 453.17 MB, 516.04 MB, 560.95 MB, 613.17 MB, 661.06 MB, 735.87 MB, 811.98 MB, 876.08 MB, 914.41 MB, 971.66 MB, 1.00 GB, 1.07 GB, 1.13 GB, 1.19 GB, 1.27 GB - FooterProcessingTime: (Avg: 9.283ms ; Min: 1.644ms ; Max: 89.115ms ; Number of samples: 13776) - AverageHdfsReadThreadConcurrency: 0.30 - AverageScannerThreadConcurrency: 24.00 - BytesRead: 1.32 GB (1416036745) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 331.405ms - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 13.78K (13776) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 27.77 MB (29117601) - PerReadThreadRawHdfsThroughput: 374.76 MB/sec - RemoteScanRanges: 41.33K (41328) - RowBatchQueueGetWaitTime: 9s603ms - RowBatchQueuePutWaitTime: 52.851ms - RowsRead: 188.29M (188286995) - RowsReturned: 188.29M (188286995) - RowsReturnedRate: 18.47 M/sec - ScanRangesComplete: 13.78K (13776) - ScannerThreadsInvoluntaryContextSwitches: 1.09K (1093) - ScannerThreadsTotalWallClockTime: 4m34s - MaterializeTupleTime(*): 3s099ms - ScannerThreadsSysTime: 711.472ms - ScannerThreadsUserTime: 6s044ms - ScannerThreadsVoluntaryContextSwitches: 80.44K (80442) - TotalRawHdfsReadTime(*): 3s603ms - TotalReadThroughput: 112.87 MB/sec Instance 3b4691e0b8ed45f6:6edea28100000001 (host=d2403.foo.com:22000):(Total: 11s529ms, non-child: 2.723ms, % non-child: 0.02%) Hdfs split stats (:<# splits>/): -1:14126/1.50 GB MemoryUsage(500.000ms): 4.70 MB, 260.22 KB, 479.25 KB, 356.25 KB, 4.42 MB, 340.24 KB, 4.69 MB, 296.26 KB, 4.65 MB, 331.24 KB, 4.54 MB, 288.25 KB, 4.92 MB, 4.57 MB, 9.53 MB, 301.25 KB, 4.92 MB, 282.30 KB, 161.25 KB, 292.23 KB, 5.06 MB, 220.25 KB, 260.30 KB ThreadUsage(500.000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25 - AverageThreadTokens: 25.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 25.85 MB (27108607) - PerHostPeakMemUsage: 25.85 MB (27108607) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 245.676us - TotalStorageWaitTime: 4m26s - TotalThreadsInvoluntaryContextSwitches: 1.51K (1508) - TotalThreadsTotalWallClockTime: 4m45s - TotalThreadsSysTime: 1s003ms - TotalThreadsUserTime: 8s254ms - TotalThreadsVoluntaryContextSwitches: 204.60K (204605) Fragment Instance Lifecycle Timings: - ExecTime: 381.473us - ExecTreeExecTime: 4.708us - OpenTime: 11s467ms - ExecTreeOpenTime: 11s435ms - PrepareTime: 61.891ms - ExecTreePrepareTime: 38.561ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 216.00 GB (231928233984) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 68.351us, non-child: 68.351us, % non-child: 100.00%) - BytesSent: 25.00 B (25) - NetworkThroughput(*): 577.83 KB/sec - OverallThroughput: 357.19 KB/sec - PeakMemoryUsage: 7.52 KB (7704) - RowsReturned: 1 (1) - SerializeBatchTime: 10.391us - TransmitDataRPCTime: 42.251us - UncompressedRowBatchSize: 25.00 B (25) CodeGen:(Total: 52.634ms, non-child: 52.634ms, % non-child: 100.00%) - CodegenTime: 1.793ms - CompileTime: 10.927ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077532) - NumFunctions: 22 (22) - NumInstructions: 387 (387) - OptimizationTime: 20.142ms - PeakMemoryUsage: 193.50 KB (198144) - PrepareTime: 21.013ms AGGREGATION_NODE (id=1):(Total: 11s474ms, non-child: 1s329ms, % non-child: 11.58%) ExecOption: Codegen Enabled - BuildTime: 1s212ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 185.25 KB (189696) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 10s145ms, non-child: 10s145ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): -1:14126/1.50 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 14126 out of 14126 Hdfs Read Thread Concurrency Bucket: 0:86.96% 1:13.04% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0% File Formats: PARQUET/SNAPPY:28252 BytesRead(500.000ms): 19.21 MB, 63.29 MB, 120.20 MB, 180.31 MB, 253.55 MB, 319.48 MB, 381.13 MB, 433.32 MB, 496.10 MB, 554.74 MB, 604.21 MB, 658.52 MB, 701.01 MB, 778.87 MB, 830.05 MB, 893.68 MB, 949.23 MB, 988.88 MB, 1.02 GB, 1.07 GB, 1.14 GB, 1.21 GB, 1.27 GB - FooterProcessingTime: (Avg: 9.087ms ; Min: 1.671ms ; Max: 84.398ms ; Number of samples: 14126) - AverageHdfsReadThreadConcurrency: 0.13 - AverageScannerThreadConcurrency: 24.00 - BytesRead: 1.32 GB (1418956564) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 324.922ms - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 14.13K (14126) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 25.81 MB (27066479) - PerReadThreadRawHdfsThroughput: 393.01 MB/sec - RemoteScanRanges: 42.38K (42378) - RowBatchQueueGetWaitTime: 9s512ms - RowBatchQueuePutWaitTime: 18.696ms - RowsRead: 185.72M (185719848) - RowsReturned: 185.72M (185719848) - RowsReturnedRate: 18.31 M/sec - ScanRangesComplete: 14.13K (14126) - ScannerThreadsInvoluntaryContextSwitches: 1.16K (1159) - ScannerThreadsTotalWallClockTime: 4m33s - MaterializeTupleTime(*): 3s133ms - ScannerThreadsSysTime: 745.669ms - ScannerThreadsUserTime: 6s144ms - ScannerThreadsVoluntaryContextSwitches: 81.32K (81323) - TotalRawHdfsReadTime(*): 3s443ms - TotalReadThroughput: 112.71 MB/sec Instance 3b4691e0b8ed45f6:6edea28100000009 (host=d2408.foo.com:22000):(Total: 11s420ms, non-child: 2.733ms, % non-child: 0.02%) Hdfs split stats (:<# splits>/): -1:13914/1.50 GB MemoryUsage(500.000ms): 213.02 KB, 4.63 MB, 312.27 KB, 4.55 MB, 297.25 KB, 9.06 MB, 305.25 KB, 9.15 MB, 4.56 MB, 326.23 KB, 4.81 MB, 332.25 KB, 4.82 MB, 4.82 MB, 339.25 KB, 13.34 MB, 296.25 KB, 245.26 KB, 284.25 KB, 9.31 MB, 4.50 MB, 365.26 KB, 246.26 KB ThreadUsage(500.000ms): 1, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25 - AverageThreadTokens: 23.96 - BloomFilterBytes: 0 - PeakMemoryUsage: 20.83 MB (21837700) - PerHostPeakMemUsage: 20.83 MB (21837700) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 296.683us - TotalStorageWaitTime: 4m23s - TotalThreadsInvoluntaryContextSwitches: 1.70K (1700) - TotalThreadsTotalWallClockTime: 4m42s - TotalThreadsSysTime: 1s124ms - TotalThreadsUserTime: 8s700ms - TotalThreadsVoluntaryContextSwitches: 208.50K (208497) Fragment Instance Lifecycle Timings: - ExecTime: 449.434us - ExecTreeExecTime: 5.420us - OpenTime: 11s360ms - ExecTreeOpenTime: 11s329ms - PrepareTime: 59.554ms - ExecTreePrepareTime: 36.763ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 216.00 GB (231928233984) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 41.555us, non-child: 41.555us, % non-child: 100.00%) - BytesSent: 25.00 B (25) - NetworkThroughput(*): 1.18 MB/sec - OverallThroughput: 587.51 KB/sec - PeakMemoryUsage: 7.52 KB (7704) - RowsReturned: 1 (1) - SerializeBatchTime: 10.690us - TransmitDataRPCTime: 20.237us - UncompressedRowBatchSize: 25.00 B (25) CodeGen:(Total: 51.363ms, non-child: 51.363ms, % non-child: 100.00%) - CodegenTime: 1.689ms - CompileTime: 10.176ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077532) - NumFunctions: 22 (22) - NumInstructions: 387 (387) - OptimizationTime: 20.107ms - PeakMemoryUsage: 193.50 KB (198144) - PrepareTime: 20.564ms AGGREGATION_NODE (id=1):(Total: 11s366ms, non-child: 1s314ms, % non-child: 11.56%) ExecOption: Codegen Enabled - BuildTime: 1s201ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 176.25 KB (180480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 10s052ms, non-child: 10s052ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): -1:13914/1.50 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 13914 out of 13914 Hdfs Read Thread Concurrency Bucket: 0:68.18% 1:22.73% 2:9.091% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0% File Formats: PARQUET/SNAPPY:27828 BytesRead(500.000ms): 0, 59.49 MB, 115.94 MB, 176.87 MB, 240.69 MB, 302.58 MB, 374.96 MB, 415.80 MB, 486.53 MB, 548.32 MB, 611.11 MB, 671.91 MB, 724.22 MB, 794.01 MB, 851.55 MB, 925.89 MB, 983.14 MB, 1020.87 MB, 1.07 GB, 1.12 GB, 1.18 GB, 1.22 GB, 1.29 GB - FooterProcessingTime: (Avg: 9.115ms ; Min: 2.200ms ; Max: 84.151ms ; Number of samples: 13914) - AverageHdfsReadThreadConcurrency: 0.41 - AverageScannerThreadConcurrency: 24.00 - BytesRead: 1.32 GB (1417546388) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 357.310ms - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 13.91K (13914) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 20.79 MB (21795572) - PerReadThreadRawHdfsThroughput: 369.20 MB/sec - RemoteScanRanges: 41.74K (41742) - RowBatchQueueGetWaitTime: 9s424ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 187.72M (187716424) - RowsReturned: 187.72M (187716424) - RowsReturnedRate: 18.67 M/sec - ScanRangesComplete: 13.91K (13914) - ScannerThreadsInvoluntaryContextSwitches: 1.34K (1341) - ScannerThreadsTotalWallClockTime: 4m31s - MaterializeTupleTime(*): 3s185ms - ScannerThreadsSysTime: 881.846ms - ScannerThreadsUserTime: 6s586ms - ScannerThreadsVoluntaryContextSwitches: 79.07K (79068) - TotalRawHdfsReadTime(*): 3s661ms - TotalReadThroughput: 114.49 MB/sec Instance 3b4691e0b8ed45f6:6edea28100000002 (host=d2406.foo.com:22000):(Total: 11s363ms, non-child: 2.831ms, % non-child: 0.02%) Hdfs split stats (:<# splits>/): -1:13852/1.50 GB MemoryUsage(500.000ms): 272.24 KB, 9.09 MB, 13.58 MB, 253.25 KB, 4.61 MB, 226.25 KB, 255.22 KB, 4.91 MB, 360.26 KB, 4.81 MB, 515.51 KB, 9.37 MB, 313.25 KB, 322.25 KB, 9.02 MB, 257.26 KB, 4.80 MB, 4.60 MB, 4.97 MB, 4.53 MB, 288.29 KB, 8.92 MB, 394.25 KB ThreadUsage(500.000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25 - AverageThreadTokens: 25.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 19.82 MB (20780866) - PerHostPeakMemUsage: 19.82 MB (20780866) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 248.514us - TotalStorageWaitTime: 4m21s - TotalThreadsInvoluntaryContextSwitches: 1.36K (1359) - TotalThreadsTotalWallClockTime: 4m40s - TotalThreadsSysTime: 1s168ms - TotalThreadsUserTime: 8s275ms - TotalThreadsVoluntaryContextSwitches: 211.11K (211108) Fragment Instance Lifecycle Timings: - ExecTime: 463.772us - ExecTreeExecTime: 7.750us - OpenTime: 11s302ms - ExecTreeOpenTime: 11s270ms - PrepareTime: 60.106ms - ExecTreePrepareTime: 36.785ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 216.00 GB (231928233984) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 47.668us, non-child: 47.668us, % non-child: 100.00%) - BytesSent: 25.00 B (25) - NetworkThroughput(*): 1.00 MB/sec - OverallThroughput: 512.17 KB/sec - PeakMemoryUsage: 7.52 KB (7704) - RowsReturned: 1 (1) - SerializeBatchTime: 12.059us - TransmitDataRPCTime: 23.781us - UncompressedRowBatchSize: 25.00 B (25) CodeGen:(Total: 52.605ms, non-child: 52.605ms, % non-child: 100.00%) - CodegenTime: 1.807ms - CompileTime: 10.861ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077532) - NumFunctions: 22 (22) - NumInstructions: 387 (387) - OptimizationTime: 20.208ms - PeakMemoryUsage: 193.50 KB (198144) - PrepareTime: 20.970ms AGGREGATION_NODE (id=1):(Total: 11s307ms, non-child: 1s313ms, % non-child: 11.62%) ExecOption: Codegen Enabled - BuildTime: 1s197ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 176.25 KB (180480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 9s993ms, non-child: 9s993ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): -1:13852/1.50 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 13852 out of 13852 Hdfs Read Thread Concurrency Bucket: 0:73.91% 1:17.39% 2:8.696% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0% File Formats: PARQUET/SNAPPY:27704 BytesRead(500.000ms): 1.51 MB, 73.17 MB, 118.39 MB, 190.80 MB, 255.58 MB, 318.35 MB, 381.11 MB, 426.40 MB, 482.80 MB, 544.91 MB, 603.18 MB, 651.63 MB, 710.44 MB, 772.77 MB, 852.00 MB, 907.92 MB, 971.31 MB, 1013.52 MB, 1.05 GB, 1.11 GB, 1.16 GB, 1.23 GB, 1.29 GB - FooterProcessingTime: (Avg: 9.084ms ; Min: 1.572ms ; Max: 116.076ms ; Number of samples: 13852) - AverageHdfsReadThreadConcurrency: 0.35 - AverageScannerThreadConcurrency: 24.00 - BytesRead: 1.32 GB (1416350504) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 333.804ms - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 13.85K (13852) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 19.78 MB (20743858) - PerReadThreadRawHdfsThroughput: 376.88 MB/sec - RemoteScanRanges: 41.56K (41556) - RowBatchQueueGetWaitTime: 9s347ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 188.26M (188262493) - RowsReturned: 188.26M (188262493) - RowsReturnedRate: 18.84 M/sec - ScanRangesComplete: 13.85K (13852) - ScannerThreadsInvoluntaryContextSwitches: 996 (996) - ScannerThreadsTotalWallClockTime: 4m29s - MaterializeTupleTime(*): 3s169ms - ScannerThreadsSysTime: 866.268ms - ScannerThreadsUserTime: 6s182ms - ScannerThreadsVoluntaryContextSwitches: 79.15K (79149) - TotalRawHdfsReadTime(*): 3s584ms - TotalReadThroughput: 115.18 MB/sec Instance 3b4691e0b8ed45f6:6edea28100000007 (host=d2410.foo.com:22000):(Total: 11s299ms, non-child: 2.747ms, % non-child: 0.02%) Hdfs split stats (:<# splits>/): -1:13459/1.50 GB MemoryUsage(500.000ms): 5.01 MB, 4.88 MB, 4.90 MB, 229.25 KB, 284.25 KB, 4.90 MB, 258.27 KB, 5.11 MB, 331.24 KB, 359.23 KB, 348.25 KB, 434.22 KB, 4.99 MB, 244.21 KB, 352.25 KB, 242.25 KB, 399.24 KB, 549.25 KB, 5.00 MB, 399.25 KB, 4.83 MB, 232.25 KB, 244.27 KB ThreadUsage(500.000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25 - AverageThreadTokens: 25.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 22.35 MB (23430909) - PerHostPeakMemUsage: 22.35 MB (23430909) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 252.228us - TotalStorageWaitTime: 4m20s - TotalThreadsInvoluntaryContextSwitches: 1.25K (1252) - TotalThreadsTotalWallClockTime: 4m39s - TotalThreadsSysTime: 1s042ms - TotalThreadsUserTime: 8s356ms - TotalThreadsVoluntaryContextSwitches: 215.93K (215932) Fragment Instance Lifecycle Timings: - ExecTime: 436.360us - ExecTreeExecTime: 5.867us - OpenTime: 11s240ms - ExecTreeOpenTime: 11s209ms - PrepareTime: 58.586ms - ExecTreePrepareTime: 35.755ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 216.00 GB (231928233984) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 45.690us, non-child: 45.690us, % non-child: 100.00%) - BytesSent: 25.00 B (25) - NetworkThroughput(*): 1.10 MB/sec - OverallThroughput: 534.34 KB/sec - PeakMemoryUsage: 7.52 KB (7704) - RowsReturned: 1 (1) - SerializeBatchTime: 11.977us - TransmitDataRPCTime: 21.620us - UncompressedRowBatchSize: 25.00 B (25) CodeGen:(Total: 51.882ms, non-child: 51.882ms, % non-child: 100.00%) - CodegenTime: 1.779ms - CompileTime: 10.672ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077532) - NumFunctions: 22 (22) - NumInstructions: 387 (387) - OptimizationTime: 20.134ms - PeakMemoryUsage: 193.50 KB (198144) - PrepareTime: 20.537ms AGGREGATION_NODE (id=1):(Total: 11s244ms, non-child: 1s280ms, % non-child: 11.39%) ExecOption: Codegen Enabled - BuildTime: 1s167ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 176.25 KB (180480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 9s964ms, non-child: 9s964ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): -1:13459/1.50 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 13459 out of 13459 Hdfs Read Thread Concurrency Bucket: 0:91.3% 1:8.696% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0% File Formats: PARQUET/SNAPPY:26918 BytesRead(500.000ms): 21.79 MB, 98.92 MB, 139.84 MB, 211.54 MB, 283.26 MB, 345.26 MB, 397.07 MB, 463.67 MB, 510.34 MB, 571.19 MB, 633.09 MB, 711.18 MB, 759.22 MB, 830.92 MB, 894.23 MB, 954.21 MB, 1009.17 MB, 1.04 GB, 1.10 GB, 1.14 GB, 1.21 GB, 1.26 GB, 1.31 GB - FooterProcessingTime: (Avg: 9.305ms ; Min: 1.133ms ; Max: 70.349ms ; Number of samples: 13459) - AverageHdfsReadThreadConcurrency: 0.04 - AverageScannerThreadConcurrency: 24.00 - BytesRead: 1.31 GB (1409360851) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 332.982ms - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 13.46K (13459) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 22.31 MB (23397997) - PerReadThreadRawHdfsThroughput: 381.18 MB/sec - RemoteScanRanges: 40.38K (40377) - RowBatchQueueGetWaitTime: 9s307ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 190.13M (190134081) - RowsReturned: 190.13M (190134081) - RowsReturnedRate: 19.08 M/sec - ScanRangesComplete: 13.46K (13459) - ScannerThreadsInvoluntaryContextSwitches: 890 (890) - ScannerThreadsTotalWallClockTime: 4m28s - MaterializeTupleTime(*): 3s253ms - ScannerThreadsSysTime: 792.639ms - ScannerThreadsUserTime: 6s240ms - ScannerThreadsVoluntaryContextSwitches: 79.13K (79126) - TotalRawHdfsReadTime(*): 3s526ms - TotalReadThroughput: 116.46 MB/sec Instance 3b4691e0b8ed45f6:6edea28100000005 (host=d2409.foo.com:22000):(Total: 11s018ms, non-child: 2.587ms, % non-child: 0.02%) Hdfs split stats (:<# splits>/): -1:13203/1.50 GB MemoryUsage(500.000ms): 4.59 MB, 13.33 MB, 13.27 MB, 309.25 KB, 172.26 KB, 178.26 KB, 4.44 MB, 347.25 KB, 402.25 KB, 8.98 MB, 338.25 KB, 240.26 KB, 329.28 KB, 4.75 MB, 4.81 MB, 4.75 MB, 288.25 KB, 4.84 MB, 373.26 KB, 223.24 KB, 4.55 MB, 268.24 KB ThreadUsage(500.000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25 - AverageThreadTokens: 25.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 22.64 MB (23741974) - PerHostPeakMemUsage: 22.64 MB (23741974) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 192.837us - TotalStorageWaitTime: 4m14s - TotalThreadsInvoluntaryContextSwitches: 1.53K (1535) - TotalThreadsTotalWallClockTime: 4m32s - TotalThreadsSysTime: 1s031ms - TotalThreadsUserTime: 8s182ms - TotalThreadsVoluntaryContextSwitches: 205.80K (205804) Fragment Instance Lifecycle Timings: - ExecTime: 334.047us - ExecTreeExecTime: 6.654us - OpenTime: 10s960ms - ExecTreeOpenTime: 10s929ms - PrepareTime: 57.833ms - ExecTreePrepareTime: 35.418ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 216.00 GB (231928233984) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 45.757us, non-child: 45.757us, % non-child: 100.00%) - BytesSent: 25.00 B (25) - NetworkThroughput(*): 1.11 MB/sec - OverallThroughput: 533.56 KB/sec - PeakMemoryUsage: 7.52 KB (7704) - RowsReturned: 1 (1) - SerializeBatchTime: 10.127us - TransmitDataRPCTime: 21.538us - UncompressedRowBatchSize: 25.00 B (25) CodeGen:(Total: 51.335ms, non-child: 51.335ms, % non-child: 100.00%) - CodegenTime: 1.690ms - CompileTime: 10.656ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077532) - NumFunctions: 22 (22) - NumInstructions: 387 (387) - OptimizationTime: 19.980ms - PeakMemoryUsage: 193.50 KB (198144) - PrepareTime: 20.173ms AGGREGATION_NODE (id=1):(Total: 10s964ms, non-child: 1s382ms, % non-child: 12.60%) ExecOption: Codegen Enabled - BuildTime: 1s260ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 176.25 KB (180480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 9s582ms, non-child: 9s582ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): -1:13203/1.50 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 13203 out of 13203 Hdfs Read Thread Concurrency Bucket: 0:68.18% 1:22.73% 2:9.091% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0% File Formats: PARQUET/SNAPPY:26406 BytesRead(500.000ms): 5.27 MB, 75.30 MB, 153.34 MB, 231.02 MB, 294.81 MB, 346.85 MB, 415.18 MB, 457.51 MB, 525.77 MB, 589.30 MB, 638.15 MB, 680.04 MB, 745.46 MB, 802.39 MB, 872.62 MB, 928.03 MB, 991.30 MB, 1.02 GB, 1.08 GB, 1.14 GB, 1.19 GB, 1.26 GB - FooterProcessingTime: (Avg: 9.239ms ; Min: 2.263ms ; Max: 82.434ms ; Number of samples: 13203) - AverageHdfsReadThreadConcurrency: 0.36 - AverageScannerThreadConcurrency: 24.00 - BytesRead: 1.31 GB (1406115575) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 338.142ms - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 13.20K (13203) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 22.60 MB (23699846) - PerReadThreadRawHdfsThroughput: 343.88 MB/sec - RemoteScanRanges: 39.61K (39609) - RowBatchQueueGetWaitTime: 8s945ms - RowBatchQueuePutWaitTime: 12.414ms - RowsRead: 191.68M (191678805) - RowsReturned: 191.68M (191678805) - RowsReturnedRate: 20.00 M/sec - ScanRangesComplete: 13.20K (13203) - ScannerThreadsInvoluntaryContextSwitches: 1.19K (1185) - ScannerThreadsTotalWallClockTime: 4m21s - MaterializeTupleTime(*): 3s207ms - ScannerThreadsSysTime: 763.054ms - ScannerThreadsUserTime: 6s018ms - ScannerThreadsVoluntaryContextSwitches: 76.36K (76364) - TotalRawHdfsReadTime(*): 3s899ms - TotalReadThroughput: 116.87 MB/sec Instance 3b4691e0b8ed45f6:6edea28100000006 (host=d2411.foo.com:22000):(Total: 10s945ms, non-child: 2.543ms, % non-child: 0.02%) Hdfs split stats (:<# splits>/): -1:13092/1.50 GB MemoryUsage(500.000ms): 241.25 KB, 229.25 KB, 479.25 KB, 383.23 KB, 5.06 MB, 320.21 KB, 8.77 MB, 13.84 MB, 291.26 KB, 132.23 KB, 4.47 MB, 4.95 MB, 4.90 MB, 431.27 KB, 287.25 KB, 373.27 KB, 4.81 MB, 4.43 MB, 4.79 MB, 4.65 MB, 382.23 KB ThreadUsage(500.000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25 - AverageThreadTokens: 25.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 27.39 MB (28721347) - PerHostPeakMemUsage: 27.39 MB (28721347) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 211.843us - TotalStorageWaitTime: 4m12s - TotalThreadsInvoluntaryContextSwitches: 1.52K (1522) - TotalThreadsTotalWallClockTime: 4m30s - TotalThreadsSysTime: 1s025ms - TotalThreadsUserTime: 8s132ms - TotalThreadsVoluntaryContextSwitches: 203.33K (203328) Fragment Instance Lifecycle Timings: - ExecTime: 333.647us - ExecTreeExecTime: 5.691us - OpenTime: 10s886ms - ExecTreeOpenTime: 10s853ms - PrepareTime: 58.259ms - ExecTreePrepareTime: 35.304ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 216.00 GB (231928233984) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 45.871us, non-child: 45.871us, % non-child: 100.00%) - BytesSent: 25.00 B (25) - NetworkThroughput(*): 1.02 MB/sec - OverallThroughput: 532.23 KB/sec - PeakMemoryUsage: 7.52 KB (7704) - RowsReturned: 1 (1) - SerializeBatchTime: 11.021us - TransmitDataRPCTime: 23.334us - UncompressedRowBatchSize: 25.00 B (25) CodeGen:(Total: 53.434ms, non-child: 53.434ms, % non-child: 100.00%) - CodegenTime: 1.916ms - CompileTime: 11.797ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077532) - NumFunctions: 22 (22) - NumInstructions: 387 (387) - OptimizationTime: 20.332ms - PeakMemoryUsage: 193.50 KB (198144) - PrepareTime: 20.672ms AGGREGATION_NODE (id=1):(Total: 10s888ms, non-child: 1s363ms, % non-child: 12.53%) ExecOption: Codegen Enabled - BuildTime: 1s246ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 176.25 KB (180480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 9s525ms, non-child: 9s525ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): -1:13092/1.50 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 13092 out of 13092 Hdfs Read Thread Concurrency Bucket: 0:76.19% 1:14.29% 2:9.524% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0% File Formats: PARQUET/SNAPPY:26184 BytesRead(500.000ms): 57.03 MB, 108.88 MB, 169.52 MB, 235.97 MB, 291.77 MB, 354.53 MB, 407.34 MB, 491.19 MB, 556.95 MB, 605.35 MB, 651.15 MB, 719.79 MB, 769.87 MB, 823.13 MB, 889.72 MB, 965.50 MB, 1.00 GB, 1.06 GB, 1.12 GB, 1.18 GB, 1.24 GB - FooterProcessingTime: (Avg: 9.275ms ; Min: 2.330ms ; Max: 81.139ms ; Number of samples: 13092) - AverageHdfsReadThreadConcurrency: 0.33 - AverageScannerThreadConcurrency: 24.00 - BytesRead: 1.31 GB (1406848689) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 342.205ms - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 13.09K (13092) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 27.35 MB (28679219) - PerReadThreadRawHdfsThroughput: 357.35 MB/sec - RemoteScanRanges: 39.28K (39276) - RowBatchQueueGetWaitTime: 8s896ms - RowBatchQueuePutWaitTime: 33.238ms - RowsRead: 192.78M (192782304) - RowsReturned: 192.78M (192782304) - RowsReturnedRate: 20.24 M/sec - ScanRangesComplete: 13.09K (13092) - ScannerThreadsInvoluntaryContextSwitches: 1.17K (1165) - ScannerThreadsTotalWallClockTime: 4m19s - MaterializeTupleTime(*): 3s160ms - ScannerThreadsSysTime: 755.342ms - ScannerThreadsUserTime: 5s990ms - ScannerThreadsVoluntaryContextSwitches: 76.82K (76817) - TotalRawHdfsReadTime(*): 3s754ms - TotalReadThroughput: 121.16 MB/sec Instance 3b4691e0b8ed45f6:6edea28100000004 (host=d2407.foo.com:22000):(Total: 10s273ms, non-child: 2.700ms, % non-child: 0.03%) Hdfs split stats (:<# splits>/): -1:12171/1.50 GB MemoryUsage(500.000ms): 13.26 MB, 4.68 MB, 4.92 MB, 289.25 KB, 4.92 MB, 4.56 MB, 9.21 MB, 4.63 MB, 558.21 KB, 348.27 KB, 9.05 MB, 376.22 KB, 9.31 MB, 9.13 MB, 9.39 MB, 13.61 MB, 4.74 MB, 4.92 MB, 308.27 KB, 4.79 MB, 4.70 MB ThreadUsage(500.000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25 - AverageThreadTokens: 25.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 22.71 MB (23811139) - PerHostPeakMemUsage: 22.71 MB (23811139) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 354.235us - TotalStorageWaitTime: 3m56s - TotalThreadsInvoluntaryContextSwitches: 1.43K (1431) - TotalThreadsTotalWallClockTime: 4m13s - TotalThreadsSysTime: 1s077ms - TotalThreadsUserTime: 7s940ms - TotalThreadsVoluntaryContextSwitches: 205.41K (205411) Fragment Instance Lifecycle Timings: - ExecTime: 533.357us - ExecTreeExecTime: 9.223us - OpenTime: 10s218ms - ExecTreeOpenTime: 10s187ms - PrepareTime: 54.379ms - ExecTreePrepareTime: 31.915ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 216.00 GB (231928233984) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 81.025us, non-child: 81.025us, % non-child: 100.00%) - BytesSent: 25.00 B (25) - NetworkThroughput(*): 515.87 KB/sec - OverallThroughput: 301.31 KB/sec - PeakMemoryUsage: 7.52 KB (7704) - RowsReturned: 1 (1) - SerializeBatchTime: 22.091us - TransmitDataRPCTime: 47.326us - UncompressedRowBatchSize: 25.00 B (25) CodeGen:(Total: 51.416ms, non-child: 51.416ms, % non-child: 100.00%) - CodegenTime: 1.695ms - CompileTime: 10.782ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077532) - NumFunctions: 22 (22) - NumInstructions: 387 (387) - OptimizationTime: 19.823ms - PeakMemoryUsage: 193.50 KB (198144) - PrepareTime: 20.216ms AGGREGATION_NODE (id=1):(Total: 10s219ms, non-child: 1s327ms, % non-child: 12.99%) ExecOption: Codegen Enabled - BuildTime: 1s213ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 176.25 KB (180480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 8s892ms, non-child: 8s892ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): -1:12171/1.50 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 12171 out of 12171 Hdfs Read Thread Concurrency Bucket: 0:42.86% 1:47.62% 2:9.524% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0% File Formats: PARQUET/SNAPPY:24342 BytesRead(500.000ms): 9.13 MB, 76.75 MB, 139.09 MB, 197.38 MB, 268.18 MB, 322.80 MB, 410.06 MB, 455.72 MB, 513.86 MB, 588.15 MB, 651.97 MB, 718.36 MB, 790.85 MB, 849.77 MB, 926.62 MB, 1000.49 MB, 1.04 GB, 1.10 GB, 1.16 GB, 1.22 GB, 1.28 GB - FooterProcessingTime: (Avg: 9.376ms ; Min: 2.092ms ; Max: 88.203ms ; Number of samples: 12171) - AverageHdfsReadThreadConcurrency: 0.62 - AverageScannerThreadConcurrency: 24.00 - BytesRead: 1.30 GB (1392972079) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 345.966ms - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 12.17K (12171) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 22.67 MB (23769011) - PerReadThreadRawHdfsThroughput: 349.59 MB/sec - RemoteScanRanges: 36.51K (36513) - RowBatchQueueGetWaitTime: 8s291ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 197.92M (197918774) - RowsReturned: 197.92M (197918774) - RowsReturnedRate: 22.26 M/sec - ScanRangesComplete: 12.17K (12171) - ScannerThreadsInvoluntaryContextSwitches: 1.10K (1097) - ScannerThreadsTotalWallClockTime: 4m3s - MaterializeTupleTime(*): 3s264ms - ScannerThreadsSysTime: 811.514ms - ScannerThreadsUserTime: 5s835ms - ScannerThreadsVoluntaryContextSwitches: 69.21K (69213) - TotalRawHdfsReadTime(*): 3s800ms - TotalReadThroughput: 125.17 MB/sec