Query (id=9744f2a34755b414:966207dc418a60b5): Summary: Session ID: 814d7f46ad1eb2c8:db524a5aeb1f53ac Session Type: BEESWAX Start Time: 2014-10-22 20:30:26.012588000 End Time: 2014-10-22 21:01:41.777421000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 1.3.0-INTERNAL DEBUG (build 77aedcc32762b3fe24db655e34b3d4cdbad9c9fd) User: mj Connected User: mj Delegated User: Network Address: 127.0.0.1:33338 Default Db: functional_parquet Sql Statement: select sum(t1.int_col * t1.id) OVER ( ORDER BY t1.int_col * t1.id ASC, t3.year + t3.tinyint_col ASC ROWS BETWEEN 66 PRECEDING AND 21 FOLLOWING) AS int_col_1 FROM alltypes t1 INNER JOIN alltypes t2 ON t2.month = t1.smallint_col INNER JOIN alltypes t3 ON t3.bigint_col = t2.bigint_col WHERE t2.id <= t3.smallint_col + t2.tinyint_col order by 1 limit 5 Coordinator: mj:22000 Plan: ---------------- Estimated Per-Host Requirements: Memory=4.09GB VCores=3 WARNING: The following tables are missing relevant table and/or column statistics. functional_parquet.alltypes F03:PLAN FRAGMENT [UNPARTITIONED] 07:TOP-N [LIMIT=5] | order by: sum(t1.int_col * t1.id) ASC | hosts=3 per-host-mem=unavailable | tuple-ids=4 row-size=8B cardinality=5 | 06:ANALYTIC | functions: sum(int_col * id) | order by: int_col * id ASC, year + tinyint_col ASC | window: ROWS BETWEEN 66 PRECEDING AND 21 FOLLOWING | hosts=3 per-host-mem=unavailable | tuple-ids=6,5 row-size=50B cardinality=unavailable | 10:MERGING-EXCHANGE [UNPARTITIONED] order by: int_col * id ASC, year + tinyint_col ASC hosts=3 per-host-mem=unavailable tuple-ids=6 row-size=42B cardinality=unavailable F00:PLAN FRAGMENT [RANDOM] DATASTREAM SINK [FRAGMENT=F03, EXCHANGE=10, UNPARTITIONED] 05:SORT | order by: int_col * id ASC, year + tinyint_col ASC | hosts=3 per-host-mem=0B | tuple-ids=6 row-size=42B cardinality=unavailable | 04:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: t2.bigint_col = t3.bigint_col | other predicates: t2.id <= t3.smallint_col + t2.tinyint_col | hosts=3 per-host-mem=2.00GB | tuple-ids=0,1,2 row-size=42B cardinality=unavailable | |--09:EXCHANGE [BROADCAST] | hosts=3 per-host-mem=0B | tuple-ids=2 row-size=15B cardinality=unavailable | 03:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: t1.smallint_col = t2.month | hosts=3 per-host-mem=2.00GB | tuple-ids=0,1 row-size=27B cardinality=unavailable | |--08:EXCHANGE [BROADCAST] | hosts=3 per-host-mem=0B | tuple-ids=1 row-size=17B cardinality=unavailable | 00:SCAN HDFS [functional_parquet.alltypes t1, RANDOM] partitions=24/24 size=156.69KB table stats: unavailable column stats: unavailable hosts=3 per-host-mem=48.00MB tuple-ids=0 row-size=10B cardinality=unavailable F02:PLAN FRAGMENT [RANDOM] DATASTREAM SINK [FRAGMENT=F00, EXCHANGE=09, BROADCAST] 02:SCAN HDFS [functional_parquet.alltypes t3, RANDOM] partitions=24/24 size=156.69KB table stats: unavailable columns missing stats: bigint_col, tinyint_col, smallint_col hosts=3 per-host-mem=48.00MB tuple-ids=2 row-size=15B cardinality=unavailable F01:PLAN FRAGMENT [RANDOM] DATASTREAM SINK [FRAGMENT=F00, EXCHANGE=08, BROADCAST] 01:SCAN HDFS [functional_parquet.alltypes t2, RANDOM] partitions=24/24 size=156.69KB table stats: unavailable columns missing stats: bigint_col, id, tinyint_col hosts=3 per-host-mem=48.00MB tuple-ids=1 row-size=17B cardinality=unavailable ---------------- Estimated Per-Host Mem: 4395630592 Estimated Per-Host VCores: 3 Tables Missing Stats: functional_parquet.alltypes Request Pool: default-pool ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail -------------------------------------------------------------------------------------------------------------------------------- 07:TOP-N 1 1s515ms 1s515ms 5 5 16.02 MB -1.00 B 06:ANALYTIC 1 10s945ms 10s945ms 5.33M -1 32.25 MB -1.00 B 10:MERGING-EXCHANGE 1 639.447ms 639.447ms 5.33M -1 0 -1.00 B UNPARTITIONED 05:SORT 1 2m23s 2m23s 5.33M -1 248.17 MB 0 04:HASH JOIN 1 28m37s 28m37s 5.33M -1 4.07 MB 2.00 GB INNER JOIN, BROADCAST |--09:EXCHANGE 1 1.952ms 1.952ms 7.30K -1 0 0 BROADCAST | 02:SCAN HDFS 1 584.840ms 584.840ms 7.30K -1 191.00 KB 48.00 MB functional_parquet.alltypes t3 03:HASH JOIN 1 1s126ms 1s126ms 3.99M -1 4.10 MB 2.00 GB INNER JOIN, BROADCAST |--08:EXCHANGE 1 1.811ms 1.811ms 7.30K -1 0 0 BROADCAST | 01:SCAN HDFS 1 585.767ms 585.767ms 7.30K -1 212.00 KB 48.00 MB functional_parquet.alltypes t2 00:SCAN HDFS 1 14.134ms 14.134ms 7.30K -1 924.00 KB 48.00 MB functional_parquet.alltypes t1 Query Timeline: 31m15s - Start execution: 62.584us (62.584us) - Planning finished: 526.568ms (526.506ms) - Ready to start remote fragments: 534.190ms (7.621ms) - Remote fragments started: 549.943ms (15.753ms) - Rows available: 31m15s (31m14s) - First row fetched: 31m15s (275.788ms) - Unregister query: 31m15s (1.618ms) ImpalaServer: - ClientFetchWaitTimer: 276.394ms - RowMaterializationTimer: 32.647us Execution Profile 9744f2a34755b414:966207dc418a60b5:(Total: 31m14s, non-child: 0ns, % non-child: 0.00%) Fragment start latencies: count: 3, last: 0.00161592, min: 0.00161592, max: 0.0111199, mean: 0.00487224, stddev: 0.00441911 Per Node Peak Memory Usage: mj:22000(426.28 MB) - FinalizationTimer: 0ns Coordinator Fragment F03:(Total: 31m14s, non-child: 3.647ms, % non-child: 0.00%) MemoryUsage(32s000ms): 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB - AverageThreadTokens: 0.00 - PeakMemoryUsage: 162.26 MB (170144624) - PerHostPeakMemUsage: 0 - PrepareTime: 593.115us - RowsProduced: 5 - TotalCpuTime: 31m15s - TotalNetworkReceiveTime: 0ns - TotalNetworkSendTime: 0ns - TotalStorageWaitTime: 0ns BlockMgr: - BlockWritesOutstanding: 0 - BlocksCreated: 128 - BlocksRecycled: 84 - BufferedPins: 0 - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 20.05 GB (21530671104) - PeakMemoryUsage: 272.00 MB (285212672) - TotalBufferWaitTime: 0ns - TotalEncryptionTime: 0ns - TotalIntegrityCheckTime: 0ns - TotalReadBlockTime: 0ns SORT_NODE (id=7):(Total: 31m14s, non-child: 1s515ms, % non-child: 0.08%) - PeakMemoryUsage: 16.02 MB (16801792) - RowsReturned: 5 - RowsReturnedRate: 0 ANALYTIC_EVAL_NODE (id=6):(Total: 31m13s, non-child: 10s945ms, % non-child: 0.58%) - EvaluationTime: 10s891ms - GetNewBlockTime: 630.486us - PeakMemoryUsage: 32.25 MB (33820672) - PinTime: 0ns - RowsReturned: 5.33M (5329000) - RowsReturnedRate: 2.84 K/sec - UnpinTime: 188.542us EXCHANGE_NODE (id=10):(Total: 31m2s, non-child: 639.447ms, % non-child: 0.03%) BytesReceived(32s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 - BytesReceived: 21.91 MB (22973816) - ConvertRowBatchTime: 0ns - DeserializeRowBatchTimer: 250.447ms - FirstBatchArrivalWaitTime: 31m1s - MergeGetNext: 635.207ms - MergeGetNextBatch: 44.483ms - PeakMemoryUsage: 0 - RowsReturned: 5.33M (5329000) - RowsReturnedRate: 2.86 K/sec - SendersBlockedTimer: 5s758ms - SendersBlockedTotalTimer(*): 5s761ms Averaged Fragment F00:(Total: 31m9s, non-child: 10.131ms, % non-child: 0.00%) split sizes: min: 156.69 KB, max: 156.69 KB, avg: 156.69 KB, stddev: 0 completion times: min:31m9s max:31m9s mean: 31m9s stddev:0ns execution rates: min:85.00 B/sec max:85.00 B/sec mean:85.00 B/sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 1.23 - PeakMemoryUsage: 251.29 MB (263491584) - PerHostPeakMemUsage: 426.28 MB (446989168) - PrepareTime: 8.364ms - RowsProduced: 5.33M (5329000) - TotalCpuTime: 38m16s - TotalNetworkReceiveTime: 1s172ms - TotalNetworkSendTime: 5s495ms - TotalStorageWaitTime: 1s374ms DataStreamSender (dst_id=10):(Total: 6s723ms, non-child: 6s723ms, % non-child: 100.00%) - BytesSent: 21.91 MB (22973820) - NetworkThroughput(*): 3.33 MB/sec - OverallThroughput: 3.26 MB/sec - PeakMemoryUsage: 4.00 KB (4096) - SerializeBatchTime: 1s191ms - ThriftTransmitTime(*): 6s583ms - UncompressedRowBatchSize: 264.29 MB (277128864) SORT_NODE (id=5):(Total: 31m2s, non-child: 2m23s, % non-child: 7.72%) - InMemorySortTime: 2m18s - InitialRunsCreated: 1 - PeakMemoryUsage: 248.17 MB (260227072) - RowsReturned: 5.33M (5329000) - RowsReturnedRate: 2.86 K/sec - SortDataSize: 243.94 MB (255792000) - TotalMergesPerformed: 0 HASH_JOIN_NODE (id=4):(Total: 28m38s, non-child: 28m36s, % non-child: 99.90%) - BuildPartitionTime: 3.949ms - BuildRows: 7.30K (7300) - BuildRowsPartitioned: 7.30K (7300) - BuildTime: 2.854ms - GetNewBlockTime: 184.980us - HashBuckets: 20.48K (20480) - LargestPartitionPercent: 10 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 4.07 MB (4268032) - PinTime: 0ns - ProbeRows: 3.99M (3985800) - ProbeRowsPartitioned: 0 - ProbeTime: 28m37s - RowsReturned: 5.33M (5329000) - RowsReturnedRate: 3.10 K/sec - SpilledPartitions: 0 - UnpinTime: 26.816us EXCHANGE_NODE (id=9):(Total: 586.43ms, non-child: 586.43ms, % non-child: 100.00%) - BytesReceived: 31.46 KB (32220) - ConvertRowBatchTime: 1.260ms - DeserializeRowBatchTimer: 659.453us - FirstBatchArrivalWaitTime: 216.653ms - PeakMemoryUsage: 0 - RowsReturned: 7.30K (7300) - RowsReturnedRate: 12.46 K/sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns HASH_JOIN_NODE (id=3):(Total: 1s133ms, non-child: 528.952ms, % non-child: 46.69%) - BuildPartitionTime: 2.764ms - BuildRows: 7.30K (7300) - BuildRowsPartitioned: 7.30K (7300) - BuildTime: 2.975ms - GetNewBlockTime: 137.741us - HashBuckets: 24.58K (24576) - LargestPartitionPercent: 8 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 4.10 MB (4300800) - PinTime: 0ns - ProbeRows: 7.30K (7300) - ProbeRowsPartitioned: 0 - ProbeTime: 488.195ms - RowsReturned: 3.99M (3985800) - RowsReturnedRate: 3.52 M/sec - SpilledPartitions: 0 - UnpinTime: 8.823us EXCHANGE_NODE (id=8):(Total: 589.922ms, non-child: 589.922ms, % non-child: 100.00%) - BytesReceived: 82.40 KB (84378) - ConvertRowBatchTime: 1.162ms - DeserializeRowBatchTimer: 724.132us - FirstBatchArrivalWaitTime: 220.909ms - PeakMemoryUsage: 0 - RowsReturned: 7.30K (7300) - RowsReturnedRate: 12.37 K/sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns HDFS_SCAN_NODE (id=0):(Total: 14.134ms, non-child: 14.134ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.25 - BytesRead: 199.37 KB (204152) - BytesReadDataNodeCache: 0 - BytesReadLocal: 199.37 KB (204152) - BytesReadShortCircuit: 199.37 KB (204152) - DecompressionTime: 391.159us - MaxCompressedTextFileLength: 0 - NumColumns: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 13 - PeakMemoryUsage: 924.00 KB (946176) - PerReadThreadRawHdfsThroughput: 117.95 MB/sec - RowsRead: 7.30K (7300) - RowsReturned: 7.30K (7300) - RowsReturnedRate: 516.46 K/sec - ScanRangesComplete: 24 - ScannerThreadsInvoluntaryContextSwitches: 617 - ScannerThreadsTotalWallClockTime: 7m13s - MaterializeTupleTime(*): 3.279ms - ScannerThreadsSysTime: 712.0ms - ScannerThreadsUserTime: 824.0ms - ScannerThreadsVoluntaryContextSwitches: 300.86K (300861) - TotalRawHdfsReadTime(*): 1.650ms - TotalReadThroughput: 118.00 B/sec Averaged Fragment F02:(Total: 588.725ms, non-child: 861.458us, % non-child: 0.15%) split sizes: min: 156.69 KB, max: 156.69 KB, avg: 156.69 KB, stddev: 0 completion times: min:589.370ms max:589.370ms mean: 589.370ms stddev:0ns execution rates: min:265.86 KB/sec max:265.86 KB/sec mean:265.86 KB/sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 7.00 - PeakMemoryUsage: 207.00 KB (211968) - PerHostPeakMemUsage: 8.53 MB (8949072) - PrepareTime: 254.89us - RowsProduced: 7.30K (7300) - TotalCpuTime: 3s339ms - TotalNetworkReceiveTime: 0ns - TotalNetworkSendTime: 227.460us - TotalStorageWaitTime: 2s407ms DataStreamSender (dst_id=9):(Total: 3.22ms, non-child: 3.22ms, % non-child: 100.00%) - BytesSent: 31.47 KB (32224) - NetworkThroughput(*): 7.70 MB/sec - OverallThroughput: 10.17 MB/sec - PeakMemoryUsage: 8.00 KB (8192) - SerializeBatchTime: 2.575ms - ThriftTransmitTime(*): 3.989ms - UncompressedRowBatchSize: 142.68 KB (146100) HDFS_SCAN_NODE (id=2):(Total: 584.840ms, non-child: 584.840ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 6.00 - BytesRead: 164.08 KB (168018) - BytesReadDataNodeCache: 0 - BytesReadLocal: 164.08 KB (168018) - BytesReadShortCircuit: 164.08 KB (168018) - DecompressionTime: 393.358us - MaxCompressedTextFileLength: 0 - NumColumns: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 7 - PeakMemoryUsage: 191.00 KB (195584) - PerReadThreadRawHdfsThroughput: 14.16 MB/sec - RowsRead: 7.30K (7300) - RowsReturned: 7.30K (7300) - RowsReturnedRate: 12.48 K/sec - ScanRangesComplete: 24 - ScannerThreadsInvoluntaryContextSwitches: 40 - ScannerThreadsTotalWallClockTime: 3s339ms - MaterializeTupleTime(*): 4.219ms - ScannerThreadsSysTime: 16.0ms - ScannerThreadsUserTime: 36.0ms - ScannerThreadsVoluntaryContextSwitches: 2.08K (2079) - TotalRawHdfsReadTime(*): 11.312ms - TotalReadThroughput: 156.57 KB/sec Averaged Fragment F01:(Total: 589.333ms, non-child: 579.548us, % non-child: 0.10%) split sizes: min: 156.69 KB, max: 156.69 KB, avg: 156.69 KB, stddev: 0 completion times: min:589.665ms max:589.665ms mean: 589.665ms stddev:0ns execution rates: min:265.73 KB/sec max:265.73 KB/sec mean:265.73 KB/sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 6.00 - PeakMemoryUsage: 225.33 KB (230736) - PerHostPeakMemUsage: 8.53 MB (8949072) - PrepareTime: 204.358us - RowsProduced: 7.30K (7300) - TotalCpuTime: 2s910ms - TotalNetworkReceiveTime: 0ns - TotalNetworkSendTime: 168.91us - TotalStorageWaitTime: 1s929ms DataStreamSender (dst_id=8):(Total: 2.985ms, non-child: 2.985ms, % non-child: 100.00%) - BytesSent: 82.40 KB (84382) - NetworkThroughput(*): 18.36 MB/sec - OverallThroughput: 26.95 MB/sec - PeakMemoryUsage: 5.33 KB (5456) - SerializeBatchTime: 2.602ms - ThriftTransmitTime(*): 4.383ms - UncompressedRowBatchSize: 199.71 KB (204500) HDFS_SCAN_NODE (id=1):(Total: 585.767ms, non-child: 585.767ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 5.00 - BytesRead: 199.58 KB (204368) - BytesReadDataNodeCache: 0 - BytesReadLocal: 199.58 KB (204368) - BytesReadShortCircuit: 199.58 KB (204368) - DecompressionTime: 428.516us - MaxCompressedTextFileLength: 0 - NumColumns: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 212.00 KB (217088) - PerReadThreadRawHdfsThroughput: 75.99 MB/sec - RowsRead: 7.30K (7300) - RowsReturned: 7.30K (7300) - RowsReturnedRate: 12.46 K/sec - ScanRangesComplete: 24 - ScannerThreadsInvoluntaryContextSwitches: 22 - ScannerThreadsTotalWallClockTime: 2s910ms - MaterializeTupleTime(*): 3.822ms - ScannerThreadsSysTime: 8.0ms - ScannerThreadsUserTime: 40.0ms - ScannerThreadsVoluntaryContextSwitches: 1.74K (1737) - TotalRawHdfsReadTime(*): 2.564ms - TotalReadThroughput: 179.27 KB/sec Fragment F00: Instance 9744f2a34755b414:966207dc418a60b7 (host=mj:22000):(Total: 31m9s, non-child: 10.131ms, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:24/156.69 KB MemoryUsage(32s000ms): 11.88 MB, 14.26 MB, 20.11 MB, 23.97 MB, 28.08 MB, 33.80 MB, 36.05 MB, 43.51 MB, 45.26 MB, 51.98 MB, 54.97 MB, 59.94 MB, 64.56 MB, 67.92 MB, 74.39 MB, 76.14 MB, 83.86 MB, 85.86 MB, 91.83 MB, 95.70 MB, 99.80 MB, 105.53 MB, 107.76 MB, 115.37 MB, 117.23 MB, 123.70 MB, 127.08 MB, 131.67 MB, 136.79 MB, 139.64 MB, 146.62 MB, 149.11 MB, 155.58 MB, 158.95 MB, 163.54 MB, 168.79 MB, 171.51 MB, 178.63 MB, 180.61 MB, 187.47 MB, 190.45 MB, 195.43 MB, 200.29 MB, 203.39 MB, 210.13 MB, 212.10 MB, 219.35 MB, 221.81 MB, 227.31 MB, 231.79 MB, 235.27 MB, 241.63 MB, 243.48 MB, 250.74 MB, 248.07 MB, 248.07 MB, 248.07 MB, 248.07 MB ThreadUsage(32s000ms): 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.23 - PeakMemoryUsage: 251.29 MB (263491584) - PerHostPeakMemUsage: 426.28 MB (446989168) - PrepareTime: 8.364ms - RowsProduced: 5.33M (5329000) - TotalCpuTime: 38m16s - TotalNetworkReceiveTime: 1s172ms - TotalNetworkSendTime: 5s495ms - TotalStorageWaitTime: 1s374ms DataStreamSender (dst_id=10):(Total: 6s723ms, non-child: 6s723ms, % non-child: 100.00%) - BytesSent: 21.91 MB (22973820) - NetworkThroughput(*): 3.33 MB/sec - OverallThroughput: 3.26 MB/sec - PeakMemoryUsage: 4.00 KB (4096) - SerializeBatchTime: 1s191ms - ThriftTransmitTime(*): 6s583ms - UncompressedRowBatchSize: 264.29 MB (277128864) SORT_NODE (id=5):(Total: 31m2s, non-child: 2m23s, % non-child: 7.72%) - InMemorySortTime: 2m18s - InitialRunsCreated: 1 - PeakMemoryUsage: 248.17 MB (260227072) - RowsReturned: 5.33M (5329000) - RowsReturnedRate: 2.86 K/sec - SortDataSize: 243.94 MB (255792000) - TotalMergesPerformed: 0 HASH_JOIN_NODE (id=4):(Total: 28m38s, non-child: 28m37s, % non-child: 99.93%) ExecOption: Join Build-Side Prepared Asynchronously, Build-Side Filter Pushed Down - BuildPartitionTime: 3.949ms - BuildRows: 7.30K (7300) - BuildRowsPartitioned: 7.30K (7300) - BuildTime: 2.854ms - GetNewBlockTime: 184.980us - HashBuckets: 20.48K (20480) - LargestPartitionPercent: 10 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 4.07 MB (4268032) - PinTime: 0ns - ProbeRows: 3.99M (3985800) - ProbeRowsPartitioned: 0 - ProbeTime: 28m37s - RowsReturned: 5.33M (5329000) - RowsReturnedRate: 3.10 K/sec - SpilledPartitions: 0 - UnpinTime: 26.816us EXCHANGE_NODE (id=9):(Total: 586.43ms, non-child: 1.952ms, % non-child: 0.33%) BytesReceived(32s000ms): 30.89 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB, 31.46 KB - BytesReceived: 31.46 KB (32220) - ConvertRowBatchTime: 1.260ms - DeserializeRowBatchTimer: 659.453us - FirstBatchArrivalWaitTime: 216.653ms - PeakMemoryUsage: 0 - RowsReturned: 7.30K (7300) - RowsReturnedRate: 12.46 K/sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns HASH_JOIN_NODE (id=3):(Total: 1s133ms, non-child: 1s126ms, % non-child: 99.42%) ExecOption: Join Build-Side Prepared Asynchronously - BuildPartitionTime: 2.764ms - BuildRows: 7.30K (7300) - BuildRowsPartitioned: 7.30K (7300) - BuildTime: 2.975ms - GetNewBlockTime: 137.741us - HashBuckets: 24.58K (24576) - LargestPartitionPercent: 8 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 4.10 MB (4300800) - PinTime: 0ns - ProbeRows: 7.30K (7300) - ProbeRowsPartitioned: 0 - ProbeTime: 488.195ms - RowsReturned: 3.99M (3985800) - RowsReturnedRate: 3.52 M/sec - SpilledPartitions: 0 - UnpinTime: 8.823us EXCHANGE_NODE (id=8):(Total: 589.922ms, non-child: 1.811ms, % non-child: 0.31%) BytesReceived(32s000ms): 80.90 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB, 82.40 KB - BytesReceived: 82.40 KB (84378) - ConvertRowBatchTime: 1.162ms - DeserializeRowBatchTimer: 724.132us - FirstBatchArrivalWaitTime: 220.909ms - PeakMemoryUsage: 0 - RowsReturned: 7.30K (7300) - RowsReturnedRate: 12.37 K/sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns HDFS_SCAN_NODE (id=0):(Total: 14.134ms, non-child: 14.134ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:24/156.69 KB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% File Formats: PARQUET/SNAPPY:72 ExecOption: Codegen enabled: 0 out of 24 BytesRead(32s000ms): 193.14 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB, 199.37 KB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.25 - BytesRead: 199.37 KB (204152) - BytesReadDataNodeCache: 0 - BytesReadLocal: 199.37 KB (204152) - BytesReadShortCircuit: 199.37 KB (204152) - DecompressionTime: 391.159us - MaxCompressedTextFileLength: 0 - NumColumns: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 13 - PeakMemoryUsage: 924.00 KB (946176) - PerReadThreadRawHdfsThroughput: 117.95 MB/sec - RowsRead: 7.30K (7300) - RowsReturned: 7.30K (7300) - RowsReturnedRate: 516.46 K/sec - ScanRangesComplete: 24 - ScannerThreadsInvoluntaryContextSwitches: 617 - ScannerThreadsTotalWallClockTime: 7m13s - MaterializeTupleTime(*): 3.279ms - ScannerThreadsSysTime: 712.0ms - ScannerThreadsUserTime: 824.0ms - ScannerThreadsVoluntaryContextSwitches: 300.86K (300861) - TotalRawHdfsReadTime(*): 1.650ms - TotalReadThroughput: 118.00 B/sec Fragment F02: Instance 9744f2a34755b414:966207dc418a60b8 (host=mj:22000):(Total: 588.725ms, non-child: 861.458us, % non-child: 0.15%) Hdfs split stats (:<# splits>/): 0:24/156.69 KB MemoryUsage(500.0ms): 76.00 KB, 130.00 KB ThreadUsage(500.0ms): 8, 6 - AverageThreadTokens: 7.00 - PeakMemoryUsage: 207.00 KB (211968) - PerHostPeakMemUsage: 8.53 MB (8949072) - PrepareTime: 254.89us - RowsProduced: 7.30K (7300) - TotalCpuTime: 3s339ms - TotalNetworkReceiveTime: 0ns - TotalNetworkSendTime: 227.460us - TotalStorageWaitTime: 2s407ms DataStreamSender (dst_id=9):(Total: 3.22ms, non-child: 3.22ms, % non-child: 100.00%) - BytesSent: 31.47 KB (32224) - NetworkThroughput(*): 7.70 MB/sec - OverallThroughput: 10.17 MB/sec - PeakMemoryUsage: 8.00 KB (8192) - SerializeBatchTime: 2.575ms - ThriftTransmitTime(*): 3.989ms - UncompressedRowBatchSize: 142.68 KB (146100) HDFS_SCAN_NODE (id=2):(Total: 584.840ms, non-child: 584.840ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:24/156.69 KB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% File Formats: PARQUET/SNAPPY:72 ExecOption: Codegen enabled: 0 out of 24 BytesRead(500.0ms): 0, 156.57 KB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 6.00 - BytesRead: 164.08 KB (168018) - BytesReadDataNodeCache: 0 - BytesReadLocal: 164.08 KB (168018) - BytesReadShortCircuit: 164.08 KB (168018) - DecompressionTime: 393.358us - MaxCompressedTextFileLength: 0 - NumColumns: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 7 - PeakMemoryUsage: 191.00 KB (195584) - PerReadThreadRawHdfsThroughput: 14.16 MB/sec - RowsRead: 7.30K (7300) - RowsReturned: 7.30K (7300) - RowsReturnedRate: 12.48 K/sec - ScanRangesComplete: 24 - ScannerThreadsInvoluntaryContextSwitches: 40 - ScannerThreadsTotalWallClockTime: 3s339ms - MaterializeTupleTime(*): 4.219ms - ScannerThreadsSysTime: 16.0ms - ScannerThreadsUserTime: 36.0ms - ScannerThreadsVoluntaryContextSwitches: 2.08K (2079) - TotalRawHdfsReadTime(*): 11.312ms - TotalReadThroughput: 156.57 KB/sec Fragment F01: Instance 9744f2a34755b414:966207dc418a60b9 (host=mj:22000):(Total: 589.333ms, non-child: 579.548us, % non-child: 0.10%) Hdfs split stats (:<# splits>/): 0:24/156.69 KB MemoryUsage(500.0ms): 13.33 KB, 147.33 KB ThreadUsage(500.0ms): 6, 6 - AverageThreadTokens: 6.00 - PeakMemoryUsage: 225.33 KB (230736) - PerHostPeakMemUsage: 8.53 MB (8949072) - PrepareTime: 204.358us - RowsProduced: 7.30K (7300) - TotalCpuTime: 2s910ms - TotalNetworkReceiveTime: 0ns - TotalNetworkSendTime: 168.91us - TotalStorageWaitTime: 1s929ms DataStreamSender (dst_id=8):(Total: 2.985ms, non-child: 2.985ms, % non-child: 100.00%) - BytesSent: 82.40 KB (84382) - NetworkThroughput(*): 18.36 MB/sec - OverallThroughput: 26.95 MB/sec - PeakMemoryUsage: 5.33 KB (5456) - SerializeBatchTime: 2.602ms - ThriftTransmitTime(*): 4.383ms - UncompressedRowBatchSize: 199.71 KB (204500) HDFS_SCAN_NODE (id=1):(Total: 585.767ms, non-child: 585.767ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:24/156.69 KB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% File Formats: PARQUET/SNAPPY:72 ExecOption: Codegen enabled: 0 out of 24 BytesRead(500.0ms): 0, 179.27 KB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 5.00 - BytesRead: 199.58 KB (204368) - BytesReadDataNodeCache: 0 - BytesReadLocal: 199.58 KB (204368) - BytesReadShortCircuit: 199.58 KB (204368) - DecompressionTime: 428.516us - MaxCompressedTextFileLength: 0 - NumColumns: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 212.00 KB (217088) - PerReadThreadRawHdfsThroughput: 75.99 MB/sec - RowsRead: 7.30K (7300) - RowsReturned: 7.30K (7300) - RowsReturnedRate: 12.46 K/sec - ScanRangesComplete: 24 - ScannerThreadsInvoluntaryContextSwitches: 22 - ScannerThreadsTotalWallClockTime: 2s910ms - MaterializeTupleTime(*): 3.822ms - ScannerThreadsSysTime: 8.0ms - ScannerThreadsUserTime: 40.0ms - ScannerThreadsVoluntaryContextSwitches: 1.74K (1737) - TotalRawHdfsReadTime(*): 2.564ms - TotalReadThroughput: 179.27 KB/sec