Query (id=da4813cec32e72cf:b6423487b52af2a9): Summary: Session ID: e945daa4e787a9ed:8b6d49c1129a0289 Session Type: BEESWAX Start Time: 2016-06-03 08:54:37.389457000 End Time: 2016-06-03 08:56:21.579518000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.5.0-cdh5-INTERNAL DEBUG (build 2916cc2eb503af075caf22bf8354e64430e967f9) User: mikeb Connected User: mikeb Delegated User: Network Address: ::ffff:127.0.0.1:58819 Default Db: default Sql Statement: select o_orderdate, o_custkey, o_comment from tpch.orders Coordinator: mikeb-desktop:22000 Query Options (non default): Plan: ---------------- Estimated Per-Host Requirements: Memory=176.00MB VCores=1 01:EXCHANGE [UNPARTITIONED] | hosts=2 per-host-mem=unavailable | tuple-ids=0 row-size=99B cardinality=1500000 | 00:SCAN HDFS [tpch.orders, RANDOM] partitions=1/1 files=1 size=162.56MB table stats: 1500000 rows total column stats: all hosts=2 per-host-mem=176.00MB tuple-ids=0 row-size=99B cardinality=1500000 ---------------- Estimated Per-Host Mem: 184549376 Estimated Per-Host VCores: 1 Request Pool: default-pool Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ------------------------------------------------------------------------------------------------------- 01:EXCHANGE 1 180.166ms 180.166ms 1.50M 1.50M 0 -1.00 B UNPARTITIONED 00:SCAN HDFS 2 137.043ms 142.180ms 1.50M 1.50M 34.89 MB 176.00 MB tpch.orders Planner Timeline: 1.600ms - Analysis finished: 504.096us (504.096us) - Equivalence classes computed: 570.937us (66.841us) - Single node plan created: 1.020ms (449.099us) - Runtime filters computed: 1.034ms (13.982us) - Distributed plan created: 1.192ms (158.367us) - Planning finished: 1.600ms (408.092us) Query Timeline: 1m44s - Start execution: 35.460us (35.460us) - Planning finished: 2.287ms (2.252ms) - Submit for admission: 2.412ms (124.932us) - Completed admission: 2.449ms (36.564us) - Ready to start 2 remote fragments: 2.764ms (314.806us) - All 2 remote fragments started: 3.419ms (655.372us) - Rows available: 185.357ms (181.938ms) - First row fetched: 204.651ms (19.293ms) - Unregister query: 1m44s (1m43s) ImpalaServer: - ClientFetchWaitTimer: 1m41s - RowMaterializationTimer: 2s167ms Execution Profile da4813cec32e72cf:b6423487b52af2a9:(Total: 442.602ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed ------------------------------------------------------------------------------------------------------------------- Fragment start latencies: Count: 2, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 1ms Per Node Peak Memory Usage: mikeb-desktop:22001(34.95 MB) mikeb-desktop:22000(56.32 MB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Coordinator Fragment F01:(Total: 364.683ms, non-child: 2.788ms, % non-child: 0.76%) MemoryUsage(2s000ms): 21.36 MB, 21.36 MB, 21.35 MB, 21.35 MB, 21.35 MB, 21.35 MB, 21.35 MB, 21.35 MB, 21.35 MB, 21.36 MB, 21.36 MB, 21.36 MB, 21.37 MB, 21.37 MB, 21.37 MB, 21.36 MB, 21.36 MB, 21.37 MB, 21.37 MB, 21.37 MB, 21.37 MB, 21.37 MB, 21.37 MB, 21.37 MB, 21.37 MB, 21.37 MB, 21.37 MB, 21.36 MB, 21.36 MB, 21.36 MB, 21.36 MB, 21.36 MB, 21.36 MB, 21.36 MB, 21.35 MB, 21.36 MB, 21.36 MB, 21.35 MB, 21.36 MB, 21.36 MB, 21.35 MB, 21.36 MB, 21.36 MB, 21.36 MB, 21.36 MB, 20.59 MB, 17.59 MB, 14.46 MB, 11.25 MB, 8.09 MB, 4.96 MB, 1.83 MB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 21.38 MB (22415336) - PerHostPeakMemUsage: 0 - PrepareTime: 31.616us - RowsProduced: 1.50M (1500000) - TotalCpuTime: 1m43s - TotalNetworkReceiveTime: 193.008ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 361.895ms, non-child: 180.166ms, % non-child: 49.78%) BytesReceived(2s000ms): 10.79 MB, 12.23 MB, 13.70 MB, 15.22 MB, 16.70 MB, 18.17 MB, 19.65 MB, 21.13 MB, 22.61 MB, 24.08 MB, 25.57 MB, 27.09 MB, 28.57 MB, 30.05 MB, 31.54 MB, 33.03 MB, 34.49 MB, 36.01 MB, 37.49 MB, 38.97 MB, 40.49 MB, 41.98 MB, 43.45 MB, 44.95 MB, 46.47 MB, 47.94 MB, 49.42 MB, 50.90 MB, 52.38 MB, 53.86 MB, 55.33 MB, 56.81 MB, 58.34 MB, 59.82 MB, 61.29 MB, 62.77 MB, 64.25 MB, 65.73 MB, 67.25 MB, 68.74 MB, 70.21 MB, 71.69 MB, 73.17 MB, 74.64 MB, 76.16 MB, 77.25 MB, 77.30 MB, 77.30 MB, 77.30 MB, 77.30 MB, 77.30 MB, 77.30 MB - BytesReceived: 77.30 MB (81054747) - ConvertRowBatchTime: 162.204ms - DeserializeRowBatchTimer: 336.524ms - FirstBatchArrivalWaitTime: 181.727ms - PeakMemoryUsage: 0 - RowsReturned: 1.50M (1500000) - RowsReturnedRate: 4.14 M/sec - SendersBlockedTimer: 1m29s - SendersBlockedTotalTimer(*): 2m Averaged Fragment F00:(Total: 1m, non-child: 59s991ms, % non-child: 98.50%) split sizes: min: 34.56 MB, max: 128.00 MB, avg: 81.28 MB, stddev: 46.72 MB completion times: min:31s215ms max:1m30s mean: 1m stddev:29s748ms execution rates: min:1.11 MB/sec max:1.41 MB/sec mean:1.26 MB/sec stddev:155.66 KB/sec num instances: 2 - AverageThreadTokens: 1.87 - BloomFilterBytes: 0 - PeakMemoryUsage: 34.95 MB (36645544) - PerHostPeakMemUsage: 45.64 MB (47852664) - PrepareTime: 118.479ms - RowsProduced: 750.00K (750000) - TotalCpuTime: 56s141ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 1m - TotalStorageWaitTime: 9.606ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 172.322ms, non-child: 172.322ms, % non-child: 100.00%) - CodegenTime: 509.470us - CompileTime: 3.365ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.12 MB (2220000) - NumFunctions: 20 (20) - NumInstructions: 343 (343) - OptimizationTime: 51.040ms - PrepareTime: 117.589ms DataStreamSender (dst_id=1):(Total: 603.609ms, non-child: 603.609ms, % non-child: 100.00%) - BytesSent: 38.65 MB (40527373) - NetworkThroughput(*): 762.78 MB/sec - OverallThroughput: 62.80 MB/sec - PeakMemoryUsage: 2.66 KB (2728) - RowsReturned: 750.00K (750000) - SerializeBatchTime: 552.423ms - TransmitDataRPCTime: 50.976ms - UncompressedRowBatchSize: 79.05 MB (82888336) HDFS_SCAN_NODE (id=0):(Total: 137.043ms, non-child: 137.043ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.87 - BytesRead: 81.28 MB (85226592) - BytesReadDataNodeCache: 0 - BytesReadLocal: 81.28 MB (85226592) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 81.28 MB (85226592) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 34.89 MB (36585472) - PerReadThreadRawHdfsThroughput: 3.39 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 750.00K (750000) - RowsReturned: 750.00K (750000) - RowsReturnedRate: 5.36 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 79 (79) - ScannerThreadsTotalWallClockTime: 55s390ms - DelimiterParseTime: 304.675ms - MaterializeTupleTime(*): 37.121ms - ScannerThreadsSysTime: 2.000ms - ScannerThreadsUserTime: 368.000ms - ScannerThreadsVoluntaryContextSwitches: 640 (640) - TotalRawHdfsReadTime(*): 21.848ms - TotalReadThroughput: 1.26 MB/sec Fragment F00: Instance da4813cec32e72cf:b6423487b52af2ac (host=mikeb-desktop:22000):(Total: 1m30s, non-child: 1m29s, % non-child: 98.62%) Hdfs split stats (:<# splits>/): 0:1/128.00 MB MemoryUsage(2s000ms): 34.95 MB, 34.95 MB, 34.95 MB, 30.90 MB, 26.85 MB, 34.95 MB, 34.95 MB, 34.95 MB, 32.92 MB, 26.85 MB, 34.95 MB, 34.95 MB, 34.95 MB, 32.92 MB, 26.85 MB, 32.85 MB, 34.85 MB, 28.85 MB, 34.95 MB, 32.92 MB, 30.85 MB, 34.85 MB, 28.85 MB, 34.85 MB, 32.85 MB, 30.85 MB, 34.85 MB, 28.85 MB, 34.85 MB, 34.85 MB, 28.85 MB, 34.85 MB, 30.85 MB, 32.85 MB, 34.85 MB, 28.88 MB, 34.95 MB, 30.90 MB, 26.92 MB, 26.95 MB, 20.88 MB, 18.85 MB, 16.85 MB, 10.70 MB, 9.35 MB ThreadUsage(2s000ms): 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1 - AverageThreadTokens: 1.96 - BloomFilterBytes: 0 - PeakMemoryUsage: 34.95 MB (36645544) - PerHostPeakMemUsage: 56.32 MB (59059784) - PrepareTime: 118.664ms - RowsProduced: 1.18M (1181503) - TotalCpuTime: 1m28s - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 1m29s - TotalStorageWaitTime: 10.888ms CodeGen:(Total: 172.314ms, non-child: 172.314ms, % non-child: 100.00%) - CodegenTime: 561.486us - CompileTime: 3.367ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.12 MB (2220000) - NumFunctions: 20 (20) - NumInstructions: 343 (343) - OptimizationTime: 50.897ms - PrepareTime: 117.739ms DataStreamSender (dst_id=1):(Total: 936.347ms, non-child: 936.347ms, % non-child: 100.00%) - BytesSent: 60.89 MB (63852739) - NetworkThroughput(*): 754.90 MB/sec - OverallThroughput: 65.03 MB/sec - PeakMemoryUsage: 2.66 KB (2728) - RowsReturned: 1.18M (1181503) - SerializeBatchTime: 855.345ms - TransmitDataRPCTime: 80.665ms - UncompressedRowBatchSize: 124.54 MB (130591664) HDFS_SCAN_NODE (id=0):(Total: 142.180ms, non-child: 142.180ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 1 out of 1 Hdfs split stats (:<# splits>/): 0:1/128.00 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: TEXT/NONE:1 BytesRead(2s000ms): 40.00 MB, 40.00 MB, 40.00 MB, 40.00 MB, 40.00 MB, 48.00 MB, 48.00 MB, 48.00 MB, 48.00 MB, 48.00 MB, 56.00 MB, 56.00 MB, 56.00 MB, 56.00 MB, 56.00 MB, 62.00 MB, 64.00 MB, 64.00 MB, 72.00 MB, 72.00 MB, 76.00 MB, 80.00 MB, 80.00 MB, 88.00 MB, 88.00 MB, 92.00 MB, 96.00 MB, 96.00 MB, 104.00 MB, 104.00 MB, 106.00 MB, 112.00 MB, 112.00 MB, 118.00 MB, 120.00 MB, 122.00 MB, 128.00 MB, 128.00 MB, 128.00 MB, 128.00 MB, 128.00 MB, 128.00 MB, 128.00 MB, 128.00 MB, 128.00 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.96 - BytesRead: 128.00 MB (134218752) - BytesReadDataNodeCache: 0 - BytesReadLocal: 128.00 MB (134218752) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 128.00 MB (134218752) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 2 (2) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 34.89 MB (36585472) - PerReadThreadRawHdfsThroughput: 3.92 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 1.18M (1181503) - RowsReturned: 1.18M (1181503) - RowsReturnedRate: 8.31 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 131 (131) - ScannerThreadsTotalWallClockTime: 1m26s - DelimiterParseTime: 484.260ms - MaterializeTupleTime(*): 58.343ms - ScannerThreadsSysTime: 4.000ms - ScannerThreadsUserTime: 588.000ms - ScannerThreadsVoluntaryContextSwitches: 1.06K (1060) - TotalRawHdfsReadTime(*): 31.918ms - TotalReadThroughput: 1.41 MB/sec Instance da4813cec32e72cf:b6423487b52af2ab (host=mikeb-desktop:22001):(Total: 31s157ms, non-child: 30s582ms, % non-child: 98.15%) Hdfs split stats (:<# splits>/): 0:1/34.56 MB MemoryUsage(500.000ms): 30.95 MB, 30.95 MB, 30.95 MB, 30.95 MB, 30.95 MB, 30.95 MB, 30.95 MB, 30.95 MB, 30.95 MB, 30.95 MB, 30.95 MB, 30.95 MB, 30.95 MB, 30.95 MB, 30.95 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 22.85 MB, 14.85 MB, 14.85 MB, 14.85 MB, 14.85 MB, 14.85 MB, 14.85 MB, 14.85 MB, 14.85 MB, 14.85 MB, 14.85 MB, 14.85 MB, 14.85 MB, 14.85 MB, 14.69 MB, 14.47 MB, 14.31 MB, 14.09 MB, 13.92 MB, 13.71 MB, 13.54 MB, 5.32 MB, 5.16 MB, 4.94 MB, 4.78 MB, 4.56 MB, 4.34 MB, 4.17 MB ThreadUsage(500.000ms): 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.77 - BloomFilterBytes: 0 - PeakMemoryUsage: 34.95 MB (36645544) - PerHostPeakMemUsage: 34.95 MB (36645544) - PrepareTime: 118.294ms - RowsProduced: 318.50K (318497) - TotalCpuTime: 24s269ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 30s771ms - TotalStorageWaitTime: 8.324ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 172.331ms, non-child: 172.331ms, % non-child: 100.00%) - CodegenTime: 457.454us - CompileTime: 3.362ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.12 MB (2220000) - NumFunctions: 20 (20) - NumInstructions: 343 (343) - OptimizationTime: 51.182ms - PrepareTime: 117.440ms DataStreamSender (dst_id=1):(Total: 270.871ms, non-child: 270.871ms, % non-child: 100.00%) - BytesSent: 16.41 MB (17202008) - NetworkThroughput(*): 770.67 MB/sec - OverallThroughput: 60.56 MB/sec - PeakMemoryUsage: 2.66 KB (2728) - RowsReturned: 318.50K (318497) - SerializeBatchTime: 249.501ms - TransmitDataRPCTime: 21.286ms - UncompressedRowBatchSize: 33.56 MB (35185008) HDFS_SCAN_NODE (id=0):(Total: 131.905ms, non-child: 131.905ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 1 out of 1 Hdfs split stats (:<# splits>/): 0:1/34.56 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: TEXT/NONE:1 BytesRead(500.000ms): 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB, 34.56 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.77 - BytesRead: 34.56 MB (36234433) - BytesReadDataNodeCache: 0 - BytesReadLocal: 34.56 MB (36234433) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 34.56 MB (36234433) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 34.89 MB (36585472) - PerReadThreadRawHdfsThroughput: 2.86 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 318.50K (318497) - RowsReturned: 318.50K (318497) - RowsReturnedRate: 2.41 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 27 (27) - ScannerThreadsTotalWallClockTime: 23s835ms - DelimiterParseTime: 125.090ms - MaterializeTupleTime(*): 15.900ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 148.000ms - ScannerThreadsVoluntaryContextSwitches: 221 (221) - TotalRawHdfsReadTime(*): 11.779ms - TotalReadThroughput: 1.11 MB/sec