Query (id=c24437489cb126d3:a3d516366656f680): Summary: Session ID: 954438aa802c812a:b91d0f11bcddc88c Session Type: BEESWAX Start Time: 2016-06-03 08:37:53.417973000 End Time: 2016-06-03 08:39:40.274687000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.5.0-cdh5-INTERNAL DEBUG (build 2916cc2eb503af075caf22bf8354e64430e967f9) User: mikeb Connected User: mikeb Delegated User: Network Address: ::ffff:127.0.0.1:58661 Default Db: default Sql Statement: select o_orderdate, o_custkey, o_comment from tpch.orders order by o_orderdate Coordinator: mikeb-desktop:22000 Query Options (non default): Plan: ---------------- Estimated Per-Host Requirements: Memory=224.00MB VCores=1 02:MERGING-EXCHANGE [UNPARTITIONED] | order by: o_orderdate ASC | hosts=2 per-host-mem=unavailable | tuple-ids=1 row-size=99B cardinality=1500000 | 01:SORT | order by: o_orderdate ASC | hosts=2 per-host-mem=48.00MB | tuple-ids=1 row-size=99B cardinality=1500000 | 00:SCAN HDFS [tpch.orders, RANDOM] partitions=1/1 files=1 size=162.56MB table stats: 1500000 rows total column stats: all hosts=2 per-host-mem=176.00MB tuple-ids=0 row-size=99B cardinality=1500000 ---------------- Estimated Per-Host Mem: 234881024 Estimated Per-Host VCores: 1 Request Pool: default-pool Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail --------------------------------------------------------------------------------------------------------------- 02:MERGING-EXCHANGE 1 474.758ms 474.758ms 1.50M 1.50M 0 -1.00 B UNPARTITIONED 01:SORT 2 1s500ms 2s368ms 1.50M 1.50M 144.15 MB 48.00 MB 00:SCAN HDFS 2 127.799ms 129.675ms 1.50M 1.50M 34.89 MB 176.00 MB tpch.orders Planner Timeline: 4.964ms - Analysis finished: 1.320ms (1.320ms) - Equivalence classes computed: 1.433ms (113.343us) - Single node plan created: 2.542ms (1.109ms) - Runtime filters computed: 2.559ms (16.687us) - Distributed plan created: 2.788ms (229.366us) - Planning finished: 4.964ms (2.175ms) Query Timeline: 1m46s - Start execution: 36.324us (36.324us) - Planning finished: 6.210ms (6.174ms) - Submit for admission: 6.353ms (143.358us) - Completed admission: 6.391ms (38.146us) - Ready to start 2 remote fragments: 6.848ms (456.472us) - All 2 remote fragments started: 7.574ms (725.882us) - Rows available: 2s483ms (2s475ms) - First row fetched: 2s518ms (35.016ms) - Unregister query: 1m46s (1m44s) ImpalaServer: - ClientFetchWaitTimer: 1m41s - RowMaterializationTimer: 2s080ms Execution Profile c24437489cb126d3:a3d516366656f680:(Total: 3s039ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed ------------------------------------------------------------------------------------------------------------------- Fragment start latencies: Count: 2, 25th %-ile: 0, 50th %-ile: 0, 75th %-ile: 0, 90th %-ile: 0, 95th %-ile: 0, 99.9th %-ile: 0 Per Node Peak Memory Usage: mikeb-desktop:22001(146.96 MB) mikeb-desktop:22002(62.96 MB) mikeb-desktop:22000(0) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Coordinator Fragment F01:(Total: 2s952ms, non-child: 2.709ms, % non-child: 0.09%) MemoryUsage(2s000ms): 15.67 MB, 23.42 MB, 23.53 MB, 23.55 MB, 23.55 MB, 23.55 MB, 23.55 MB, 23.55 MB, 23.55 MB, 23.55 MB, 23.55 MB, 23.57 MB, 23.57 MB, 23.55 MB, 23.58 MB, 23.52 MB, 23.52 MB, 23.52 MB, 23.48 MB, 23.45 MB, 23.45 MB, 23.42 MB, 23.48 MB, 23.60 MB, 23.52 MB, 23.48 MB, 23.56 MB, 23.55 MB, 23.53 MB, 23.53 MB, 23.62 MB, 23.59 MB, 23.46 MB, 23.45 MB, 23.46 MB, 23.59 MB, 23.61 MB, 23.57 MB, 23.64 MB, 23.62 MB, 23.49 MB, 23.57 MB, 23.59 MB, 23.60 MB, 23.57 MB, 23.57 MB, 21.63 MB, 18.51 MB, 15.39 MB, 12.21 MB, 9.00 MB, 5.84 MB, 2.71 MB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 23.76 MB (24918784) - PerHostPeakMemUsage: 0 - PrepareTime: 48.828us - RowsProduced: 1.50M (1500000) - TotalCpuTime: 1m46s - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns EXCHANGE_NODE (id=2):(Total: 2s950ms, non-child: 474.758ms, % non-child: 16.09%) BytesReceived(2s000ms): 6.77 MB, 10.56 MB, 11.98 MB, 13.34 MB, 14.68 MB, 16.02 MB, 17.39 MB, 18.76 MB, 20.11 MB, 21.46 MB, 22.80 MB, 24.19 MB, 25.54 MB, 26.88 MB, 28.24 MB, 29.56 MB, 30.91 MB, 32.24 MB, 33.54 MB, 34.85 MB, 36.20 MB, 37.54 MB, 38.88 MB, 40.28 MB, 41.60 MB, 42.92 MB, 44.30 MB, 45.65 MB, 46.97 MB, 48.31 MB, 49.70 MB, 51.07 MB, 52.37 MB, 53.72 MB, 55.06 MB, 56.45 MB, 57.80 MB, 59.17 MB, 60.54 MB, 61.87 MB, 63.12 MB, 64.49 MB, 65.85 MB, 67.19 MB, 68.52 MB, 69.85 MB, 70.32 MB, 70.32 MB, 70.32 MB, 70.32 MB, 70.32 MB, 70.32 MB, 70.32 MB - BytesReceived: 70.32 MB (73736963) - ConvertRowBatchTime: 0.000ns - DeserializeRowBatchTimer: 322.328ms - FirstBatchArrivalWaitTime: 2s475ms - MergeGetNext: 470.851ms - MergeGetNextBatch: 14.997ms - PeakMemoryUsage: 0 - RowsReturned: 1.50M (1500000) - RowsReturnedRate: 508.43 K/sec - SendersBlockedTimer: 1m30s - SendersBlockedTotalTimer(*): 2m4s Averaged Fragment F00:(Total: 1m4s, non-child: 1m2s, % non-child: 96.29%) split sizes: min: 34.56 MB, max: 128.00 MB, avg: 81.28 MB, stddev: 46.72 MB completion times: min:37s167ms max:1m31s mean: 1m4s stddev:27s366ms execution rates: min:952.05 KB/sec max:1.39 MB/sec mean:1.16 MB/sec stddev:237.10 KB/sec num instances: 2 - AverageThreadTokens: 1.01 - BloomFilterBytes: 0 - PeakMemoryUsage: 104.96 MB (110058152) - PerHostPeakMemUsage: 104.96 MB (110058152) - PrepareTime: 116.276ms - RowsProduced: 750.00K (750000) - TotalCpuTime: 2s700ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 1m2s - TotalStorageWaitTime: 9.360ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 11 (11) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 92.00 MB (96468992) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 199.640ms, non-child: 199.640ms, % non-child: 100.00%) - CodegenTime: 585.029us - CompileTime: 4.171ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.12 MB (2220000) - NumFunctions: 47 (47) - NumInstructions: 891 (891) - OptimizationTime: 79.900ms - PrepareTime: 115.221ms DataStreamSender (dst_id=2):(Total: 565.837ms, non-child: 565.837ms, % non-child: 100.00%) - BytesSent: 35.16 MB (36868481) - NetworkThroughput(*): 759.80 MB/sec - OverallThroughput: 62.44 MB/sec - PeakMemoryUsage: 2.66 KB (2728) - RowsReturned: 750.00K (750000) - SerializeBatchTime: 518.840ms - TransmitDataRPCTime: 46.821ms - UncompressedRowBatchSize: 79.05 MB (82888340) SORT_NODE (id=1):(Total: 1s628ms, non-child: 1s500ms, % non-child: 92.15%) - InMemorySortTime: 944.185ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 100.10 MB (104964096) - RowsReturned: 750.00K (750000) - RowsReturnedRate: 446.66 K/sec - SortDataSize: 76.18 MB (79885404) - TotalMergesPerformed: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 127.799ms, non-child: 127.799ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.50 - BytesRead: 81.28 MB (85226592) - BytesReadDataNodeCache: 0 - BytesReadLocal: 81.28 MB (85226592) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 81.28 MB (85226592) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 34.84 MB (36528128) - PerReadThreadRawHdfsThroughput: 3.37 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 750.00K (750000) - RowsReturned: 750.00K (750000) - RowsReturnedRate: 5.82 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 44 (44) - ScannerThreadsTotalWallClockTime: 466.557ms - DelimiterParseTime: 286.357ms - MaterializeTupleTime(*): 35.616ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 330.000ms - ScannerThreadsVoluntaryContextSwitches: 503 (503) - TotalRawHdfsReadTime(*): 23.137ms - TotalReadThroughput: 64.00 MB/sec Fragment F00: Instance c24437489cb126d3:a3d516366656f683 (host=mikeb-desktop:22001):(Total: 1m31s, non-child: 1m28s, % non-child: 96.09%) Hdfs split stats (:<# splits>/): 0:1/128.00 MB MemoryUsage(2s000ms): 123.44 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB, 136.02 MB ThreadUsage(2s000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.01 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.96 MB (154094248) - PerHostPeakMemUsage: 146.96 MB (154094248) - PrepareTime: 116.782ms - RowsProduced: 1.18M (1181503) - TotalCpuTime: 4s152ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 1m28s - TotalStorageWaitTime: 10.648ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 17 (17) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 136.00 MB (142606336) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 200.069ms, non-child: 200.069ms, % non-child: 100.00%) - CodegenTime: 510.146us - CompileTime: 4.187ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.12 MB (2220000) - NumFunctions: 47 (47) - NumInstructions: 891 (891) - OptimizationTime: 79.721ms - PrepareTime: 115.848ms DataStreamSender (dst_id=2):(Total: 893.807ms, non-child: 893.807ms, % non-child: 100.00%) - BytesSent: 55.35 MB (58036509) - NetworkThroughput(*): 744.72 MB/sec - OverallThroughput: 61.92 MB/sec - PeakMemoryUsage: 2.66 KB (2728) - RowsReturned: 1.18M (1181503) - SerializeBatchTime: 819.261ms - TransmitDataRPCTime: 74.319ms - UncompressedRowBatchSize: 124.54 MB (130591672) SORT_NODE (id=1):(Total: 2s498ms, non-child: 2s368ms, % non-child: 94.81%) ExecOption: Codegen Enabled - InMemorySortTime: 1s501ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 144.15 MB (151150592) - RowsReturned: 1.18M (1181503) - RowsReturnedRate: 472.87 K/sec - SortDataSize: 120.03 MB (125861036) - TotalMergesPerformed: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 129.675ms, non-child: 129.675ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 1 out of 1 Hdfs split stats (:<# splits>/): 0:1/128.00 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: TEXT/NONE:1 BytesRead(500.000ms): 64.00 MB, 128.00 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 128.00 MB (134218752) - BytesReadDataNodeCache: 0 - BytesReadLocal: 128.00 MB (134218752) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 128.00 MB (134218752) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 2 (2) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 34.89 MB (36585472) - PerReadThreadRawHdfsThroughput: 3.48 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 1.18M (1181503) - RowsReturned: 1.18M (1181503) - RowsReturnedRate: 9.11 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 65 (65) - ScannerThreadsTotalWallClockTime: 739.702ms - DelimiterParseTime: 431.276ms - MaterializeTupleTime(*): 53.335ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 500.000ms - ScannerThreadsVoluntaryContextSwitches: 922 (922) - TotalRawHdfsReadTime(*): 35.881ms - TotalReadThroughput: 128.00 MB/sec Instance c24437489cb126d3:a3d516366656f682 (host=mikeb-desktop:22002):(Total: 37s081ms, non-child: 35s886ms, % non-child: 96.78%) Hdfs split stats (:<# splits>/): 0:1/34.56 MB MemoryUsage(1s000ms): 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB, 48.02 MB ThreadUsage(1s000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 62.96 MB (66022056) - PerHostPeakMemUsage: 62.96 MB (66022056) - PrepareTime: 115.771ms - RowsProduced: 318.50K (318497) - TotalCpuTime: 1s249ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 36s102ms - TotalStorageWaitTime: 8.072ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 6 (6) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 48.00 MB (50331648) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 199.211ms, non-child: 199.211ms, % non-child: 100.00%) - CodegenTime: 659.912us - CompileTime: 4.155ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.12 MB (2220000) - NumFunctions: 47 (47) - NumInstructions: 891 (891) - OptimizationTime: 80.079ms - PrepareTime: 114.595ms DataStreamSender (dst_id=2):(Total: 237.866ms, non-child: 237.866ms, % non-child: 100.00%) - BytesSent: 14.97 MB (15700454) - NetworkThroughput(*): 774.88 MB/sec - OverallThroughput: 62.95 MB/sec - PeakMemoryUsage: 2.66 KB (2728) - RowsReturned: 318.50K (318497) - SerializeBatchTime: 218.418ms - TransmitDataRPCTime: 19.323ms - UncompressedRowBatchSize: 33.56 MB (35185008) SORT_NODE (id=1):(Total: 757.527ms, non-child: 631.603ms, % non-child: 83.38%) ExecOption: Codegen Enabled - InMemorySortTime: 386.508ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 56.05 MB (58777600) - RowsReturned: 318.50K (318497) - RowsReturnedRate: 420.44 K/sec - SortDataSize: 32.34 MB (33909772) - TotalMergesPerformed: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 125.923ms, non-child: 125.923ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 1 out of 1 Hdfs split stats (:<# splits>/): 0:1/34.56 MB Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: TEXT/NONE:1 - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 34.56 MB (36234433) - BytesReadDataNodeCache: 0 - BytesReadLocal: 34.56 MB (36234433) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 34.56 MB (36234433) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 34.78 MB (36470784) - PerReadThreadRawHdfsThroughput: 3.25 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 318.50K (318497) - RowsReturned: 318.50K (318497) - RowsReturnedRate: 2.53 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 23 (23) - ScannerThreadsTotalWallClockTime: 193.412ms - DelimiterParseTime: 141.438ms - MaterializeTupleTime(*): 17.898ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 160.000ms - ScannerThreadsVoluntaryContextSwitches: 85 (85) - TotalRawHdfsReadTime(*): 10.394ms - TotalReadThroughput: 0.00 /sec