Query (id=f74fb20d9f37b8f3:2dc99d3387051091) Summary Session ID: 5e45af61483af402:125a81cc6e9680b9 Session Type: HIVESERVER2 HiveServer2 Protocol Version: V6 Start Time: 2016-10-14 11:00:14.796253000 End Time: Query Type: QUERY Query State: CREATED Query Status: OK Impala Version: impalad version 2.5.0-cdh5.7.1 RELEASE (build 27a4325c18c2a01c7a8097681a0eccf6d4335ea1) User: x194245 Connected User: x194245 Delegated User: Network Address: 48.24.145.193:57948 Default Db: retpocd01 Sql Statement: SELECT `v_plan_asset_no_acctg_fund_dim`.`plan_asset_key`, `v_plan_asset_no_acctg_fund_dim`.`yyyymm_key` FROM `retpocd01`.`v_plan_asset_no_acctg_fund_dim` `v_plan_asset_no_acctg_fund_dim` WHERE (NOT (`v_plan_asset_no_acctg_fund_dim`.`fund_invst` IS NULL)) Coordinator: paehowchq1d02.prudential.com:22000 Query Options (non default): Plan: ---------------- Estimated Per-Host Requirements: Memory=72.00MB VCores=1 WARNING: The following tables are missing relevant table and/or column statistics. retpocd01.v_plan_asset_no_acctg_fund_dim 01:EXCHANGE [UNPARTITIONED] | hosts=2 per-host-mem=unavailable | tuple-ids=0 row-size=23B cardinality=unavailable | 00:SCAN HDFS [retpocd01.v_plan_asset_no_acctg_fund_dim v_plan_asset_no_acctg_fund_dim, RANDOM] partitions=1/1 files=2 size=17.10MB predicates: (NOT (v_plan_asset_no_acctg_fund_dim.fund_invst IS NULL)) table stats: unavailable column stats: unavailable hosts=2 per-host-mem=72.00MB tuple-ids=0 row-size=23B cardinality=unavailable ---------------- Estimated Per-Host Mem: 75497472 Estimated Per-Host VCores: 1 Tables Missing Stats: retpocd01.v_plan_asset_no_acctg_fund_dim Request Pool: root.retpocfad01 Admission result: Admitted immediately Planner Timeline Analysis finished: 1135583 Equivalence classes computed: 1361684 Single node plan created: 3209658 Runtime filters computed: 3259457 Distributed plan created: 3859693 Lineage info computed: 4479986 Planning finished: 5784621 Query Timeline Start execution: 71880 Planning finished: 7953225 Submit for admission: 8570882 Completed admission: 8724936 Ready to start 2 remote fragments: 9230775 All 2 remote fragments started: 11437578 Rows available: 427537129 ImpalaServer - ClientFetchWaitTimer: 0 - InactiveTotalTime: 0 - RowMaterializationTimer: 0 - TotalTime: 0 Execution Profile f74fb20d9f37b8f3:2dc99d3387051091 Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node Targets Type Partition filter ----------------------------------------------------------- Fragment start latencies: Count: 2, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 2ms, 90th %-ile: 2ms, 95th %-ile: 2ms, 99.9th %-ile: 2ms - FiltersReceived: 0 - FinalizationTimer: 0 - InactiveTotalTime: 0 - TotalTime: 418522788 Coordinator Fragment F01 - AverageThreadTokens: 0.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 26969504 - PerHostPeakMemUsage: 0 - PrepareTime: 42811 - RowsProduced: 0 - TotalCpuTime: 0 - TotalNetworkReceiveTime: 415640467 - TotalNetworkSendTime: 0 - TotalStorageWaitTime: 0 - TotalTime: 415917746 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 0 - BlocksRecycled: 0 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 150323855360 - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 EXCHANGE_NODE (id=1) - BytesReceived: 10515955 - ConvertRowBatchTime: 0 - DeserializeRowBatchTimer: 82594871 - FirstBatchArrivalWaitTime: 415632069 - InactiveTotalTime: 415634882 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 415642332 Averaged Fragment F00 split sizes: min: 5.12 MB, max: 11.97 MB, avg: 8.55 MB, stddev: 3.42 MB - AverageThreadTokens: 1.4997301672962746 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 13000960 - PerHostPeakMemUsage: 13000960 - PrepareTime: 238512625 - RowsProduced: 321992 - TotalCpuTime: 282259142 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 292902447 - TotalStorageWaitTime: 14622017 - TotalTime: 604633132 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 0 - BlocksRecycled: 0 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 150323855360 - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - CodegenTime: 0 - CompileTime: 998866 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1925128 - OptimizationTime: 161597540 - PrepareTime: 236718273 - TotalTime: 400567307 DataStreamSender (dst_id=1) - BytesSent: 5266198 - InactiveTotalTime: 0 - NetworkThroughput(*): 153239871 - OverallThroughput: 63689272 - PeakMemoryUsage: 4096 - RowsReturned: 321480 - SerializeBatchTime: 49615309 - TotalTime: 85770089 - TransmitDataRPCTime: 36103238 - UncompressedRowBatchSize: 12236954 HDFS_SCAN_NODE (id=0) - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5 - BytesRead: 2402429 - BytesReadDataNodeCache: 0 - BytesReadLocal: 737531 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 737531 - DecompressionTime: 2583687 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 0 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 12943612 - PerReadThreadRawHdfsThroughput: 674480982 - RemoteScanRanges: 0 - RowsRead: 414664 - RowsReturned: 321992 - RowsReturnedRate: 649027 - ScanRangesComplete: 0 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 47421381 - MaterializeTupleTime(*): 38097804 - ScannerThreadsSysTime: 9498500 - ScannerThreadsUserTime: 29995000 - ScannerThreadsVoluntaryContextSwitches: 3 - TotalRawHdfsReadTime(*): 3464371 - TotalReadThroughput: 1797 - TotalTime: 496031460 Fragment F00 Instance f74fb20d9f37b8f3:2dc99d3387051094 (host=paehowchq1d03.prudential.com:22000) Hdfs split stats (:<# splits>/): 9:1/5.12 MB - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 11615848 - PerHostPeakMemUsage: 11615848 - PrepareTime: 238565350 - RowsProduced: 244624 - TotalCpuTime: 564518285 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 244181655 - TotalStorageWaitTime: 14370704 - TotalTime: 560384301 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 0 - BlocksRecycled: 0 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 150323855360 - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - CodegenTime: 0 - CompileTime: 1119661 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1925128 - OptimizationTime: 164788094 - PrepareTime: 236611612 - TotalTime: 403884427 DataStreamSender (dst_id=1) - BytesSent: 4114678 - InactiveTotalTime: 0 - NetworkThroughput(*): 100084018 - OverallThroughput: 43158785 - PeakMemoryUsage: 4096 - RowsReturned: 244624 - SerializeBatchTime: 54135740 - TotalTime: 95338133 - TransmitDataRPCTime: 41112238 - UncompressedRowBatchSize: 9296668 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 1 Hdfs split stats (:<# splits>/): 9:1/5.12 MB Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% File Formats: PARQUET/SNAPPY:3 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 1475062 - BytesReadDataNodeCache: 0 - BytesReadLocal: 1475062 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1475062 - DecompressionTime: 1778806 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 11546208 - PerReadThreadRawHdfsThroughput: 612107432 - RemoteScanRanges: 0 - RowsRead: 244624 - RowsReturned: 244624 - RowsReturnedRate: 493507 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 94842763 - MaterializeTupleTime(*): 76195609 - ScannerThreadsSysTime: 18997000 - ScannerThreadsUserTime: 59990000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 2409809 - TotalReadThroughput: 0 - TotalTime: 495684907 Instance f74fb20d9f37b8f3:2dc99d3387051093 (host=paehowchq1d01.prudential.com:22000) Hdfs split stats (:<# splits>/): 9:1/11.97 MB - AverageThreadTokens: 1.9994603345925526 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 14386072 - PerHostPeakMemUsage: 14386072 - PrepareTime: 238459901 - RowsProduced: 399360 - TotalCpuTime: 0 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 341623240 - TotalStorageWaitTime: 14873330 - TotalTime: 648881964 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 0 - BlocksRecycled: 0 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 150323855360 - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - CodegenTime: 0 - CompileTime: 878071 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1925128 - OptimizationTime: 158406987 - PrepareTime: 236824934 - TotalTime: 397250188 DataStreamSender (dst_id=1) - BytesSent: 6417718 - InactiveTotalTime: 0 - NetworkThroughput(*): 206395724 - OverallThroughput: 84219760 - PeakMemoryUsage: 4096 - RowsReturned: 398336 - SerializeBatchTime: 45094878 - TotalTime: 76202045 - TransmitDataRPCTime: 31094239 - UncompressedRowBatchSize: 15177240 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Enabled Hdfs split stats (:<# splits>/): 9:1/11.97 MB - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 3329797 - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 3388569 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 0 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 14341016 - PerReadThreadRawHdfsThroughput: 736854532 - RemoteScanRanges: 0 - RowsRead: 584704 - RowsReturned: 399360 - RowsReturnedRate: 804548 - ScanRangesComplete: 0 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 0 - MaterializeTupleTime(*): 0 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 0 - ScannerThreadsVoluntaryContextSwitches: 0 - TotalRawHdfsReadTime(*): 4518934 - TotalReadThroughput: 3595 - TotalTime: 496378014