Query (id=fe474137bf0d8467:b099e2da035aad9b) Summary Session ID: 3140bb298b086ea7:4a0c8403eb6a9cbf Session Type: HIVESERVER2 HiveServer2 Protocol Version: V6 Start Time: 2016-10-14 11:00:14.576785000 End Time: Query Type: QUERY Query State: CREATED Query Status: OK Impala Version: impalad version 2.5.0-cdh5.7.1 RELEASE (build 27a4325c18c2a01c7a8097681a0eccf6d4335ea1) User: x194245 Connected User: x194245 Delegated User: Network Address: 48.24.145.193:57947 Default Db: retpocd01 Sql Statement: SELECT `v_t_partcpnt_ext_dim`.`yyyymm_key`, `v_t_partcpnt_ext_dim`.`partcpnt_key`, (CASE WHEN ((`v_t_partcpnt_ext_dim`.`partcpnt_if_selt_bal_ind` = 'Y') OR (`v_t_partcpnt_ext_dim`.`partcpnt_if_trgt_bal_ind` = 'Y')) THEN 'Y' ELSE 'N' END) as `EXPR_0`, (CASE WHEN ((`v_t_partcpnt_ext_dim`.`partcpnt_if_selt_alloctn_ind` = 'Y') OR (`v_t_partcpnt_ext_dim`.`partcpnt_if_trgt_alloctn_ind` = 'Y')) THEN 'Y' ELSE 'N' END) as `EXPR_1` FROM `retpocd01`.`v_t_partcpnt_ext_dim` `v_t_partcpnt_ext_dim` Coordinator: paehowchq1d02.prudential.com:22000 Query Options (non default): Plan: ---------------- Estimated Per-Host Requirements: Memory=528.00MB VCores=1 WARNING: The following tables are missing relevant table and/or column statistics. retpocd01.v_t_partcpnt_ext_dim 01:EXCHANGE [UNPARTITIONED] | hosts=3 per-host-mem=unavailable | tuple-ids=0 row-size=68B cardinality=unavailable | 00:SCAN HDFS [retpocd01.v_t_partcpnt_ext_dim v_t_partcpnt_ext_dim, RANDOM] partitions=1/1 files=3 size=562.32MB table stats: unavailable column stats: unavailable hosts=3 per-host-mem=528.00MB tuple-ids=0 row-size=68B cardinality=unavailable ---------------- Estimated Per-Host Mem: 553648128 Estimated Per-Host VCores: 1 Tables Missing Stats: retpocd01.v_t_partcpnt_ext_dim Request Pool: root.retpocfad01 Admission result: Admitted immediately Planner Timeline Analysis finished: 1817090 Equivalence classes computed: 2051122 Single node plan created: 3568539 Runtime filters computed: 3617161 Distributed plan created: 4485292 Lineage info computed: 5176627 Planning finished: 6742245 Query Timeline Start execution: 66788 Planning finished: 8340660 Submit for admission: 8942417 Completed admission: 9054178 Ready to start 3 remote fragments: 9761209 All 3 remote fragments started: 12533321 Rows available: 48703132 ImpalaServer - ClientFetchWaitTimer: 0 - InactiveTotalTime: 0 - RowMaterializationTimer: 0 - TotalTime: 0 Execution Profile fe474137bf0d8467:b099e2da035aad9b Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node Targets Type Partition filter ----------------------------------------------------------- Fragment start latencies: Count: 3, 25th %-ile: 2ms, 50th %-ile: 2ms, 75th %-ile: 2ms, 90th %-ile: 2ms, 95th %-ile: 2ms, 99.9th %-ile: 2ms - FiltersReceived: 0 - FinalizationTimer: 0 - InactiveTotalTime: 0 - TotalTime: 39145032 Coordinator Fragment F01 - AverageThreadTokens: 0.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 33554432 - PerHostPeakMemUsage: 0 - PrepareTime: 50002 - RowsProduced: 0 - TotalCpuTime: 0 - TotalNetworkReceiveTime: 35442990 - TotalNetworkSendTime: 0 - TotalStorageWaitTime: 0 - TotalTime: 35744198 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 0 - BlocksRecycled: 0 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 150323855360 - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 EXCHANGE_NODE (id=1) - BytesReceived: 10591089 - ConvertRowBatchTime: 0 - DeserializeRowBatchTimer: 100227625 - FirstBatchArrivalWaitTime: 35434221 - InactiveTotalTime: 35437402 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 35445092 Averaged Fragment F00 split sizes: min: 184.00 MB, max: 191.37 MB, avg: 187.44 MB, stddev: 3.03 MB - AverageThreadTokens: 2.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 60249920 - PerHostPeakMemUsage: 71434730 - PrepareTime: 140084 - RowsProduced: 123562 - TotalCpuTime: 0 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 115008773 - TotalStorageWaitTime: 31801715 - TotalTime: 208577205 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 0 - BlocksRecycled: 0 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 150323855360 - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 DataStreamSender (dst_id=1) - BytesSent: 3562669 - InactiveTotalTime: 0 - NetworkThroughput(*): 135824398 - OverallThroughput: 43425854 - PeakMemoryUsage: 1632 - RowsReturned: 122538 - SerializeBatchTime: 55932405 - TotalTime: 82742068 - TransmitDataRPCTime: 26940821 - UncompressedRowBatchSize: 10873997 HDFS_SCAN_NODE (id=0) - AverageHdfsReadThreadConcurrency: 3.683251370815554E-4 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 42398188 - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 1801329 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 6 - NumDisksAccessed: 0 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 60158176 - PerReadThreadRawHdfsThroughput: 571819161 - RemoteScanRanges: 0 - RowsRead: 308906 - RowsReturned: 123562 - RowsReturnedRate: 3483532 - ScanRangesComplete: 0 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 0 - MaterializeTupleTime(*): 0 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 0 - ScannerThreadsVoluntaryContextSwitches: 0 - TotalRawHdfsReadTime(*): 74246774 - TotalReadThroughput: 46865 - TotalTime: 35499255 Fragment F00 Instance fe474137bf0d8467:b099e2da035aad9f (host=paehowchq1d03.prudential.com:22000) Hdfs split stats (:<# splits>/): 4:1/191.37 MB - AverageThreadTokens: 2.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 60193000 - PerHostPeakMemUsage: 60193000 - PrepareTime: 130093 - RowsProduced: 111616 - TotalCpuTime: 0 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 119806105 - TotalStorageWaitTime: 32126033 - TotalTime: 208060881 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 0 - BlocksRecycled: 0 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 150323855360 - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 DataStreamSender (dst_id=1) - BytesSent: 3290861 - InactiveTotalTime: 0 - NetworkThroughput(*): 116929573 - OverallThroughput: 41888286 - PeakMemoryUsage: 1632 - RowsReturned: 110592 - SerializeBatchTime: 50647171 - TotalTime: 78562798 - TransmitDataRPCTime: 28143958 - UncompressedRowBatchSize: 9822644 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Disabled Hdfs split stats (:<# splits>/): 4:1/191.37 MB - AverageHdfsReadThreadConcurrency: 5.515719801434088E-4 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 42363410 - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 1760785 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 6 - NumDisksAccessed: 0 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 60101256 - PerReadThreadRawHdfsThroughput: 543607260 - RemoteScanRanges: 0 - RowsRead: 296960 - RowsReturned: 111616 - RowsReturnedRate: 3119523 - ScanRangesComplete: 0 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 0 - MaterializeTupleTime(*): 0 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 0 - ScannerThreadsVoluntaryContextSwitches: 0 - TotalRawHdfsReadTime(*): 77930177 - TotalReadThroughput: 46732 - TotalTime: 35779824 Instance fe474137bf0d8467:b099e2da035aad9e (host=paehowchq1d02.prudential.com:22000) Hdfs split stats (:<# splits>/): 11:1/184.00 MB - AverageThreadTokens: 2.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 60281616 - PerHostPeakMemUsage: 93836048 - PrepareTime: 156373 - RowsProduced: 138240 - TotalCpuTime: 0 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 105587043 - TotalStorageWaitTime: 31380164 - TotalTime: 209041311 DataStreamSender (dst_id=1) - BytesSent: 3938331 - InactiveTotalTime: 0 - NetworkThroughput(*): 122888771 - OverallThroughput: 40285832 - PeakMemoryUsage: 1632 - RowsReturned: 137216 - SerializeBatchTime: 65842960 - TotalTime: 97759703 - TransmitDataRPCTime: 32047932 - UncompressedRowBatchSize: 12165660 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Disabled Hdfs split stats (:<# splits>/): 11:1/184.00 MB - AverageHdfsReadThreadConcurrency: 5.534034311012728E-4 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 42396810 - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 1911692 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 6 - NumDisksAccessed: 0 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 60189872 - PerReadThreadRawHdfsThroughput: 594028163 - RemoteScanRanges: 0 - RowsRead: 323584 - RowsReturned: 138240 - RowsReturnedRate: 3939561 - ScanRangesComplete: 0 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 0 - MaterializeTupleTime(*): 0 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 0 - ScannerThreadsVoluntaryContextSwitches: 0 - TotalRawHdfsReadTime(*): 71371717 - TotalReadThroughput: 46925 - TotalTime: 35090197 Instance fe474137bf0d8467:b099e2da035aad9d (host=paehowchq1d01.prudential.com:22000) Hdfs split stats (:<# splits>/): 1:1/186.96 MB - AverageThreadTokens: 2.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 60275144 - PerHostPeakMemUsage: 60275144 - PrepareTime: 133788 - RowsProduced: 120832 - TotalCpuTime: 0 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 119633172 - TotalStorageWaitTime: 31898948 - TotalTime: 208629425 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 0 - BlocksRecycled: 0 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 150323855360 - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 DataStreamSender (dst_id=1) - BytesSent: 3458816 - InactiveTotalTime: 0 - NetworkThroughput(*): 167654852 - OverallThroughput: 48103446 - PeakMemoryUsage: 1632 - RowsReturned: 119808 - SerializeBatchTime: 51307085 - TotalTime: 71903704 - TransmitDataRPCTime: 20630575 - UncompressedRowBatchSize: 10633688 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Disabled Hdfs split stats (:<# splits>/): 1:1/186.96 MB - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 42434345 - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 1731512 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 6 - NumDisksAccessed: 0 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 60183400 - PerReadThreadRawHdfsThroughput: 577822061 - RemoteScanRanges: 0 - RowsRead: 306176 - RowsReturned: 120832 - RowsReturnedRate: 3391514 - ScanRangesComplete: 0 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 0 - MaterializeTupleTime(*): 0 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 0 - ScannerThreadsVoluntaryContextSwitches: 0 - TotalRawHdfsReadTime(*): 73438430 - TotalReadThroughput: 46940 - TotalTime: 35627746