Query (id=9a40a083f48316ec:a667b1e100000000): Summary: Session ID: f14871189e75ce34:c6a27cda5f218cac Session Type: BEESWAX Start Time: 2017-05-23 18:46:32.780608000 End Time: 2017-05-23 18:47:24.734111000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.9.0-SNAPSHOT RELEASE (build 74f551bff4c21b8581671adc5aac356e791a9beb) User: systest@Hfoo Connected User: systest@Hfoo Delegated User: Network Address: 10.17.234.12:21969 Default Db: scan_primitives_tpch_3tb Sql Statement: select l_commitdate, count(*) from lineitem_kudu_ts where l_shipdate between "1992-01-01" and "1992-03-01" group by l_commitdate limit 10 Coordinator: max2.hfoo:22000 Query Options (non default): MEM_LIMIT=161061273600,NUM_NODES=1,MT_DOP=0 Plan: ---------------- Per-Host Resource Reservation: Memory=264.00MB Per-Host Resource Estimates: Memory=10.00MB F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B | 01:AGGREGATE [FINALIZE] | output: count(*) | group by: l_commitdate | limit: 10 | mem-estimate=10.00MB mem-reservation=264.00MB | tuple-ids=1 row-size=24B cardinality=10 | 00:SCAN KUDU [scan_primitives_tpch_3tb.lineitem_kudu_ts] kudu predicates: l_shipdate <= '1992-03-01', l_shipdate >= '1992-01-01' mem-estimate=0B mem-reservation=0B tuple-ids=0 row-size=16B cardinality=1761428521 ---------------- Estimated Per-Host Mem: 10485760 Per-Host Memory Reservation: 276824064 Request Pool: root.systest Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ------------------------------------------------------------------------------------------------------------------------ 01:AGGREGATE 1 4s761ms 4s761ms 10 10 2.56 MB 10.00 MB FINALIZE 00:SCAN KUDU 1 46s785ms 46s785ms 110.72M 1.76B 4.83 MB 0 scan_primitives_tpch_3tb.li... Errors: Planner Timeline: 159.271ms - Analysis finished: 7.337ms (7.337ms) - Equivalence classes computed: 7.620ms (283.737us) - Single node plan created: 156.583ms (148.962ms) - Runtime filters computed: 156.640ms (57.299us) - Distributed plan created: 156.722ms (82.373us) - Lineage info computed: 156.818ms (95.811us) - Planning finished: 159.271ms (2.452ms) Query Timeline: 51s959ms - Query submitted: 335.493us (335.493us) - Planning finished: 162.230ms (161.895ms) - Submit for admission: 165.732ms (3.501ms) - Completed admission: 172.495ms (6.763ms) - Ready to start on 1 backends: 175.879ms (3.384ms) - All 1 execution backends (1 fragment instances) started: 178.478ms (2.599ms) - Rows available: 51s797ms (51s618ms) - First row fetched: 51s945ms (148.025ms) - Unregister query: 51s953ms (7.874ms) - ComputeScanRangeAssignmentTimer: 715.877us ImpalaServer: - ClientFetchWaitTimer: 153.231ms - RowMaterializationTimer: 2.643ms Execution Profile 9a40a083f48316ec:a667b1e100000000:(Total: 51s626ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- Backend startup latencies: Count: 1, min / max: 2ms / 2ms, 25th %-ile: 2ms, 50th %-ile: 2ms, 75th %-ile: 2ms, 90th %-ile: 2ms, 95th %-ile: 2ms, 99.9th %-ile: 2ms Per Node Peak Memory Usage: max2.hfoo:22000(7.38 MB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Averaged Fragment F00:(Total: 51s766ms, non-child: 150.113ms, % non-child: 0.29%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:51s768ms max:51s768ms mean: 51s768ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 7.38 MB (7733938) - PerHostPeakMemUsage: 7.38 MB (7733938) - RowsProduced: 10 (10) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 19s056ms - TotalThreadsInvoluntaryContextSwitches: 745.18K (745182) - TotalThreadsTotalWallClockTime: 1h58m - TotalThreadsSysTime: 1h6m - TotalThreadsUserTime: 50s951ms - TotalThreadsVoluntaryContextSwitches: 2.23M (2228896) Fragment Instance Lifecycle Timings: - ExecTime: 148.745ms - ExecTreeExecTime: 120.922us - OpenTime: 51s593ms - ExecTreeOpenTime: 51s546ms - PrepareTime: 24.175ms - ExecTreePrepareTime: 621.244us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 32 (32) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 120.00 GB (128849018880) - PeakMemoryUsage: 528.00 KB (540672) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 69.255ms, non-child: 69.255ms, % non-child: 100.00%) - CodegenTime: 1.842ms - CompileTime: 11.568ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2076176) - NumFunctions: 49 (49) - NumInstructions: 936 (936) - OptimizationTime: 34.495ms - PeakMemoryUsage: 468.00 KB (479232) - PrepareTime: 22.764ms AGGREGATION_NODE (id=1):(Total: 51s547ms, non-child: 4s761ms, % non-child: 9.24%) - BuildTime: 4s582ms - GetNewBlockTime: 176.155us - GetResultsTime: 12.162us - HTResizeTime: 95.993ms - HashBuckets: 4.10K (4096) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 2.56 MB (2689152) - PinTime: 0.000ns - RowsRepartitioned: 0 (0) - RowsReturned: 10 (10) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) - UnpinTime: 1.494us KUDU_SCAN_NODE (id=0):(Total: 46s785ms, non-child: 46s785ms, % non-child: 100.00%) - BytesRead: 0 - KuduRemoteScanTokens: 80 (80) - NumScannerThreadsStarted: 140 (140) - PeakMemoryUsage: 4.83 MB (5068800) - RowsRead: 110.72M (110722748) - RowsReturned: 110.72M (110722748) - RowsReturnedRate: 2.37 M/sec - ScanRangesComplete: 140 (140) - ScannerThreadsInvoluntaryContextSwitches: 745.16K (745156) - ScannerThreadsTotalWallClockTime: 1h57m - MaterializeTupleTime(*): 46s718ms - ScannerThreadsSysTime: 1h6m - ScannerThreadsUserTime: 42s878ms - ScannerThreadsVoluntaryContextSwitches: 2.15M (2149225) - TotalKuduScanRoundTrips: 1.92K (1918) - TotalReadThroughput: 0.00 /sec Coordinator Fragment F00: Instance 9a40a083f48316ec:a667b1e100000000 (host=max2.hfoo:22000):(Total: 51s766ms, non-child: 150.113ms, % non-child: 0.29%) Hdfs split stats (:<# splits>/): MemoryUsage(1s000ms): 4.09 MB, 5.90 MB, 5.91 MB, 5.92 MB, 5.89 MB, 5.92 MB, 6.07 MB, 5.91 MB, 5.89 MB, 5.89 MB, 5.89 MB, 5.89 MB, 5.92 MB, 5.89 MB, 5.94 MB, 5.91 MB, 5.92 MB, 5.91 MB, 5.89 MB, 5.88 MB, 5.98 MB, 5.89 MB, 5.90 MB, 5.89 MB, 5.89 MB, 6.03 MB, 5.89 MB, 5.90 MB, 5.89 MB, 5.89 MB, 5.89 MB, 5.90 MB, 5.89 MB, 5.89 MB, 5.90 MB, 5.89 MB, 5.89 MB, 5.90 MB, 5.89 MB, 5.91 MB, 5.89 MB, 5.91 MB, 5.89 MB, 5.89 MB, 6.00 MB, 6.01 MB, 5.89 MB, 5.96 MB, 5.94 MB, 5.89 MB, 5.88 MB, 3.98 MB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 7.38 MB (7733938) - PerHostPeakMemUsage: 7.38 MB (7733938) - RowsProduced: 10 (10) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 19s056ms - TotalThreadsInvoluntaryContextSwitches: 745.18K (745182) - TotalThreadsTotalWallClockTime: 1h58m - TotalThreadsSysTime: 1h6m - TotalThreadsUserTime: 50s951ms - TotalThreadsVoluntaryContextSwitches: 2.23M (2228896) Fragment Instance Lifecycle Timings: - ExecTime: 148.745ms - ExecTreeExecTime: 120.922us - OpenTime: 51s593ms - ExecTreeOpenTime: 51s546ms - PrepareTime: 24.175ms - ExecTreePrepareTime: 621.244us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 32 (32) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 120.00 GB (128849018880) - PeakMemoryUsage: 528.00 KB (540672) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 69.255ms, non-child: 69.255ms, % non-child: 100.00%) - CodegenTime: 1.842ms - CompileTime: 11.568ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2076176) - NumFunctions: 49 (49) - NumInstructions: 936 (936) - OptimizationTime: 34.495ms - PeakMemoryUsage: 468.00 KB (479232) - PrepareTime: 22.764ms AGGREGATION_NODE (id=1):(Total: 51s547ms, non-child: 4s761ms, % non-child: 9.24%) ExecOption: Codegen Enabled - BuildTime: 4s582ms - GetNewBlockTime: 176.155us - GetResultsTime: 12.162us - HTResizeTime: 95.993ms - HashBuckets: 4.10K (4096) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 2.56 MB (2689152) - PinTime: 0.000ns - RowsRepartitioned: 0 (0) - RowsReturned: 10 (10) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) - UnpinTime: 1.494us KUDU_SCAN_NODE (id=0):(Total: 46s785ms, non-child: 46s785ms, % non-child: 100.00%) BytesRead(1s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 - BytesRead: 0 - KuduRemoteScanTokens: 80 (80) - NumScannerThreadsStarted: 140 (140) - PeakMemoryUsage: 4.83 MB (5068800) - RowsRead: 110.72M (110722748) - RowsReturned: 110.72M (110722748) - RowsReturnedRate: 2.37 M/sec - ScanRangesComplete: 140 (140) - ScannerThreadsInvoluntaryContextSwitches: 745.16K (745156) - ScannerThreadsTotalWallClockTime: 1h57m - MaterializeTupleTime(*): 46s718ms - ScannerThreadsSysTime: 1h6m - ScannerThreadsUserTime: 42s878ms - ScannerThreadsVoluntaryContextSwitches: 2.15M (2149225) - TotalKuduScanRoundTrips: 1.92K (1918) - TotalReadThroughput: 0.00 /sec