Query (id=994cd53bb0fef854:d949184600000000): Summary: Session ID: f14871189e75ce34:c6a27cda5f218cac Session Type: BEESWAX Start Time: 2017-05-23 18:52:12.189346000 End Time: 2017-05-23 18:52:21.368761000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.9.0-SNAPSHOT RELEASE (build 74f551bff4c21b8581671adc5aac356e791a9beb) User: systest@Hfoo Connected User: systest@Hfoo Delegated User: Network Address: 10.17.234.12:21969 Default Db: scan_primitives_tpch_3tb Sql Statement: select cast(l_shipdate as timestamp), count(*) from lineitem_kudu_ts where l_shipdate between "1992-01-01" and "1992-03-01" group by cast(l_shipdate as timestamp) limit 10 Coordinator: max2.hfoo:22000 Query Options (non default): MEM_LIMIT=161061273600,NUM_NODES=1,MT_DOP=0 Plan: ---------------- Per-Host Resource Reservation: Memory=264.00MB Per-Host Resource Estimates: Memory=10.00MB F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B | 01:AGGREGATE [FINALIZE] | output: count(*) | group by: CAST(l_shipdate AS TIMESTAMP) | limit: 10 | mem-estimate=10.00MB mem-reservation=264.00MB | tuple-ids=1 row-size=34B cardinality=10 | 00:SCAN KUDU [scan_primitives_tpch_3tb.lineitem_kudu_ts] kudu predicates: l_shipdate <= '1992-03-01', l_shipdate >= '1992-01-01' mem-estimate=0B mem-reservation=0B tuple-ids=0 row-size=26B cardinality=1761428521 ---------------- Estimated Per-Host Mem: 10485760 Per-Host Memory Reservation: 276824064 Request Pool: root.systest Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail -------------------------------------------------------------------------------------------------------------------------- 01:AGGREGATE 1 8s388ms 8s388ms 10 10 2.59 MB 10.00 MB FINALIZE 00:SCAN KUDU 1 475.922ms 475.922ms 110.72M 1.76B 13.56 MB 0 scan_primitives_tpch_3tb.li... Errors: Planner Timeline: 146.258ms - Analysis finished: 6.921ms (6.921ms) - Equivalence classes computed: 7.082ms (161.177us) - Single node plan created: 143.565ms (136.482ms) - Runtime filters computed: 143.618ms (53.219us) - Distributed plan created: 143.699ms (80.696us) - Lineage info computed: 143.790ms (91.187us) - Planning finished: 146.258ms (2.467ms) Query Timeline: 9s186ms - Query submitted: 259.224us (259.224us) - Planning finished: 149.348ms (149.089ms) - Submit for admission: 151.516ms (2.168ms) - Completed admission: 152.871ms (1.355ms) - Ready to start on 1 backends: 156.412ms (3.540ms) - All 1 execution backends (1 fragment instances) started: 158.927ms (2.515ms) - Rows available: 9s110ms (8s951ms) - First row fetched: 9s169ms (59.832ms) - Unregister query: 9s179ms (9.298ms) - ComputeScanRangeAssignmentTimer: 657.826us ImpalaServer: - ClientFetchWaitTimer: 65.735ms - RowMaterializationTimer: 3.359ms Execution Profile 994cd53bb0fef854:d949184600000000:(Total: 8s959ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- Backend startup latencies: Count: 1, min / max: 2ms / 2ms, 25th %-ile: 2ms, 50th %-ile: 2ms, 75th %-ile: 2ms, 90th %-ile: 2ms, 95th %-ile: 2ms, 99.9th %-ile: 2ms Per Node Peak Memory Usage: max2.hfoo:22000(16.10 MB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Averaged Fragment F00:(Total: 9s010ms, non-child: 62.412ms, % non-child: 0.69%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:9s013ms max:9s013ms mean: 9s013ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 16.10 MB (16886741) - PerHostPeakMemUsage: 16.10 MB (16886741) - RowsProduced: 10 (10) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 22s517ms - TotalThreadsInvoluntaryContextSwitches: 2.79K (2795) - TotalThreadsTotalWallClockTime: 18m22s - TotalThreadsSysTime: 1s302ms - TotalThreadsUserTime: 20s288ms - TotalThreadsVoluntaryContextSwitches: 112.76K (112762) Fragment Instance Lifecycle Timings: - ExecTime: 60.677ms - ExecTreeExecTime: 63.342us - OpenTime: 8s923ms - ExecTreeOpenTime: 8s863ms - PrepareTime: 26.794ms - ExecTreePrepareTime: 674.775us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 32 (32) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 120.00 GB (128849018880) - PeakMemoryUsage: 528.00 KB (540672) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 84.102ms, non-child: 84.102ms, % non-child: 100.00%) - CodegenTime: 2.330ms - CompileTime: 13.484ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2076176) - NumFunctions: 63 (63) - NumInstructions: 1.09K (1093) - OptimizationTime: 45.051ms - PeakMemoryUsage: 546.50 KB (559616) - PrepareTime: 25.101ms AGGREGATION_NODE (id=1):(Total: 8s864ms, non-child: 8s388ms, % non-child: 94.63%) - BuildTime: 8s334ms - GetNewBlockTime: 183.865us - GetResultsTime: 10.761us - HTResizeTime: 75.777ms - HashBuckets: 4.10K (4096) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 2.59 MB (2720896) - PinTime: 0.000ns - RowsRepartitioned: 0 (0) - RowsReturned: 10 (10) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) - UnpinTime: 1.617us KUDU_SCAN_NODE (id=0):(Total: 475.922ms, non-child: 475.922ms, % non-child: 100.00%) - BytesRead: 0 - KuduRemoteScanTokens: 80 (80) - NumScannerThreadsStarted: 140 (140) - PeakMemoryUsage: 13.56 MB (14221312) - RowsRead: 110.72M (110722748) - RowsReturned: 110.72M (110722748) - RowsReturnedRate: 232.65 M/sec - ScanRangesComplete: 140 (140) - ScannerThreadsInvoluntaryContextSwitches: 1.91K (1911) - ScannerThreadsTotalWallClockTime: 18m13s - MaterializeTupleTime(*): 409.659ms - ScannerThreadsSysTime: 1s106ms - ScannerThreadsUserTime: 11s811ms - ScannerThreadsVoluntaryContextSwitches: 112.42K (112425) - TotalKuduScanRoundTrips: 2.82K (2822) - TotalReadThroughput: 0.00 /sec Coordinator Fragment F00: Instance 994cd53bb0fef854:d949184600000000 (host=max2.hfoo:22000):(Total: 9s010ms, non-child: 62.412ms, % non-child: 0.69%) Hdfs split stats (:<# splits>/): MemoryUsage(500.000ms): 16.10 MB, 16.05 MB, 16.10 MB, 16.10 MB, 16.10 MB, 16.10 MB, 16.10 MB, 16.10 MB, 16.10 MB, 16.10 MB, 16.04 MB, 16.05 MB, 16.10 MB, 16.10 MB, 15.77 MB, 13.58 MB, 11.17 MB, 9.15 MB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 16.10 MB (16886741) - PerHostPeakMemUsage: 16.10 MB (16886741) - RowsProduced: 10 (10) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 22s517ms - TotalThreadsInvoluntaryContextSwitches: 2.79K (2795) - TotalThreadsTotalWallClockTime: 18m22s - TotalThreadsSysTime: 1s302ms - TotalThreadsUserTime: 20s288ms - TotalThreadsVoluntaryContextSwitches: 112.76K (112762) Fragment Instance Lifecycle Timings: - ExecTime: 60.677ms - ExecTreeExecTime: 63.342us - OpenTime: 8s923ms - ExecTreeOpenTime: 8s863ms - PrepareTime: 26.794ms - ExecTreePrepareTime: 674.775us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 32 (32) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 120.00 GB (128849018880) - PeakMemoryUsage: 528.00 KB (540672) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 84.102ms, non-child: 84.102ms, % non-child: 100.00%) - CodegenTime: 2.330ms - CompileTime: 13.484ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2076176) - NumFunctions: 63 (63) - NumInstructions: 1.09K (1093) - OptimizationTime: 45.051ms - PeakMemoryUsage: 546.50 KB (559616) - PrepareTime: 25.101ms AGGREGATION_NODE (id=1):(Total: 8s864ms, non-child: 8s388ms, % non-child: 94.63%) ExecOption: Codegen Enabled - BuildTime: 8s334ms - GetNewBlockTime: 183.865us - GetResultsTime: 10.761us - HTResizeTime: 75.777ms - HashBuckets: 4.10K (4096) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 2.59 MB (2720896) - PinTime: 0.000ns - RowsRepartitioned: 0 (0) - RowsReturned: 10 (10) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) - UnpinTime: 1.617us KUDU_SCAN_NODE (id=0):(Total: 475.922ms, non-child: 475.922ms, % non-child: 100.00%) BytesRead(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 - BytesRead: 0 - KuduRemoteScanTokens: 80 (80) - NumScannerThreadsStarted: 140 (140) - PeakMemoryUsage: 13.56 MB (14221312) - RowsRead: 110.72M (110722748) - RowsReturned: 110.72M (110722748) - RowsReturnedRate: 232.65 M/sec - ScanRangesComplete: 140 (140) - ScannerThreadsInvoluntaryContextSwitches: 1.91K (1911) - ScannerThreadsTotalWallClockTime: 18m13s - MaterializeTupleTime(*): 409.659ms - ScannerThreadsSysTime: 1s106ms - ScannerThreadsUserTime: 11s811ms - ScannerThreadsVoluntaryContextSwitches: 112.42K (112425) - TotalKuduScanRoundTrips: 2.82K (2822) - TotalReadThroughput: 0.00 /sec