Query (id=3b4103fe92884226:7e6bbf6a00000000): Summary: Session ID: f14871189e75ce34:c6a27cda5f218cac Session Type: BEESWAX Start Time: 2017-05-23 18:49:04.071392000 End Time: 2017-05-23 18:49:08.109339000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.9.0-SNAPSHOT RELEASE (build 74f551bff4c21b8581671adc5aac356e791a9beb) User: systest@Hfoo Connected User: systest@Hfoo Delegated User: Network Address: 10.17.234.12:21969 Default Db: scan_primitives_tpch_3tb Sql Statement: select l_shipdate, count(*) from lineitem_kudu_ts where l_shipdate between "1992-01-01" and "1992-03-01" group by l_shipdate limit 10 Coordinator: max2.hfoo:22000 Query Options (non default): MEM_LIMIT=161061273600,NUM_NODES=1,MT_DOP=0 Plan: ---------------- Per-Host Resource Reservation: Memory=264.00MB Per-Host Resource Estimates: Memory=10.00MB F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B | 01:AGGREGATE [FINALIZE] | output: count(*) | group by: l_shipdate | limit: 10 | mem-estimate=10.00MB mem-reservation=264.00MB | tuple-ids=1 row-size=34B cardinality=10 | 00:SCAN KUDU [scan_primitives_tpch_3tb.lineitem_kudu_ts] kudu predicates: l_shipdate <= '1992-03-01', l_shipdate >= '1992-01-01' mem-estimate=0B mem-reservation=0B tuple-ids=0 row-size=26B cardinality=1761428521 ---------------- Estimated Per-Host Mem: 10485760 Per-Host Memory Reservation: 276824064 Request Pool: root.systest Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail -------------------------------------------------------------------------------------------------------------------------- 01:AGGREGATE 1 3s292ms 3s292ms 10 10 2.59 MB 10.00 MB FINALIZE 00:SCAN KUDU 1 462.321ms 462.321ms 110.72M 1.76B 13.56 MB 0 scan_primitives_tpch_3tb.li... Errors: Planner Timeline: 154.244ms - Analysis finished: 10.443ms (10.443ms) - Equivalence classes computed: 10.663ms (219.365us) - Single node plan created: 151.529ms (140.866ms) - Runtime filters computed: 151.588ms (58.558us) - Distributed plan created: 151.674ms (86.202us) - Lineage info computed: 151.780ms (105.694us) - Planning finished: 154.244ms (2.464ms) Query Timeline: 4s043ms - Query submitted: 313.980us (313.980us) - Planning finished: 157.182ms (156.868ms) - Submit for admission: 159.884ms (2.701ms) - Completed admission: 161.373ms (1.489ms) - Ready to start on 1 backends: 164.581ms (3.208ms) - All 1 execution backends (1 fragment instances) started: 167.248ms (2.666ms) - Rows available: 4s001ms (3s834ms) - First row fetched: 4s029ms (28.327ms) - Unregister query: 4s037ms (8.068ms) - ComputeScanRangeAssignmentTimer: 838.382us ImpalaServer: - ClientFetchWaitTimer: 33.879ms - RowMaterializationTimer: 2.493ms Execution Profile 3b4103fe92884226:7e6bbf6a00000000:(Total: 3s841ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- Backend startup latencies: Count: 1, min / max: 1ms / 1ms, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 1ms Per Node Peak Memory Usage: max2.hfoo:22000(16.10 MB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Averaged Fragment F00:(Total: 3s862ms, non-child: 30.268ms, % non-child: 0.78%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:3s864ms max:3s864ms mean: 3s864ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 16.10 MB (16886622) - PerHostPeakMemUsage: 16.10 MB (16886622) - RowsProduced: 10 (10) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 25s811ms - TotalThreadsInvoluntaryContextSwitches: 3.23K (3228) - TotalThreadsTotalWallClockTime: 8m11s - TotalThreadsSysTime: 1s282ms - TotalThreadsUserTime: 14s425ms - TotalThreadsVoluntaryContextSwitches: 115.25K (115249) Fragment Instance Lifecycle Timings: - ExecTime: 28.881ms - ExecTreeExecTime: 38.533us - OpenTime: 3s807ms - ExecTreeOpenTime: 3s754ms - PrepareTime: 25.544ms - ExecTreePrepareTime: 697.821us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 32 (32) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 120.00 GB (128849018880) - PeakMemoryUsage: 528.00 KB (540672) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 77.114ms, non-child: 77.114ms, % non-child: 100.00%) - CodegenTime: 1.716ms - CompileTime: 15.530ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2076176) - NumFunctions: 41 (41) - NumInstructions: 729 (729) - OptimizationTime: 37.082ms - PeakMemoryUsage: 364.50 KB (373248) - PrepareTime: 24.066ms AGGREGATION_NODE (id=1):(Total: 3s754ms, non-child: 3s292ms, % non-child: 87.69%) - BuildTime: 3s232ms - GetNewBlockTime: 200.600us - GetResultsTime: 12.736us - HTResizeTime: 83.579ms - HashBuckets: 4.10K (4096) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 2.59 MB (2720896) - PinTime: 0.000ns - RowsRepartitioned: 0 (0) - RowsReturned: 10 (10) - RowsReturnedRate: 2.00 /sec - SpilledPartitions: 0 (0) - UnpinTime: 1.428us KUDU_SCAN_NODE (id=0):(Total: 462.321ms, non-child: 462.321ms, % non-child: 100.00%) - BytesRead: 0 - KuduRemoteScanTokens: 80 (80) - NumScannerThreadsStarted: 138 (138) - PeakMemoryUsage: 13.56 MB (14221312) - RowsRead: 110.72M (110722748) - RowsReturned: 110.72M (110722748) - RowsReturnedRate: 239.49 M/sec - ScanRangesComplete: 140 (140) - ScannerThreadsInvoluntaryContextSwitches: 2.83K (2830) - ScannerThreadsTotalWallClockTime: 8m7s - MaterializeTupleTime(*): 394.766ms - ScannerThreadsSysTime: 1s055ms - ScannerThreadsUserTime: 10s896ms - ScannerThreadsVoluntaryContextSwitches: 114.40K (114401) - TotalKuduScanRoundTrips: 2.82K (2822) - TotalReadThroughput: 0.00 /sec Coordinator Fragment F00: Instance 3b4103fe92884226:7e6bbf6a00000000 (host=max2.hfoo:22000):(Total: 3s862ms, non-child: 30.268ms, % non-child: 0.78%) Hdfs split stats (:<# splits>/): MemoryUsage(500.000ms): 16.10 MB, 16.10 MB, 16.10 MB, 16.10 MB, 16.10 MB, 16.05 MB, 16.10 MB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 16.10 MB (16886622) - PerHostPeakMemUsage: 16.10 MB (16886622) - RowsProduced: 10 (10) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 25s811ms - TotalThreadsInvoluntaryContextSwitches: 3.23K (3228) - TotalThreadsTotalWallClockTime: 8m11s - TotalThreadsSysTime: 1s282ms - TotalThreadsUserTime: 14s425ms - TotalThreadsVoluntaryContextSwitches: 115.25K (115249) Fragment Instance Lifecycle Timings: - ExecTime: 28.881ms - ExecTreeExecTime: 38.533us - OpenTime: 3s807ms - ExecTreeOpenTime: 3s754ms - PrepareTime: 25.544ms - ExecTreePrepareTime: 697.821us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 32 (32) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 120.00 GB (128849018880) - PeakMemoryUsage: 528.00 KB (540672) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 77.114ms, non-child: 77.114ms, % non-child: 100.00%) - CodegenTime: 1.716ms - CompileTime: 15.530ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2076176) - NumFunctions: 41 (41) - NumInstructions: 729 (729) - OptimizationTime: 37.082ms - PeakMemoryUsage: 364.50 KB (373248) - PrepareTime: 24.066ms AGGREGATION_NODE (id=1):(Total: 3s754ms, non-child: 3s292ms, % non-child: 87.69%) ExecOption: Codegen Enabled - BuildTime: 3s232ms - GetNewBlockTime: 200.600us - GetResultsTime: 12.736us - HTResizeTime: 83.579ms - HashBuckets: 4.10K (4096) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 2.59 MB (2720896) - PinTime: 0.000ns - RowsRepartitioned: 0 (0) - RowsReturned: 10 (10) - RowsReturnedRate: 2.00 /sec - SpilledPartitions: 0 (0) - UnpinTime: 1.428us KUDU_SCAN_NODE (id=0):(Total: 462.321ms, non-child: 462.321ms, % non-child: 100.00%) BytesRead(500.000ms): 0, 0, 0, 0, 0, 0, 0 - BytesRead: 0 - KuduRemoteScanTokens: 80 (80) - NumScannerThreadsStarted: 138 (138) - PeakMemoryUsage: 13.56 MB (14221312) - RowsRead: 110.72M (110722748) - RowsReturned: 110.72M (110722748) - RowsReturnedRate: 239.49 M/sec - ScanRangesComplete: 140 (140) - ScannerThreadsInvoluntaryContextSwitches: 2.83K (2830) - ScannerThreadsTotalWallClockTime: 8m7s - MaterializeTupleTime(*): 394.766ms - ScannerThreadsSysTime: 1s055ms - ScannerThreadsUserTime: 10s896ms - ScannerThreadsVoluntaryContextSwitches: 114.40K (114401) - TotalKuduScanRoundTrips: 2.82K (2822) - TotalReadThroughput: 0.00 /sec