Query 2648620a9ab1302f:cb667f5a00000000
Query (id=2648620a9ab1302f:cb667f5a00000000):
Summary:
Session ID: ac49af0b653f73c7:19f24f487f2ae86
Session Type: BEESWAX
Start Time: 2018-01-04 17:12:37.476242000
End Time: 2018-01-04 17:13:28.475208000
Query Type: QUERY
Query State: FINISHED
Query Status: OK
Impala Version: impalad version 2.11.0-RELEASE RELEASE (build b49d149343c7ea0700e41cda3bdc2efcb8ea7e6d)
User: impala
Connected User: impala
Delegated User:
Network Address: ::ffff:10.120.196.61:48162
Default Db: kudu_1000g_2
Sql Statement: select count(*) from supplier as s, partsupp as p where s.s_suppkey = p.ps_suppkey
Coordinator: hadoop958.hz.163.org:22000
Query Options (set by configuration):
Query Options (set by configuration and planner): MT_DOP=0
Plan:
----------------
Max Per-Host Resource Reservation: Memory=34.00MB
Per-Host Resource Estimates: Memory=103.92MB
F02:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
PLAN-ROOT SINK
| mem-estimate=0B mem-reservation=0B
|
06:AGGREGATE [FINALIZE]
| output: count:merge(*)
| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
| tuple-ids=2 row-size=8B cardinality=1
|
05:EXCHANGE [UNPARTITIONED]
| mem-estimate=0B mem-reservation=0B
| tuple-ids=2 row-size=8B cardinality=1
|
F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
Per-Host Resources: mem-estimate=93.92MB mem-reservation=34.00MB
03:AGGREGATE
| output: count(*)
| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
| tuple-ids=2 row-size=8B cardinality=1
|
02:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: p.ps_suppkey = s.s_suppkey
| fk/pk conjuncts: p.ps_suppkey = s.s_suppkey
| runtime filters: RF001[min_max] <- s.s_suppkey
| mem-estimate=83.92MB mem-reservation=34.00MB spill-buffer=2.00MB
| tuple-ids=1,0 row-size=16B cardinality=773574091
|
|--04:EXCHANGE [BROADCAST]
| | mem-estimate=0B mem-reservation=0B
| | tuple-ids=0 row-size=8B cardinality=10000000
| |
| F01:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
| Per-Host Resources: mem-estimate=0B mem-reservation=0B
| 00:SCAN KUDU [kudu_1000g_2.supplier s]
| mem-estimate=0B mem-reservation=0B
| tuple-ids=0 row-size=8B cardinality=10000000
|
01:SCAN KUDU [kudu_1000g_2.partsupp p]
runtime filters: RF001[min_max] -> p.ps_suppkey
mem-estimate=0B mem-reservation=0B
tuple-ids=1 row-size=8B cardinality=800000000
----------------
Estimated Per-Host Mem: 108971520
Per Host Min Reservation: hadoop956.hz.163.org:22000(34.00 MB) hadoop957.hz.163.org:22000(34.00 MB) hadoop958.hz.163.org:22000(34.00 MB)
Request Pool: default-pool
Admission result: Admitted immediately
ExecSummary:
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
-----------------------------------------------------------------------------------------------------------------------
06:AGGREGATE 1 130.059us 130.059us 1 1 28.00 KB 10.00 MB FINALIZE
05:EXCHANGE 1 49s597ms 49s597ms 3 1 0 0 UNPARTITIONED
03:AGGREGATE 3 557.369ms 588.188ms 3 1 48.00 KB 10.00 MB
02:HASH JOIN 3 44s771ms 47s008ms 800.00M 773.57M 352.05 MB 83.92 MB INNER JOIN, BROADCAST
|--04:EXCHANGE 3 455.709ms 501.068ms 10.00M 10.00M 0 0 BROADCAST
| 00:SCAN KUDU 3 79.346ms 84.366ms 10.00M 10.00M 1.61 MB 0 kudu_1000g_2.supplier s
01:SCAN KUDU 3 1s592ms 1s611ms 800.00M 800.00M 2.20 MB 0 kudu_1000g_2.partsupp p
Errors:
Planner Timeline: 1s325ms
- Analysis finished: 144.861ms (144.861ms)
- Value transfer graph computed: 215.082ms (70.220ms)
- Single node plan created: 1s182ms (967.668ms)
- Runtime filters computed: 1s200ms (17.400ms)
- Distributed plan created: 1s200ms (464.945us)
- Planning finished: 1s325ms (124.523ms)
Query Timeline: 51s004ms
- Query submitted: 90.188us (90.188us)
- Planning finished: 1s351ms (1s351ms)
- Submit for admission: 1s352ms (514.613us)
- Completed admission: 1s352ms (72.367us)
- Ready to start on 3 backends: 1s352ms (191.307us)
- All 3 execution backends (7 fragment instances) started: 1s354ms (1.994ms)
- Rows available: 50s983ms (49s628ms)
- First row fetched: 50s995ms (11.907ms)
- Unregister query: 50s998ms (3.979ms)
- ComputeScanRangeAssignmentTimer: 301.671us
ImpalaServer:
- ClientFetchWaitTimer: 13.831ms
- RowMaterializationTimer: 2.016ms
Execution Profile 2648620a9ab1302f:cb667f5a00000000:(Total: 49s632ms, non-child: 0.000ns, % non-child: 0.00%)
Number of filters: 1
Filter routing table:
ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled
-------------------------------------------------------------------------------------------------------------------
1 2 1 LOCAL false 0 (3) N/A N/A true
Backend startup latencies: Count: 3, min / max: 1ms / 2ms, 25th %-ile: 1ms, 50th %-ile: 2ms, 75th %-ile: 2ms, 90th %-ile: 2ms, 95th %-ile: 2ms, 99.9th %-ile: 2ms
Final filter table:
ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled
-------------------------------------------------------------------------------------------------------------------
1 2 1 LOCAL false 0 (3) N/A N/A true
Per Node Peak Memory Usage: hadoop956.hz.163.org:22000(354.31 MB) hadoop957.hz.163.org:22000(354.31 MB) hadoop958.hz.163.org:22000(354.33 MB)
- FiltersReceived: 0 (0)
- FinalizationTimer: 0.000ns
- NumBackends: 3 (3)
- NumFragmentInstances: 7 (7)
- NumFragments: 3 (3)
Averaged Fragment F02:(Total: 49s641ms, non-child: 12.377ms, % non-child: 0.02%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:49s641ms max:49s641ms mean: 49s641ms stddev:0.000ns
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 1
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 36.02 KB (36880)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 354.33 MB (371545190)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 49s597ms
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 49s609ms
- TotalThreadsSysTime: 0.000ns
- TotalThreadsUserTime: 0.000ns
- TotalThreadsVoluntaryContextSwitches: 5 (5)
Fragment Instance Lifecycle Timings:
- ExecTime: 11.964ms
- ExecTreeExecTime: 5.914us
- OpenTime: 49s597ms
- ExecTreeOpenTime: 49s597ms
- PrepareTime: 31.327ms
- ExecTreePrepareTime: 97.323us
PLAN_ROOT_SINK:
- PeakMemoryUsage: 0
CodeGen:(Total: 30.902ms, non-child: 30.902ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 29.971ms
AGGREGATION_NODE (id=6):(Total: 49s597ms, non-child: 130.059us, % non-child: 0.00%)
- BuildTime: 4.980us
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 28.00 KB (28672)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=5):(Total: 49s597ms, non-child: 49s597ms, % non-child: 100.00%)
- ConvertRowBatchTime: 3.493us
- PeakMemoryUsage: 0
- RowsReturned: 3 (3)
- RowsReturnedRate: 0
DataStreamReceiver:
- BytesReceived: 48.00 B (48)
- DeserializeRowBatchTimer: 12.656us
- FirstBatchArrivalWaitTime: 45s722ms
- PeakMemoryUsage: 4.02 KB (4112)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Coordinator Fragment F02:
Instance 2648620a9ab1302f:cb667f5a00000000 (host=hadoop958.hz.163.org:22000):(Total: 49s641ms, non-child: 12.377ms, % non-child: 0.02%)
MemoryUsage(1s000ms): 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 20.00 KB, 30.01 KB, 32.01 KB, 32.01 KB
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 36.02 KB (36880)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 354.33 MB (371545190)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 49s597ms
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 49s609ms
- TotalThreadsSysTime: 0.000ns
- TotalThreadsUserTime: 0.000ns
- TotalThreadsVoluntaryContextSwitches: 5 (5)
Fragment Instance Lifecycle Timings:
- ExecTime: 11.964ms
- ExecTreeExecTime: 5.914us
- OpenTime: 49s597ms
- ExecTreeOpenTime: 49s597ms
- PrepareTime: 31.327ms
- ExecTreePrepareTime: 97.323us
PLAN_ROOT_SINK:
- PeakMemoryUsage: 0
CodeGen:(Total: 30.902ms, non-child: 30.902ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 29.971ms
AGGREGATION_NODE (id=6):(Total: 49s597ms, non-child: 130.059us, % non-child: 0.00%)
- BuildTime: 4.980us
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 28.00 KB (28672)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=5):(Total: 49s597ms, non-child: 49s597ms, % non-child: 100.00%)
- ConvertRowBatchTime: 3.493us
- PeakMemoryUsage: 0
- RowsReturned: 3 (3)
- RowsReturnedRate: 0
DataStreamReceiver:
BytesReceived(1s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8.00 B, 24.00 B, 32.00 B, 32.00 B
- BytesReceived: 48.00 B (48)
- DeserializeRowBatchTimer: 12.656us
- FirstBatchArrivalWaitTime: 45s722ms
- PeakMemoryUsage: 4.02 KB (4112)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Averaged Fragment F00:(Total: 47s348ms, non-child: 8.613ms, % non-child: 0.02%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:45s754ms max:49s629ms mean: 47s348ms stddev:1s654ms
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 3
- AverageThreadTokens: 40.16
- BloomFilterBytes: 0
- PeakMemoryUsage: 354.31 MB (371516510)
- PeakReservation: 352.00 MB (369098752)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 354.31 MB (371526070)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 186.988ms
- TotalNetworkSendTime: 199.235us
- TotalStorageWaitTime: 13s659ms
- TotalThreadsInvoluntaryContextSwitches: 348 (348)
- TotalThreadsTotalWallClockTime: 31m39s
- TotalThreadsSysTime: 1s806ms
- TotalThreadsUserTime: 1m
- TotalThreadsVoluntaryContextSwitches: 266.16K (266159)
Fragment Instance Lifecycle Timings:
- ExecTime: 298.771us
- ExecTreeExecTime: 4.103us
- OpenTime: 47s302ms
- ExecTreeOpenTime: 47s069ms
- PrepareTime: 44.979ms
- ExecTreePrepareTime: 297.197us
DataStreamSender (dst_id=5):(Total: 174.528us, non-child: 174.528us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 531.58 KB/sec
- OverallThroughput: 90.29 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 8.941us
- TransmitDataRPCTime: 30.254us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 269.080ms, non-child: 269.080ms, % non-child: 100.00%)
- CodegenTime: 429.406us
- CompileTime: 69.111ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 60 (60)
- NumInstructions: 2.50K (2497)
- OptimizationTime: 162.607ms
- PeakMemoryUsage: 1.22 MB (1278464)
- PrepareTime: 36.356ms
AGGREGATION_NODE (id=3):(Total: 47s070ms, non-child: 557.369ms, % non-child: 1.18%)
- BuildTime: 411.930ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 45.33 KB (46421)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
HASH_JOIN_NODE (id=2):(Total: 46s512ms, non-child: 44s771ms, % non-child: 96.26%)
- BuildRows: 10.00M (10000000)
- BuildTime: 1s058ms
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 352.05 MB (369150208)
- ProbeRows: 266.67M (266666666)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 43s871ms
- RowsReturned: 266.67M (266666666)
- RowsReturnedRate: 5.74 M/sec
Buffer pool:
- AllocTime: 733.013us
- CumulativeAllocationBytes: 352.00 MB (369098752)
- CumulativeAllocations: 64 (64)
- PeakReservation: 352.00 MB (369098752)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 352.00 MB (369098752)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
- BuildRowsPartitionTime: 563.180ms
- BuildRowsPartitioned: 10.00M (10000000)
- HashBuckets: 16.78M (16777216)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 493.090ms
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 455.709ms, non-child: 455.709ms, % non-child: 100.00%)
- ConvertRowBatchTime: 260.624ms
- PeakMemoryUsage: 0
- RowsReturned: 10.00M (10000000)
- RowsReturnedRate: 22.05 M/sec
DataStreamReceiver:
- BytesReceived: 76.63 MB (80347219)
- DeserializeRowBatchTimer: 237.075ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.02 MB (10502144)
- SendersBlockedTimer: 132.277ms
- SendersBlockedTotalTimer(*): 305.706ms
KUDU_SCAN_NODE (id=1):(Total: 1s592ms, non-child: 1s592ms, % non-child: 100.00%)
- BytesRead: 0
- CollectionItemsRead: 0 (0)
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 40 (40)
- PeakMemoryUsage: 2.20 MB (2310144)
- RowsRead: 266.67M (266666666)
- RowsReturned: 266.67M (266666666)
- RowsReturnedRate: 167.51 M/sec
- ScanRangesComplete: 40 (40)
- ScannerThreadsInvoluntaryContextSwitches: 258 (258)
- ScannerThreadsTotalWallClockTime: 30m51s
- MaterializeTupleTime(*): 833.940ms
- ScannerThreadsSysTime: 1s228ms
- ScannerThreadsUserTime: 14s374ms
- ScannerThreadsVoluntaryContextSwitches: 262.53K (262533)
- TotalKuduScanRoundTrips: 2.04K (2040)
- TotalReadThroughput: 0.00 /sec
Fragment F00:
Instance 2648620a9ab1302f:cb667f5a00000004 (host=hadoop957.hz.163.org:22000):(Total: 49s628ms, non-child: 8.367ms, % non-child: 0.02%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
Filter 1 arrival: 1s312ms
MemoryUsage(1s000ms): 49.10 MB, 266.28 MB, 354.30 MB, 354.29 MB, 354.30 MB, 354.30 MB, 354.29 MB, 354.30 MB, 354.30 MB, 354.29 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.29 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.29 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB
ThreadUsage(1s000ms): 2, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41
- AverageThreadTokens: 40.19
- BloomFilterBytes: 0
- PeakMemoryUsage: 354.31 MB (371516510)
- PeakReservation: 352.00 MB (369098752)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 354.31 MB (371516510)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 252.844ms
- TotalNetworkSendTime: 200.387us
- TotalStorageWaitTime: 12s313ms
- TotalThreadsInvoluntaryContextSwitches: 402 (402)
- TotalThreadsTotalWallClockTime: 33m13s
- TotalThreadsSysTime: 1s764ms
- TotalThreadsUserTime: 1m3s
- TotalThreadsVoluntaryContextSwitches: 266.82K (266822)
Fragment Instance Lifecycle Timings:
- ExecTime: 307.967us
- ExecTreeExecTime: 3.945us
- OpenTime: 49s583ms
- ExecTreeOpenTime: 49s350ms
- PrepareTime: 44.635ms
- ExecTreePrepareTime: 260.995us
DataStreamSender (dst_id=5):(Total: 153.713us, non-child: 153.713us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 541.35 KB/sec
- OverallThroughput: 101.65 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 9.478us
- TransmitDataRPCTime: 28.863us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 269.598ms, non-child: 269.598ms, % non-child: 100.00%)
- CodegenTime: 402.708us
- CompileTime: 69.048ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 60 (60)
- NumInstructions: 2.50K (2497)
- OptimizationTime: 163.245ms
- PeakMemoryUsage: 1.22 MB (1278464)
- PrepareTime: 36.288ms
AGGREGATION_NODE (id=3):(Total: 49s350ms, non-child: 560.935ms, % non-child: 1.14%)
ExecOption: Codegen Enabled
- BuildTime: 415.554ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 48.00 KB (49152)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
HASH_JOIN_NODE (id=2):(Total: 48s789ms, non-child: 47s008ms, % non-child: 96.35%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 10.00M (10000000)
- BuildTime: 1s012ms
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 352.05 MB (369150208)
- ProbeRows: 266.66M (266664154)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 46s119ms
- RowsReturned: 266.66M (266664154)
- RowsReturnedRate: 5.47 M/sec
Buffer pool:
- AllocTime: 717.800us
- CumulativeAllocationBytes: 352.00 MB (369098752)
- CumulativeAllocations: 64 (64)
- PeakReservation: 352.00 MB (369098752)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 352.00 MB (369098752)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
Runtime filters: 1 of 1 Runtime Filter Published
- BuildRowsPartitionTime: 518.712ms
- BuildRowsPartitioned: 10.00M (10000000)
- HashBuckets: 16.78M (16777216)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 492.060ms
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 501.068ms, non-child: 501.068ms, % non-child: 100.00%)
- ConvertRowBatchTime: 240.730ms
- PeakMemoryUsage: 0
- RowsReturned: 10.00M (10000000)
- RowsReturnedRate: 19.96 M/sec
DataStreamReceiver:
BytesReceived(500.000ms): 18.60 MB, 54.09 MB, 76.63 MB
- BytesReceived: 76.63 MB (80347219)
- DeserializeRowBatchTimer: 230.510ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.02 MB (10502144)
- SendersBlockedTimer: 129.846ms
- SendersBlockedTotalTimer(*): 259.113ms
KUDU_SCAN_NODE (id=1):(Total: 1s611ms, non-child: 1s611ms, % non-child: 100.00%)
Runtime filters: Not all filters arrived (arrived: [], missing [1]), waited for 722ms
BytesRead(1s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- CollectionItemsRead: 0 (0)
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 40 (40)
- PeakMemoryUsage: 2.20 MB (2310144)
- RowsRead: 266.66M (266664154)
- RowsReturned: 266.66M (266664154)
- RowsReturnedRate: 165.50 M/sec
- ScanRangesComplete: 40 (40)
- ScannerThreadsInvoluntaryContextSwitches: 308 (308)
- ScannerThreadsTotalWallClockTime: 32m22s
- MaterializeTupleTime(*): 861.542ms
- ScannerThreadsSysTime: 1s200ms
- ScannerThreadsUserTime: 14s636ms
- ScannerThreadsVoluntaryContextSwitches: 262.53K (262534)
- TotalKuduScanRoundTrips: 2.04K (2040)
- TotalReadThroughput: 0.00 /sec
Instance 2648620a9ab1302f:cb667f5a00000006 (host=hadoop958.hz.163.org:22000):(Total: 46s661ms, non-child: 8.996ms, % non-child: 0.02%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
Filter 1 arrival: 1s313ms
MemoryUsage(1s000ms): 49.11 MB, 266.28 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.29 MB, 354.29 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.29 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.29 MB
ThreadUsage(1s000ms): 2, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 40
- AverageThreadTokens: 40.14
- BloomFilterBytes: 0
- PeakMemoryUsage: 354.31 MB (371516510)
- PeakReservation: 352.00 MB (369098752)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 354.33 MB (371545190)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 152.468ms
- TotalNetworkSendTime: 164.650us
- TotalStorageWaitTime: 16s521ms
- TotalThreadsInvoluntaryContextSwitches: 291 (291)
- TotalThreadsTotalWallClockTime: 31m11s
- TotalThreadsSysTime: 1s852ms
- TotalThreadsUserTime: 1m
- TotalThreadsVoluntaryContextSwitches: 265.75K (265747)
Fragment Instance Lifecycle Timings:
- ExecTime: 238.276us
- ExecTreeExecTime: 5.175us
- OpenTime: 46s615ms
- ExecTreeOpenTime: 46s381ms
- PrepareTime: 45.846ms
- ExecTreePrepareTime: 365.019us
DataStreamSender (dst_id=5):(Total: 191.852us, non-child: 191.852us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 419.43 KB/sec
- OverallThroughput: 81.44 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 9.757us
- TransmitDataRPCTime: 37.253us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 270.891ms, non-child: 270.891ms, % non-child: 100.00%)
- CodegenTime: 470.966us
- CompileTime: 69.433ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 60 (60)
- NumInstructions: 2.50K (2497)
- OptimizationTime: 163.644ms
- PeakMemoryUsage: 1.22 MB (1278464)
- PrepareTime: 36.820ms
AGGREGATION_NODE (id=3):(Total: 46s381ms, non-child: 522.983ms, % non-child: 1.13%)
ExecOption: Codegen Enabled
- BuildTime: 384.392ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 48.00 KB (49152)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
HASH_JOIN_NODE (id=2):(Total: 45s858ms, non-child: 44s124ms, % non-child: 96.22%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 10.00M (10000000)
- BuildTime: 1s091ms
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 352.05 MB (369150208)
- ProbeRows: 266.67M (266669618)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 43s190ms
- RowsReturned: 266.67M (266669618)
- RowsReturnedRate: 5.82 M/sec
Buffer pool:
- AllocTime: 738.902us
- CumulativeAllocationBytes: 352.00 MB (369098752)
- CumulativeAllocations: 64 (64)
- PeakReservation: 352.00 MB (369098752)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 352.00 MB (369098752)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
Runtime filters: 1 of 1 Runtime Filter Published
- BuildRowsPartitionTime: 584.910ms
- BuildRowsPartitioned: 10.00M (10000000)
- HashBuckets: 16.78M (16777216)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 504.846ms
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 430.857ms, non-child: 430.857ms, % non-child: 100.00%)
- ConvertRowBatchTime: 269.925ms
- PeakMemoryUsage: 0
- RowsReturned: 10.00M (10000000)
- RowsReturnedRate: 23.21 M/sec
DataStreamReceiver:
BytesReceived(500.000ms): 18.97 MB, 54.46 MB, 76.63 MB
- BytesReceived: 76.63 MB (80347219)
- DeserializeRowBatchTimer: 240.121ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.02 MB (10502144)
- SendersBlockedTimer: 139.445ms
- SendersBlockedTotalTimer(*): 276.304ms
KUDU_SCAN_NODE (id=1):(Total: 1s585ms, non-child: 1s585ms, % non-child: 100.00%)
Runtime filters: Not all filters arrived (arrived: [], missing [1]), waited for 722ms
BytesRead(1s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- CollectionItemsRead: 0 (0)
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 40 (40)
- PeakMemoryUsage: 2.20 MB (2310144)
- RowsRead: 266.67M (266669618)
- RowsReturned: 266.67M (266669618)
- RowsReturnedRate: 168.19 M/sec
- ScanRangesComplete: 40 (40)
- ScannerThreadsInvoluntaryContextSwitches: 204 (204)
- ScannerThreadsTotalWallClockTime: 30m23s
- MaterializeTupleTime(*): 834.480ms
- ScannerThreadsSysTime: 1s228ms
- ScannerThreadsUserTime: 14s268ms
- ScannerThreadsVoluntaryContextSwitches: 262.53K (262533)
- TotalKuduScanRoundTrips: 2.04K (2040)
- TotalReadThroughput: 0.00 /sec
Instance 2648620a9ab1302f:cb667f5a00000005 (host=hadoop956.hz.163.org:22000):(Total: 45s753ms, non-child: 8.476ms, % non-child: 0.02%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
Filter 1 arrival: 1s311ms
MemoryUsage(1s000ms): 21.96 MB, 186.29 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.29 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.29 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.30 MB, 354.29 MB, 354.30 MB, 354.29 MB, 354.30 MB, 354.28 MB
ThreadUsage(1s000ms): 1, 42, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 40
- AverageThreadTokens: 40.14
- BloomFilterBytes: 0
- PeakMemoryUsage: 354.31 MB (371516510)
- PeakReservation: 352.00 MB (369098752)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 354.31 MB (371516510)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 155.652ms
- TotalNetworkSendTime: 232.669us
- TotalStorageWaitTime: 12s144ms
- TotalThreadsInvoluntaryContextSwitches: 351 (351)
- TotalThreadsTotalWallClockTime: 30m34s
- TotalThreadsSysTime: 1s804ms
- TotalThreadsUserTime: 59s216ms
- TotalThreadsVoluntaryContextSwitches: 265.91K (265908)
Fragment Instance Lifecycle Timings:
- ExecTime: 350.071us
- ExecTreeExecTime: 3.189us
- OpenTime: 45s708ms
- ExecTreeOpenTime: 45s478ms
- PrepareTime: 44.456ms
- ExecTreePrepareTime: 265.577us
DataStreamSender (dst_id=5):(Total: 178.019us, non-child: 178.019us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 633.98 KB/sec
- OverallThroughput: 87.77 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 7.588us
- TransmitDataRPCTime: 24.646us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 266.752ms, non-child: 266.752ms, % non-child: 100.00%)
- CodegenTime: 414.546us
- CompileTime: 68.851ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 60 (60)
- NumInstructions: 2.50K (2497)
- OptimizationTime: 160.930ms
- PeakMemoryUsage: 1.22 MB (1278464)
- PrepareTime: 35.962ms
AGGREGATION_NODE (id=3):(Total: 45s478ms, non-child: 588.188ms, % non-child: 1.29%)
ExecOption: Codegen Enabled
- BuildTime: 435.842ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 40.00 KB (40960)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
HASH_JOIN_NODE (id=2):(Total: 44s890ms, non-child: 43s182ms, % non-child: 96.20%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 10.00M (10000000)
- BuildTime: 1s070ms
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 352.05 MB (369150208)
- ProbeRows: 266.67M (266666228)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 42s302ms
- RowsReturned: 266.67M (266666228)
- RowsReturnedRate: 5.94 M/sec
Buffer pool:
- AllocTime: 742.339us
- CumulativeAllocationBytes: 352.00 MB (369098752)
- CumulativeAllocations: 64 (64)
- PeakReservation: 352.00 MB (369098752)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 352.00 MB (369098752)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
Runtime filters: 1 of 1 Runtime Filter Published
- BuildRowsPartitionTime: 585.919ms
- BuildRowsPartitioned: 10.00M (10000000)
- HashBuckets: 16.78M (16777216)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 482.363ms
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 435.200ms, non-child: 435.200ms, % non-child: 100.00%)
- ConvertRowBatchTime: 271.218ms
- PeakMemoryUsage: 0
- RowsReturned: 10.00M (10000000)
- RowsReturnedRate: 22.98 M/sec
DataStreamReceiver:
BytesReceived(500.000ms): 4.28 MB, 29.53 MB, 65.04 MB, 76.63 MB
- BytesReceived: 76.63 MB (80347219)
- DeserializeRowBatchTimer: 240.595ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.02 MB (10502144)
- SendersBlockedTimer: 127.541ms
- SendersBlockedTotalTimer(*): 381.701ms
KUDU_SCAN_NODE (id=1):(Total: 1s579ms, non-child: 1s579ms, % non-child: 100.00%)
Runtime filters: Not all filters arrived (arrived: [], missing [1]), waited for 742ms
BytesRead(1s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- CollectionItemsRead: 0 (0)
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 40 (40)
- PeakMemoryUsage: 2.20 MB (2310144)
- RowsRead: 266.67M (266666228)
- RowsReturned: 266.67M (266666228)
- RowsReturnedRate: 168.83 M/sec
- ScanRangesComplete: 40 (40)
- ScannerThreadsInvoluntaryContextSwitches: 264 (264)
- ScannerThreadsTotalWallClockTime: 29m47s
- MaterializeTupleTime(*): 805.798ms
- ScannerThreadsSysTime: 1s256ms
- ScannerThreadsUserTime: 14s220ms
- ScannerThreadsVoluntaryContextSwitches: 262.53K (262534)
- TotalKuduScanRoundTrips: 2.04K (2040)
- TotalReadThroughput: 0.00 /sec
Averaged Fragment F01:(Total: 1s299ms, non-child: 639.043ms, % non-child: 49.19%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:1s290ms max:1s312ms mean: 1s299ms stddev:9.328ms
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 3
- AverageThreadTokens: 3.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 1.66 MB (1744896)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 100.21 MB (105079902)
- RowsProduced: 3.33M (3333333)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 857.494ms
- TotalStorageWaitTime: 168.517ms
- TotalThreadsInvoluntaryContextSwitches: 6 (6)
- TotalThreadsTotalWallClockTime: 3s673ms
- TotalThreadsSysTime: 41.333ms
- TotalThreadsUserTime: 529.333ms
- TotalThreadsVoluntaryContextSwitches: 7.92K (7915)
Fragment Instance Lifecycle Timings:
- ExecTime: 1s242ms
- ExecTreeExecTime: 31.898ms
- OpenTime: 24.628ms
- ExecTreeOpenTime: 24.613ms
- PrepareTime: 32.328ms
- ExecTreePrepareTime: 50.820us
DataStreamSender (dst_id=4):(Total: 549.138ms, non-child: 549.138ms, % non-child: 100.00%)
- BytesSent: 76.63 MB (80347219)
- NetworkThroughput(*): 343.51 MB/sec
- OverallThroughput: 139.75 MB/sec
- PeakMemoryUsage: 48.00 KB (49152)
- RowsReturned: 3.33M (3333333)
- SerializeBatchTime: 321.261ms
- TransmitDataRPCTime: 227.084ms
- UncompressedRowBatchSize: 152.59 MB (160000000)
CodeGen:(Total: 31.635ms, non-child: 31.635ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 30.699ms
KUDU_SCAN_NODE (id=0):(Total: 79.346ms, non-child: 79.346ms, % non-child: 100.00%)
- BytesRead: 0
- CollectionItemsRead: 0 (0)
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 2 (2)
- PeakMemoryUsage: 1.60 MB (1682090)
- RowsRead: 3.33M (3333333)
- RowsReturned: 3.33M (3333333)
- RowsReturnedRate: 42.10 M/sec
- ScanRangesComplete: 2 (2)
- ScannerThreadsInvoluntaryContextSwitches: 3 (3)
- ScannerThreadsTotalWallClockTime: 2s406ms
- MaterializeTupleTime(*): 29.758ms
- ScannerThreadsSysTime: 17.333ms
- ScannerThreadsUserTime: 160.000ms
- ScannerThreadsVoluntaryContextSwitches: 3.16K (3162)
- TotalKuduScanRoundTrips: 26 (26)
- TotalReadThroughput: 0.00 /sec
Fragment F01:
Instance 2648620a9ab1302f:cb667f5a00000001 (host=hadoop957.hz.163.org:22000):(Total: 1s311ms, non-child: 683.248ms, % non-child: 52.08%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(500.000ms): 1.66 MB, 1.66 MB
ThreadUsage(500.000ms): 3, 3
- AverageThreadTokens: 3.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 1.66 MB (1744896)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 100.15 MB (105018462)
- RowsProduced: 3.33M (3332123)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 869.041ms
- TotalStorageWaitTime: 169.162ms
- TotalThreadsInvoluntaryContextSwitches: 5 (5)
- TotalThreadsTotalWallClockTime: 3s721ms
- TotalThreadsSysTime: 48.000ms
- TotalThreadsUserTime: 528.000ms
- TotalThreadsVoluntaryContextSwitches: 7.89K (7894)
Fragment Instance Lifecycle Timings:
- ExecTime: 1s256ms
- ExecTreeExecTime: 30.337ms
- OpenTime: 23.634ms
- ExecTreeOpenTime: 23.615ms
- PrepareTime: 31.770ms
- ExecTreePrepareTime: 50.483us
DataStreamSender (dst_id=4):(Total: 521.581ms, non-child: 521.581ms, % non-child: 100.00%)
- BytesSent: 76.60 MB (80318232)
- NetworkThroughput(*): 389.31 MB/sec
- OverallThroughput: 146.86 MB/sec
- PeakMemoryUsage: 48.00 KB (49152)
- RowsReturned: 3.33M (3332123)
- SerializeBatchTime: 324.083ms
- TransmitDataRPCTime: 196.751ms
- UncompressedRowBatchSize: 152.53 MB (159941904)
CodeGen:(Total: 31.127ms, non-child: 31.127ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 30.169ms
KUDU_SCAN_NODE (id=0):(Total: 75.901ms, non-child: 75.901ms, % non-child: 100.00%)
BytesRead(500.000ms): 0, 0
- BytesRead: 0
- CollectionItemsRead: 0 (0)
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 2 (2)
- PeakMemoryUsage: 1.61 MB (1687552)
- RowsRead: 3.33M (3332123)
- RowsReturned: 3.33M (3332123)
- RowsReturnedRate: 43.90 M/sec
- ScanRangesComplete: 2 (2)
- ScannerThreadsInvoluntaryContextSwitches: 2 (2)
- ScannerThreadsTotalWallClockTime: 2s441ms
- MaterializeTupleTime(*): 28.289ms
- ScannerThreadsSysTime: 20.000ms
- ScannerThreadsUserTime: 160.000ms
- ScannerThreadsVoluntaryContextSwitches: 3.17K (3167)
- TotalKuduScanRoundTrips: 26 (26)
- TotalReadThroughput: 0.00 /sec
Instance 2648620a9ab1302f:cb667f5a00000002 (host=hadoop956.hz.163.org:22000):(Total: 1s296ms, non-child: 604.667ms, % non-child: 46.66%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(500.000ms): 1.66 MB, 1.66 MB, 1.66 MB
ThreadUsage(500.000ms): 3, 3, 3
- AverageThreadTokens: 3.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 1.66 MB (1744896)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 100.24 MB (105108574)
- RowsProduced: 3.33M (3333642)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 869.037ms
- TotalStorageWaitTime: 165.320ms
- TotalThreadsInvoluntaryContextSwitches: 7 (7)
- TotalThreadsTotalWallClockTime: 3s644ms
- TotalThreadsSysTime: 56.000ms
- TotalThreadsUserTime: 488.000ms
- TotalThreadsVoluntaryContextSwitches: 7.93K (7932)
Fragment Instance Lifecycle Timings:
- ExecTime: 1s235ms
- ExecTreeExecTime: 32.847ms
- OpenTime: 26.663ms
- ExecTreeOpenTime: 26.643ms
- PrepareTime: 33.480ms
- ExecTreePrepareTime: 51.334us
DataStreamSender (dst_id=4):(Total: 574.239ms, non-child: 574.239ms, % non-child: 100.00%)
- BytesSent: 76.63 MB (80354574)
- NetworkThroughput(*): 283.66 MB/sec
- OverallThroughput: 133.45 MB/sec
- PeakMemoryUsage: 48.00 KB (49152)
- RowsReturned: 3.33M (3333642)
- SerializeBatchTime: 303.281ms
- TransmitDataRPCTime: 270.152ms
- UncompressedRowBatchSize: 152.60 MB (160014816)
CodeGen:(Total: 32.733ms, non-child: 32.733ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 31.766ms
KUDU_SCAN_NODE (id=0):(Total: 84.366ms, non-child: 84.366ms, % non-child: 100.00%)
BytesRead(500.000ms): 0, 0, 0
- BytesRead: 0
- CollectionItemsRead: 0 (0)
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 2 (2)
- PeakMemoryUsage: 1.60 MB (1679360)
- RowsRead: 3.33M (3333642)
- RowsReturned: 3.33M (3333642)
- RowsReturnedRate: 39.51 M/sec
- ScanRangesComplete: 2 (2)
- ScannerThreadsInvoluntaryContextSwitches: 3 (3)
- ScannerThreadsTotalWallClockTime: 2s381ms
- MaterializeTupleTime(*): 30.727ms
- ScannerThreadsSysTime: 16.000ms
- ScannerThreadsUserTime: 156.000ms
- ScannerThreadsVoluntaryContextSwitches: 3.17K (3172)
- TotalKuduScanRoundTrips: 26 (26)
- TotalReadThroughput: 0.00 /sec
Instance 2648620a9ab1302f:cb667f5a00000003 (host=hadoop958.hz.163.org:22000):(Total: 1s289ms, non-child: 629.215ms, % non-child: 48.79%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(500.000ms): 1.66 MB, 1.66 MB
ThreadUsage(500.000ms): 3, 3
- AverageThreadTokens: 3.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 1.66 MB (1744896)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 100.24 MB (105112670)
- RowsProduced: 3.33M (3334235)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 834.405ms
- TotalStorageWaitTime: 171.068ms
- TotalThreadsInvoluntaryContextSwitches: 6 (6)
- TotalThreadsTotalWallClockTime: 3s654ms
- TotalThreadsSysTime: 20.000ms
- TotalThreadsUserTime: 572.000ms
- TotalThreadsVoluntaryContextSwitches: 7.92K (7920)
Fragment Instance Lifecycle Timings:
- ExecTime: 1s234ms
- ExecTreeExecTime: 32.511ms
- OpenTime: 23.588ms
- ExecTreeOpenTime: 23.580ms
- PrepareTime: 31.733ms
- ExecTreePrepareTime: 50.643us
DataStreamSender (dst_id=4):(Total: 551.593ms, non-child: 551.593ms, % non-child: 100.00%)
- BytesSent: 76.65 MB (80368851)
- NetworkThroughput(*): 357.57 MB/sec
- OverallThroughput: 138.95 MB/sec
- PeakMemoryUsage: 48.00 KB (49152)
- RowsReturned: 3.33M (3334235)
- SerializeBatchTime: 336.420ms
- TransmitDataRPCTime: 214.350ms
- UncompressedRowBatchSize: 152.63 MB (160043280)
CodeGen:(Total: 31.045ms, non-child: 31.045ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 30.163ms
KUDU_SCAN_NODE (id=0):(Total: 77.771ms, non-child: 77.771ms, % non-child: 100.00%)
BytesRead(500.000ms): 0, 0
- BytesRead: 0
- CollectionItemsRead: 0 (0)
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 2 (2)
- PeakMemoryUsage: 1.60 MB (1679360)
- RowsRead: 3.33M (3334235)
- RowsReturned: 3.33M (3334235)
- RowsReturnedRate: 42.87 M/sec
- ScanRangesComplete: 2 (2)
- ScannerThreadsInvoluntaryContextSwitches: 4 (4)
- ScannerThreadsTotalWallClockTime: 2s397ms
- MaterializeTupleTime(*): 30.257ms
- ScannerThreadsSysTime: 16.000ms
- ScannerThreadsUserTime: 164.000ms
- ScannerThreadsVoluntaryContextSwitches: 3.15K (3148)
- TotalKuduScanRoundTrips: 26 (26)
- TotalReadThroughput: 0.00 /sec