Query 1542725b414611ca:d430e72b00000000
Query (id=1542725b414611ca:d430e72b00000000):
Summary:
Session ID: ac49af0b653f73c7:19f24f487f2ae86
Session Type: BEESWAX
Start Time: 2018-01-04 17:13:38.856968000
End Time: 2018-01-04 17:14:07.693651000
Query Type: QUERY
Query State: FINISHED
Query Status: OK
Impala Version: impalad version 2.11.0-RELEASE RELEASE (build b49d149343c7ea0700e41cda3bdc2efcb8ea7e6d)
User: impala
Connected User: impala
Delegated User:
Network Address: ::ffff:10.120.196.61:48162
Default Db: parquet_1000g
Sql Statement: select count(*) from supplier as s, partsupp as p where s.s_suppkey = p.ps_suppkey
Coordinator: hadoop958.hz.163.org:22000
Query Options (set by configuration):
Query Options (set by configuration and planner): MT_DOP=0
Plan:
----------------
Max Per-Host Resource Reservation: Memory=34.00MB
Per-Host Resource Estimates: Memory=279.92MB
F02:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
PLAN-ROOT SINK
| mem-estimate=0B mem-reservation=0B
|
06:AGGREGATE [FINALIZE]
| output: count:merge(*)
| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
| tuple-ids=2 row-size=8B cardinality=1
|
05:EXCHANGE [UNPARTITIONED]
| mem-estimate=0B mem-reservation=0B
| tuple-ids=2 row-size=8B cardinality=1
|
F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
Per-Host Resources: mem-estimate=181.92MB mem-reservation=34.00MB
03:AGGREGATE
| output: count(*)
| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
| tuple-ids=2 row-size=8B cardinality=1
|
02:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: p.ps_suppkey = s.s_suppkey
| fk/pk conjuncts: p.ps_suppkey = s.s_suppkey
| runtime filters: RF000[bloom] <- s.s_suppkey
| mem-estimate=83.92MB mem-reservation=34.00MB spill-buffer=2.00MB
| tuple-ids=1,0 row-size=16B cardinality=773574091
|
|--04:EXCHANGE [BROADCAST]
| | mem-estimate=0B mem-reservation=0B
| | tuple-ids=0 row-size=8B cardinality=10000000
| |
| F01:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
| Per-Host Resources: mem-estimate=88.00MB mem-reservation=0B
| 00:SCAN HDFS [parquet_1000g.supplier s, RANDOM]
| partitions=1/1 files=5 size=762.07MB
| stored statistics:
| table: rows=10000000 size=762.07MB
| columns: all
| extrapolated-rows=disabled
| mem-estimate=88.00MB mem-reservation=0B
| tuple-ids=0 row-size=8B cardinality=10000000
|
01:SCAN HDFS [parquet_1000g.partsupp p, RANDOM]
partitions=1/1 files=170 size=41.38GB
runtime filters: RF000[bloom] -> p.ps_suppkey
stored statistics:
table: rows=800000000 size=41.38GB
columns: all
extrapolated-rows=disabled
mem-estimate=88.00MB mem-reservation=0B
tuple-ids=1 row-size=8B cardinality=800000000
----------------
Estimated Per-Host Mem: 293520896
Per Host Min Reservation: hadoop956.hz.163.org:22000(34.00 MB) hadoop957.hz.163.org:22000(34.00 MB) hadoop958.hz.163.org:22000(34.00 MB)
Request Pool: default-pool
Admission result: Admitted immediately
ExecSummary:
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
------------------------------------------------------------------------------------------------------------------------
06:AGGREGATE 1 121.474us 121.474us 1 1 28.00 KB 10.00 MB FINALIZE
05:EXCHANGE 1 28s522ms 28s522ms 3 1 0 0 UNPARTITIONED
03:AGGREGATE 3 482.904ms 489.880ms 3 1 42.00 KB 10.00 MB
02:HASH JOIN 3 25s052ms 26s185ms 800.00M 773.57M 352.05 MB 83.92 MB INNER JOIN, BROADCAST
|--04:EXCHANGE 3 404.335ms 446.852ms 10.00M 10.00M 0 0 BROADCAST
| 00:SCAN HDFS 3 407.210ms 430.300ms 10.00M 10.00M 20.06 MB 88.00 MB parquet_1000g.supplier s
01:SCAN HDFS 3 1s115ms 1s132ms 800.00M 800.00M 805.02 MB 88.00 MB parquet_1000g.partsupp p
Errors:
Planner Timeline: 149.148ms
- Analysis finished: 7.655ms (7.655ms)
- Value transfer graph computed: 7.912ms (256.861us)
- Single node plan created: 136.790ms (128.878ms)
- Runtime filters computed: 137.749ms (958.840us)
- Distributed plan created: 137.856ms (106.627us)
- Planning finished: 149.148ms (11.292ms)
Query Timeline: 28s842ms
- Query submitted: 73.971us (73.971us)
- Planning finished: 157.931ms (157.857ms)
- Submit for admission: 158.542ms (610.774us)
- Completed admission: 158.555ms (12.877us)
- Ready to start on 3 backends: 158.767ms (212.130us)
- All 3 execution backends (7 fragment instances) started: 160.033ms (1.266ms)
- Rows available: 28s706ms (28s546ms)
- First row fetched: 28s832ms (125.356ms)
- Unregister query: 28s836ms (4.552ms)
- ComputeScanRangeAssignmentTimer: 367.487us
ImpalaServer:
- ClientFetchWaitTimer: 127.569ms
- RowMaterializationTimer: 2.298ms
Execution Profile 1542725b414611ca:d430e72b00000000:(Total: 28s550ms, non-child: 0.000ns, % non-child: 0.00%)
Number of filters: 1
Filter routing table:
ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled
-------------------------------------------------------------------------------------------------------------------
0 2 1 LOCAL false 0 (3) N/A N/A true
Backend startup latencies: Count: 3, min / max: 1ms / 1ms, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 1ms
Final filter table:
ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled
-------------------------------------------------------------------------------------------------------------------
0 2 1 LOCAL false 0 (3) N/A N/A true
Per Node Peak Memory Usage: hadoop956.hz.163.org:22000(1.13 GB) hadoop957.hz.163.org:22000(1.13 GB) hadoop958.hz.163.org:22000(1.13 GB)
- FiltersReceived: 0 (0)
- FinalizationTimer: 0.000ns
- NumBackends: 3 (3)
- NumFragmentInstances: 7 (7)
- NumFragments: 3 (3)
Averaged Fragment F02:(Total: 28s672ms, non-child: 125.766ms, % non-child: 0.44%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:28s672ms max:28s672ms mean: 28s672ms stddev:0.000ns
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 1
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 36.02 KB (36880)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 1.13 GB (1213609016)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 28s522ms
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 28s648ms
- TotalThreadsSysTime: 0.000ns
- TotalThreadsUserTime: 4.000ms
- TotalThreadsVoluntaryContextSwitches: 5 (5)
Fragment Instance Lifecycle Timings:
- ExecTime: 125.423ms
- ExecTreeExecTime: 3.336us
- OpenTime: 28s522ms
- ExecTreeOpenTime: 28s522ms
- PrepareTime: 23.993ms
- ExecTreePrepareTime: 89.638us
PLAN_ROOT_SINK:
- PeakMemoryUsage: 0
CodeGen:(Total: 23.624ms, non-child: 23.624ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 22.789ms
AGGREGATION_NODE (id=6):(Total: 28s522ms, non-child: 121.474us, % non-child: 0.00%)
- BuildTime: 4.203us
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 28.00 KB (28672)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=5):(Total: 28s522ms, non-child: 28s522ms, % non-child: 100.00%)
- ConvertRowBatchTime: 4.102us
- PeakMemoryUsage: 0
- RowsReturned: 3 (3)
- RowsReturnedRate: 0
DataStreamReceiver:
- BytesReceived: 48.00 B (48)
- DeserializeRowBatchTimer: 15.880us
- FirstBatchArrivalWaitTime: 25s405ms
- PeakMemoryUsage: 4.02 KB (4112)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Coordinator Fragment F02:
Instance 1542725b414611ca:d430e72b00000000 (host=hadoop958.hz.163.org:22000):(Total: 28s672ms, non-child: 125.766ms, % non-child: 0.44%)
MemoryUsage(500.000ms): 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 28.01 KB, 28.01 KB, 28.01 KB, 28.01 KB, 28.01 KB, 32.01 KB
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 36.02 KB (36880)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 1.13 GB (1213609016)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 28s522ms
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 28s648ms
- TotalThreadsSysTime: 0.000ns
- TotalThreadsUserTime: 4.000ms
- TotalThreadsVoluntaryContextSwitches: 5 (5)
Fragment Instance Lifecycle Timings:
- ExecTime: 125.423ms
- ExecTreeExecTime: 3.336us
- OpenTime: 28s522ms
- ExecTreeOpenTime: 28s522ms
- PrepareTime: 23.993ms
- ExecTreePrepareTime: 89.638us
PLAN_ROOT_SINK:
- PeakMemoryUsage: 0
CodeGen:(Total: 23.624ms, non-child: 23.624ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 22.789ms
AGGREGATION_NODE (id=6):(Total: 28s522ms, non-child: 121.474us, % non-child: 0.00%)
- BuildTime: 4.203us
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 28.00 KB (28672)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=5):(Total: 28s522ms, non-child: 28s522ms, % non-child: 100.00%)
- ConvertRowBatchTime: 4.102us
- PeakMemoryUsage: 0
- RowsReturned: 3 (3)
- RowsReturnedRate: 0
DataStreamReceiver:
BytesReceived(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16.00 B, 16.00 B, 16.00 B, 16.00 B, 16.00 B, 32.00 B
- BytesReceived: 48.00 B (48)
- DeserializeRowBatchTimer: 15.880us
- FirstBatchArrivalWaitTime: 25s405ms
- PeakMemoryUsage: 4.02 KB (4112)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Averaged Fragment F00:(Total: 27s389ms, non-child: 9.009ms, % non-child: 0.03%)
split sizes: min: 13.77 GB, max: 13.85 GB, avg: 13.79 GB, stddev: 38.92 MB
completion times: min:25s430ms max:28s547ms mean: 27s390ms stddev:1s393ms
execution rates: min:496.71 MB/sec max:554.35 MB/sec mean:517.03 MB/sec stddev:26.42 MB/sec
num instances: 3
- AverageThreadTokens: 32.02
- BloomFilterBytes: 4.00 MB (4194304)
- PeakMemoryUsage: 1.13 GB (1216218172)
- PeakReservation: 352.00 MB (369098752)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 1.13 GB (1216224999)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 197.996ms
- TotalNetworkSendTime: 184.322us
- TotalStorageWaitTime: 16s080ms
- TotalThreadsInvoluntaryContextSwitches: 246 (246)
- TotalThreadsTotalWallClockTime: 14m31s
- TotalThreadsSysTime: 1s224ms
- TotalThreadsUserTime: 40s602ms
- TotalThreadsVoluntaryContextSwitches: 263.93K (263934)
Fragment Instance Lifecycle Timings:
- ExecTime: 280.665us
- ExecTreeExecTime: 3.577us
- OpenTime: 27s274ms
- ExecTreeOpenTime: 26s985ms
- PrepareTime: 115.072ms
- ExecTreePrepareTime: 68.642ms
DataStreamSender (dst_id=5):(Total: 151.091us, non-child: 151.091us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 632.36 KB/sec
- OverallThroughput: 104.15 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 7.257us
- TransmitDataRPCTime: 26.408us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 326.267ms, non-child: 326.267ms, % non-child: 100.00%)
- CodegenTime: 2.633ms
- CompileTime: 88.240ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 87 (87)
- NumInstructions: 2.88K (2881)
- OptimizationTime: 199.218ms
- PeakMemoryUsage: 1.41 MB (1475072)
- PrepareTime: 37.811ms
AGGREGATION_NODE (id=3):(Total: 27s054ms, non-child: 482.904ms, % non-child: 1.78%)
- BuildTime: 420.146ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 42.00 KB (43008)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
HASH_JOIN_NODE (id=2):(Total: 26s571ms, non-child: 25s052ms, % non-child: 94.28%)
- BuildRows: 10.00M (10000000)
- BuildTime: 1s262ms
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 352.05 MB (369151232)
- ProbeRows: 266.67M (266666666)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 23s514ms
- RowsReturned: 266.67M (266666666)
- RowsReturnedRate: 10.06 M/sec
Buffer pool:
- AllocTime: 755.186us
- CumulativeAllocationBytes: 352.00 MB (369098752)
- CumulativeAllocations: 64 (64)
- PeakReservation: 352.00 MB (369098752)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 352.00 MB (369098752)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
- BuildRowsPartitionTime: 779.256ms
- BuildRowsPartitioned: 10.00M (10000000)
- HashBuckets: 16.78M (16777216)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 478.085ms
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 404.335ms, non-child: 404.335ms, % non-child: 100.00%)
- ConvertRowBatchTime: 198.830ms
- PeakMemoryUsage: 0
- RowsReturned: 10.00M (10000000)
- RowsReturnedRate: 24.86 M/sec
DataStreamReceiver:
- BytesReceived: 76.43 MB (80147364)
- DeserializeRowBatchTimer: 234.379ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 7.28 MB (7636309)
- SendersBlockedTimer: 16.509ms
- SendersBlockedTotalTimer(*): 28.137ms
HDFS_SCAN_NODE (id=1):(Total: 1s115ms, non-child: 1s115ms, % non-child: 100.00%)
- AverageHdfsReadThreadConcurrency: 0.23
- AverageScannerThreadConcurrency: 31.33
- BytesRead: 1.01 GB (1084002883)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 1.01 GB (1084002883)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 1.01 GB (1084002883)
- CachedFileHandlesHitCount: 0 (0)
- CachedFileHandlesMissCount: 113 (113)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 6s364ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 5 (5)
- NumRowGroups: 56 (56)
- NumScannerThreadsStarted: 40 (40)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 803.77 MB (842808825)
- PerReadThreadRawHdfsThroughput: 167.88 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 146.446ms
- RowBatchQueuePutWaitTime: 13m28s
- RowsRead: 266.67M (266666666)
- RowsReturned: 266.67M (266666666)
- RowsReturnedRate: 239.18 M/sec
- ScanRangesComplete: 56 (56)
- ScannerThreadsInvoluntaryContextSwitches: 197 (197)
- ScannerThreadsTotalWallClockTime: 14m2s
- MaterializeTupleTime(*): 6s493ms
- ScannerThreadsSysTime: 644.000ms
- ScannerThreadsUserTime: 14s249ms
- ScannerThreadsVoluntaryContextSwitches: 260.77K (260772)
- TotalRawHdfsReadTime(*): 6s166ms
- TotalReadThroughput: 38.18 MB/sec
Filter 0 (4.00 MB):
- Files processed: 0 (0)
- Files rejected: 0 (0)
- Files total: 0 (0)
- RowGroups processed: 0 (0)
- RowGroups rejected: 0 (0)
- RowGroups total: 0 (0)
- Rows processed: 928.43K (928426)
- Rows rejected: 0 (0)
- Rows total: 266.67M (266666666)
- Splits processed: 0 (0)
- Splits rejected: 0 (0)
- Splits total: 0 (0)
Fragment F00:
Instance 1542725b414611ca:d430e72b00000004 (host=hadoop958.hz.163.org:22000):(Total: 28s546ms, non-child: 8.867ms, % non-child: 0.03%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:11/2.50 GB 2:10/2.46 GB 4:8/1.97 GB 5:9/2.21 GB 0:6/1.45 GB 3:14/3.26 GB
Filter 0 arrival: 1s603ms
MemoryUsage(500.000ms): 70.25 KB, 68.13 MB, 106.00 MB, 212.08 MB, 487.10 MB, 953.22 MB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.12 GB, 1.12 GB, 1.08 GB, 1.01 GB, 870.48 MB, 838.60 MB, 838.60 MB, 838.60 MB, 838.60 MB, 838.60 MB, 838.60 MB, 838.60 MB, 838.60 MB, 838.58 MB, 838.60 MB, 838.60 MB, 830.66 MB, 830.66 MB, 790.73 MB, 718.73 MB, 627.26 MB, 555.41 MB, 555.42 MB, 619.04 MB, 746.67 MB, 691.26 MB, 691.26 MB, 687.19 MB, 687.18 MB, 679.10 MB, 663.16 MB, 631.28 MB, 551.60 MB, 551.60 MB, 551.60 MB, 551.60 MB, 551.66 MB, 535.72 MB, 435.92 MB, 415.75 MB
ThreadUsage(500.000ms): 1, 2, 2, 2, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 19, 19, 18, 18, 17, 17, 17, 17, 17, 17, 17, 17, 17, 16, 15
- AverageThreadTokens: 31.98
- BloomFilterBytes: 4.00 MB (4194304)
- PeakMemoryUsage: 1.13 GB (1213596728)
- PeakReservation: 352.00 MB (369098752)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 1.13 GB (1213609016)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 244.922ms
- TotalNetworkSendTime: 171.527us
- TotalStorageWaitTime: 14s187ms
- TotalThreadsInvoluntaryContextSwitches: 217 (217)
- TotalThreadsTotalWallClockTime: 15m10s
- TotalThreadsSysTime: 1s324ms
- TotalThreadsUserTime: 41s612ms
- TotalThreadsVoluntaryContextSwitches: 265.56K (265558)
Fragment Instance Lifecycle Timings:
- ExecTime: 240.970us
- ExecTreeExecTime: 3.080us
- OpenTime: 28s439ms
- ExecTreeOpenTime: 28s167ms
- PrepareTime: 106.457ms
- ExecTreePrepareTime: 60.351ms
DataStreamSender (dst_id=5):(Total: 153.180us, non-child: 153.180us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 450.25 KB/sec
- OverallThroughput: 102.00 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 6.816us
- TransmitDataRPCTime: 34.703us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 309.586ms, non-child: 309.586ms, % non-child: 100.00%)
- CodegenTime: 2.580ms
- CompileTime: 81.002ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 87 (87)
- NumInstructions: 2.88K (2881)
- OptimizationTime: 190.067ms
- PeakMemoryUsage: 1.41 MB (1475072)
- PrepareTime: 37.553ms
AGGREGATION_NODE (id=3):(Total: 28s228ms, non-child: 489.880ms, % non-child: 1.74%)
ExecOption: Codegen Enabled
- BuildTime: 415.866ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 42.00 KB (43008)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
HASH_JOIN_NODE (id=2):(Total: 27s738ms, non-child: 26s185ms, % non-child: 94.40%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 10.00M (10000000)
- BuildTime: 1s257ms
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 352.05 MB (369151232)
- ProbeRows: 267.69M (267694474)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 24s646ms
- RowsReturned: 267.69M (267694474)
- RowsReturnedRate: 9.65 M/sec
Buffer pool:
- AllocTime: 810.862us
- CumulativeAllocationBytes: 352.00 MB (369098752)
- CumulativeAllocations: 64 (64)
- PeakReservation: 352.00 MB (369098752)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 352.00 MB (369098752)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
Runtime filters: 1 of 1 Runtime Filter Published
- BuildRowsPartitionTime: 762.689ms
- BuildRowsPartitioned: 10.00M (10000000)
- HashBuckets: 16.78M (16777216)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 489.674ms
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 446.852ms, non-child: 446.852ms, % non-child: 100.00%)
- ConvertRowBatchTime: 194.969ms
- PeakMemoryUsage: 0
- RowsReturned: 10.00M (10000000)
- RowsReturnedRate: 22.38 M/sec
DataStreamReceiver:
BytesReceived(500.000ms): 0, 30.00 MB, 63.89 MB, 76.43 MB
- BytesReceived: 76.43 MB (80147364)
- DeserializeRowBatchTimer: 231.533ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 6.06 MB (6353920)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
HDFS_SCAN_NODE (id=1):(Total: 1s105ms, non-child: 1s105ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:11/2.50 GB 2:10/2.46 GB 4:8/1.97 GB 5:9/2.21 GB 0:6/1.45 GB 3:14/3.26 GB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 58 out of 58
Runtime filters: All filters arrived. Waited 0
Hdfs Read Thread Concurrency Bucket: 0:94.64% 1:1.786% 2:0% 3:0% 4:1.786% 5:0% 6:1.786% 7:0% 8:0% 9:0% 10:0%
File Formats: PARQUET/SNAPPY:58
BytesRead(500.000ms): 0, 0, 0, 0, 98.73 MB, 539.39 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 734.79 MB, 753.22 MB, 771.64 MB, 771.64 MB, 848.86 MB, 990.79 MB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB
- FooterProcessingTime: (Avg: 80.884ms ; Min: 9.902ms ; Max: 275.815ms ; Number of samples: 58)
- AverageHdfsReadThreadConcurrency: 0.20
- AverageScannerThreadConcurrency: 31.48
- BytesRead: 1.01 GB (1088383085)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 1.01 GB (1088383085)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 1.01 GB (1088383085)
- CachedFileHandlesHitCount: 0 (0)
- CachedFileHandlesMissCount: 116 (116)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 6s352ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 6 (6)
- NumRowGroups: 58 (58)
- NumScannerThreadsStarted: 40 (40)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 801.26 MB (840184650)
- PerReadThreadRawHdfsThroughput: 161.02 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 111.138ms
- RowBatchQueuePutWaitTime: 14m10s
- RowsRead: 267.69M (267694474)
- RowsReturned: 267.69M (267694474)
- RowsReturnedRate: 242.06 M/sec
- ScanRangesComplete: 58 (58)
- ScannerThreadsInvoluntaryContextSwitches: 165 (165)
- ScannerThreadsTotalWallClockTime: 14m40s
- MaterializeTupleTime(*): 6s496ms
- ScannerThreadsSysTime: 660.000ms
- ScannerThreadsUserTime: 14s188ms
- ScannerThreadsVoluntaryContextSwitches: 261.76K (261761)
- TotalRawHdfsReadTime(*): 6s446ms
- TotalReadThroughput: 36.42 MB/sec
Filter 0 (4.00 MB):
- Files processed: 0 (0)
- Files rejected: 0 (0)
- Files total: 0 (0)
- RowGroups processed: 0 (0)
- RowGroups rejected: 0 (0)
- RowGroups total: 0 (0)
- Rows processed: 950.27K (950272)
- Rows rejected: 0 (0)
- Rows total: 267.69M (267694474)
- Splits processed: 0 (0)
- Splits rejected: 0 (0)
- Splits total: 0 (0)
Instance 1542725b414611ca:d430e72b00000006 (host=hadoop957.hz.163.org:22000):(Total: 28s192ms, non-child: 9.115ms, % non-child: 0.03%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 4:7/1.72 GB 5:10/2.46 GB 2:13/3.20 GB 1:18/4.43 GB 3:8/1.97 GB
Filter 0 arrival: 1s603ms
MemoryUsage(500.000ms): 1.48 MB, 41.70 MB, 100.09 MB, 212.08 MB, 438.19 MB, 844.62 MB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.11 GB, 1.06 GB, 1001.66 MB, 882.19 MB, 842.35 MB, 842.35 MB, 842.35 MB, 842.35 MB, 842.35 MB, 842.35 MB, 842.35 MB, 842.35 MB, 842.33 MB, 842.35 MB, 842.35 MB, 842.35 MB, 826.41 MB, 794.60 MB, 714.73 MB, 595.10 MB, 523.36 MB, 571.03 MB, 638.48 MB, 717.82 MB, 679.10 MB, 679.10 MB, 679.09 MB, 679.09 MB, 679.09 MB, 647.23 MB, 551.60 MB, 551.60 MB, 551.60 MB, 551.60 MB, 551.60 MB, 551.60 MB, 471.72 MB, 423.85 MB
ThreadUsage(500.000ms): 1, 2, 2, 2, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 37, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17
- AverageThreadTokens: 32.12
- BloomFilterBytes: 4.00 MB (4194304)
- PeakMemoryUsage: 1.13 GB (1217528891)
- PeakReservation: 352.00 MB (369098752)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 1.13 GB (1217528891)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 183.689ms
- TotalNetworkSendTime: 183.503us
- TotalStorageWaitTime: 17s145ms
- TotalThreadsInvoluntaryContextSwitches: 282 (282)
- TotalThreadsTotalWallClockTime: 14m58s
- TotalThreadsSysTime: 1s212ms
- TotalThreadsUserTime: 41s688ms
- TotalThreadsVoluntaryContextSwitches: 263.16K (263161)
Fragment Instance Lifecycle Timings:
- ExecTime: 281.569us
- ExecTreeExecTime: 3.636us
- OpenTime: 28s072ms
- ExecTreeOpenTime: 27s753ms
- PrepareTime: 120.268ms
- ExecTreePrepareTime: 73.614ms
DataStreamSender (dst_id=5):(Total: 134.786us, non-child: 134.786us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 848.17 KB/sec
- OverallThroughput: 115.92 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 7.912us
- TransmitDataRPCTime: 18.422us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 356.164ms, non-child: 356.164ms, % non-child: 100.00%)
- CodegenTime: 2.682ms
- CompileTime: 102.421ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 87 (87)
- NumInstructions: 2.88K (2881)
- OptimizationTime: 214.814ms
- PeakMemoryUsage: 1.41 MB (1475072)
- PrepareTime: 37.933ms
AGGREGATION_NODE (id=3):(Total: 27s827ms, non-child: 471.043ms, % non-child: 1.69%)
ExecOption: Codegen Enabled
- BuildTime: 410.778ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 42.00 KB (43008)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
HASH_JOIN_NODE (id=2):(Total: 27s356ms, non-child: 25s868ms, % non-child: 94.56%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 10.00M (10000000)
- BuildTime: 1s247ms
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 352.05 MB (369151232)
- ProbeRows: 266.15M (266153451)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 24s349ms
- RowsReturned: 266.15M (266153451)
- RowsReturnedRate: 9.73 M/sec
Buffer pool:
- AllocTime: 571.972us
- CumulativeAllocationBytes: 352.00 MB (369098752)
- CumulativeAllocations: 64 (64)
- PeakReservation: 352.00 MB (369098752)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 352.00 MB (369098752)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
Runtime filters: 1 of 1 Runtime Filter Published
- BuildRowsPartitionTime: 767.790ms
- BuildRowsPartitioned: 10.00M (10000000)
- HashBuckets: 16.78M (16777216)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 475.106ms
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 380.792ms, non-child: 380.792ms, % non-child: 100.00%)
- ConvertRowBatchTime: 189.741ms
- PeakMemoryUsage: 0
- RowsReturned: 10.00M (10000000)
- RowsReturnedRate: 26.26 M/sec
DataStreamReceiver:
BytesReceived(500.000ms): 0, 29.64 MB, 63.53 MB, 76.43 MB
- BytesReceived: 76.43 MB (80147364)
- DeserializeRowBatchTimer: 229.670ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.01 MB (10497024)
- SendersBlockedTimer: 49.528ms
- SendersBlockedTotalTimer(*): 84.411ms
HDFS_SCAN_NODE (id=1):(Total: 1s107ms, non-child: 1s107ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 4:7/1.72 GB 5:10/2.46 GB 2:13/3.20 GB 1:18/4.43 GB 3:8/1.97 GB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 56 out of 56
Runtime filters: All filters arrived. Waited 0
Hdfs Read Thread Concurrency Bucket: 0:90.91% 1:1.818% 2:3.636% 3:0% 4:1.818% 5:1.818% 6:0% 7:0% 8:0% 9:0% 10:0%
File Formats: PARQUET/SNAPPY:56
BytesRead(500.000ms): 0, 0, 0, 0, 41.95 MB, 452.19 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 736.92 MB, 781.96 MB, 871.85 MB, 988.07 MB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB
- FooterProcessingTime: (Avg: 106.280ms ; Min: 11.046ms ; Max: 227.568ms ; Number of samples: 56)
- AverageHdfsReadThreadConcurrency: 0.25
- AverageScannerThreadConcurrency: 31.64
- BytesRead: 1.01 GB (1081810605)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 1.01 GB (1081810605)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 1.01 GB (1081810605)
- CachedFileHandlesHitCount: 0 (0)
- CachedFileHandlesMissCount: 112 (112)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 6s493ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 5 (5)
- NumRowGroups: 56 (56)
- NumScannerThreadsStarted: 40 (40)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 805.01 MB (844116813)
- PerReadThreadRawHdfsThroughput: 166.64 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 173.032ms
- RowBatchQueuePutWaitTime: 13m53s
- RowsRead: 266.15M (266153451)
- RowsReturned: 266.15M (266153451)
- RowsReturnedRate: 240.41 M/sec
- ScanRangesComplete: 56 (56)
- ScannerThreadsInvoluntaryContextSwitches: 231 (231)
- ScannerThreadsTotalWallClockTime: 14m28s
- MaterializeTupleTime(*): 6s507ms
- ScannerThreadsSysTime: 688.000ms
- ScannerThreadsUserTime: 14s492ms
- ScannerThreadsVoluntaryContextSwitches: 260.31K (260305)
- TotalRawHdfsReadTime(*): 6s191ms
- TotalReadThroughput: 36.85 MB/sec
Filter 0 (4.00 MB):
- Files processed: 0 (0)
- Files rejected: 0 (0)
- Files total: 0 (0)
- RowGroups processed: 0 (0)
- RowGroups rejected: 0 (0)
- RowGroups total: 0 (0)
- Rows processed: 917.50K (917504)
- Rows rejected: 0 (0)
- Rows total: 266.15M (266153451)
- Splits processed: 0 (0)
- Splits rejected: 0 (0)
- Splits total: 0 (0)
Instance 1542725b414611ca:d430e72b00000005 (host=hadoop956.hz.163.org:22000):(Total: 25s429ms, non-child: 9.044ms, % non-child: 0.04%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 5:8/1.97 GB 4:13/3.20 GB 2:5/1.23 GB 3:7/1.72 GB 0:23/5.65 GB
Filter 0 arrival: 1s603ms
MemoryUsage(500.000ms): 43.12 MB, 68.65 MB, 100.28 MB, 308.08 MB, 575.29 MB, 992.94 MB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.11 GB, 1.07 GB, 1009.72 MB, 898.16 MB, 842.35 MB, 842.35 MB, 842.35 MB, 842.35 MB, 842.35 MB, 842.35 MB, 842.35 MB, 842.35 MB, 842.35 MB, 842.35 MB, 826.35 MB, 794.60 MB, 730.98 MB, 643.42 MB, 539.94 MB, 595.53 MB, 658.93 MB, 689.36 MB, 679.09 MB, 679.10 MB, 679.10 MB, 679.09 MB, 647.22 MB, 559.56 MB, 551.60 MB, 551.60 MB, 551.60 MB, 551.60 MB, 503.92 MB, 424.10 MB
ThreadUsage(500.000ms): 2, 2, 2, 2, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 32, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17
- AverageThreadTokens: 31.94
- BloomFilterBytes: 4.00 MB (4194304)
- PeakMemoryUsage: 1.13 GB (1217528898)
- PeakReservation: 352.00 MB (369098752)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 1.13 GB (1217537090)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 165.376ms
- TotalNetworkSendTime: 197.936us
- TotalStorageWaitTime: 16s908ms
- TotalThreadsInvoluntaryContextSwitches: 240 (240)
- TotalThreadsTotalWallClockTime: 13m24s
- TotalThreadsSysTime: 1s136ms
- TotalThreadsUserTime: 38s508ms
- TotalThreadsVoluntaryContextSwitches: 263.08K (263083)
Fragment Instance Lifecycle Timings:
- ExecTime: 319.458us
- ExecTreeExecTime: 4.017us
- OpenTime: 25s310ms
- ExecTreeOpenTime: 25s035ms
- PrepareTime: 118.493ms
- ExecTreePrepareTime: 71.959ms
DataStreamSender (dst_id=5):(Total: 165.308us, non-child: 165.308us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 598.66 KB/sec
- OverallThroughput: 94.52 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 7.044us
- TransmitDataRPCTime: 26.100us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 313.049ms, non-child: 313.049ms, % non-child: 100.00%)
- CodegenTime: 2.637ms
- CompileTime: 81.297ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 87 (87)
- NumInstructions: 2.88K (2881)
- OptimizationTime: 192.773ms
- PeakMemoryUsage: 1.41 MB (1475072)
- PrepareTime: 37.945ms
AGGREGATION_NODE (id=3):(Total: 25s107ms, non-child: 487.789ms, % non-child: 1.94%)
ExecOption: Codegen Enabled
- BuildTime: 433.793ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 42.00 KB (43008)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
HASH_JOIN_NODE (id=2):(Total: 24s619ms, non-child: 23s102ms, % non-child: 93.84%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 10.00M (10000000)
- BuildTime: 1s281ms
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 352.05 MB (369151232)
- ProbeRows: 266.15M (266152075)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 21s548ms
- RowsReturned: 266.15M (266152075)
- RowsReturnedRate: 10.81 M/sec
Buffer pool:
- AllocTime: 882.726us
- CumulativeAllocationBytes: 352.00 MB (369098752)
- CumulativeAllocations: 64 (64)
- PeakReservation: 352.00 MB (369098752)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 352.00 MB (369098752)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
Runtime filters: 1 of 1 Runtime Filter Published
- BuildRowsPartitionTime: 807.290ms
- BuildRowsPartitioned: 10.00M (10000000)
- HashBuckets: 16.78M (16777216)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 469.474ms
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 385.360ms, non-child: 385.360ms, % non-child: 100.00%)
- ConvertRowBatchTime: 211.781ms
- PeakMemoryUsage: 0
- RowsReturned: 10.00M (10000000)
- RowsReturnedRate: 25.95 M/sec
DataStreamReceiver:
BytesReceived(500.000ms): 5.63 MB, 40.26 MB, 71.68 MB, 76.43 MB
- BytesReceived: 76.43 MB (80147364)
- DeserializeRowBatchTimer: 241.935ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 5.78 MB (6057984)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
HDFS_SCAN_NODE (id=1):(Total: 1s132ms, non-child: 1s132ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 5:8/1.97 GB 4:13/3.20 GB 2:5/1.23 GB 3:7/1.72 GB 0:23/5.65 GB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 56 out of 56
Runtime filters: All filters arrived. Waited 0
Hdfs Read Thread Concurrency Bucket: 0:90% 1:6% 2:0% 3:0% 4:2% 5:2% 6:0% 7:0% 8:0% 9:0% 10:0%
File Formats: PARQUET/SNAPPY:56
BytesRead(500.000ms): 0, 0, 0, 0, 179.22 MB, 576.08 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 736.93 MB, 755.36 MB, 818.73 MB, 900.52 MB, 985.75 MB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB
- FooterProcessingTime: (Avg: 107.583ms ; Min: 12.611ms ; Max: 279.000ms ; Number of samples: 56)
- AverageHdfsReadThreadConcurrency: 0.24
- AverageScannerThreadConcurrency: 30.86
- BytesRead: 1.01 GB (1081814959)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 1.01 GB (1081814959)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 1.01 GB (1081814959)
- CachedFileHandlesHitCount: 0 (0)
- CachedFileHandlesMissCount: 112 (112)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 6s247ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 5 (5)
- NumRowGroups: 56 (56)
- NumScannerThreadsStarted: 40 (40)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 805.02 MB (844125012)
- PerReadThreadRawHdfsThroughput: 175.97 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 155.167ms
- RowBatchQueuePutWaitTime: 12m22s
- RowsRead: 266.15M (266152075)
- RowsReturned: 266.15M (266152075)
- RowsReturnedRate: 235.08 M/sec
- ScanRangesComplete: 56 (56)
- ScannerThreadsInvoluntaryContextSwitches: 196 (196)
- ScannerThreadsTotalWallClockTime: 12m57s
- MaterializeTupleTime(*): 6s475ms
- ScannerThreadsSysTime: 584.000ms
- ScannerThreadsUserTime: 14s068ms
- ScannerThreadsVoluntaryContextSwitches: 260.25K (260250)
- TotalRawHdfsReadTime(*): 5s862ms
- TotalReadThroughput: 41.27 MB/sec
Filter 0 (4.00 MB):
- Files processed: 0 (0)
- Files rejected: 0 (0)
- Files total: 0 (0)
- RowGroups processed: 0 (0)
- RowGroups rejected: 0 (0)
- RowGroups total: 0 (0)
- Rows processed: 917.50K (917504)
- Rows rejected: 0 (0)
- Rows total: 266.15M (266152075)
- Splits processed: 0 (0)
- Splits rejected: 0 (0)
- Splits total: 0 (0)
Averaged Fragment F01:(Total: 1s480ms, non-child: 532.646ms, % non-child: 35.98%)
split sizes: min: 223.57 MB, max: 286.03 MB, avg: 254.02 MB, stddev: 25.52 MB
completion times: min:1s326ms max:1s603ms mean: 1s481ms stddev:115.269ms
execution rates: min:166.79 MB/sec max:178.40 MB/sec mean:171.24 MB/sec stddev:5.12 MB/sec
num instances: 3
- AverageThreadTokens: 2.11
- BloomFilterBytes: 0
- PeakMemoryUsage: 17.36 MB (18199813)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 111.53 MB (116950173)
- RowsProduced: 3.33M (3333333)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 774.773ms
- TotalStorageWaitTime: 454.499ms
- TotalThreadsInvoluntaryContextSwitches: 3 (3)
- TotalThreadsTotalWallClockTime: 3s010ms
- TotalThreadsSysTime: 44.000ms
- TotalThreadsUserTime: 449.333ms
- TotalThreadsVoluntaryContextSwitches: 7.79K (7790)
Fragment Instance Lifecycle Timings:
- ExecTime: 1s370ms
- ExecTreeExecTime: 271.785ms
- OpenTime: 17.112ms
- ExecTreeOpenTime: 46.619us
- PrepareTime: 93.251ms
- ExecTreePrepareTime: 68.319ms
DataStreamSender (dst_id=4):(Total: 499.551ms, non-child: 499.551ms, % non-child: 100.00%)
- BytesSent: 76.43 MB (80147364)
- NetworkThroughput(*): 372.82 MB/sec
- OverallThroughput: 156.80 MB/sec
- PeakMemoryUsage: 42.66 KB (43680)
- RowsReturned: 3.33M (3333333)
- SerializeBatchTime: 292.682ms
- TransmitDataRPCTime: 206.204ms
- UncompressedRowBatchSize: 162.12 MB (170000000)
CodeGen:(Total: 41.152ms, non-child: 41.152ms, % non-child: 100.00%)
- CodegenTime: 581.173us
- CompileTime: 6.335ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 7 (7)
- NumInstructions: 105 (105)
- OptimizationTime: 9.919ms
- PeakMemoryUsage: 52.50 KB (53760)
- PrepareTime: 24.128ms
HDFS_SCAN_NODE (id=0):(Total: 407.210ms, non-child: 407.210ms, % non-child: 100.00%)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.11
- BytesRead: 13.04 MB (13677091)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 13.04 MB (13677091)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 13.04 MB (13677091)
- CachedFileHandlesHitCount: 0 (0)
- CachedFileHandlesMissCount: 3 (3)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 70.920ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 1 (1)
- NumRowGroups: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 17.30 MB (18141563)
- PerReadThreadRawHdfsThroughput: 161.57 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 259.514ms
- RowBatchQueuePutWaitTime: 1s051ms
- RowsRead: 3.33M (3333333)
- RowsReturned: 3.33M (3333333)
- RowsReturnedRate: 8.23 M/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 1 (1)
- ScannerThreadsTotalWallClockTime: 1s623ms
- MaterializeTupleTime(*): 34.082ms
- ScannerThreadsSysTime: 8.000ms
- ScannerThreadsUserTime: 117.333ms
- ScannerThreadsVoluntaryContextSwitches: 3.15K (3149)
- TotalRawHdfsReadTime(*): 85.390ms
- TotalReadThroughput: 8.70 MB/sec
Fragment F01:
Instance 1542725b414611ca:d430e72b00000001 (host=hadoop958.hz.163.org:22000):(Total: 1s602ms, non-child: 570.033ms, % non-child: 35.57%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:1/252.49 MB 4:1/33.54 MB
MemoryUsage(500.000ms): 306.88 KB, 17.80 MB, 9.83 MB
ThreadUsage(500.000ms): 3, 2, 2
- AverageThreadTokens: 2.33
- BloomFilterBytes: 0
- PeakMemoryUsage: 20.11 MB (21091748)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 116.15 MB (121787512)
- RowsProduced: 3.76M (3755601)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 839.905ms
- TotalStorageWaitTime: 554.874ms
- TotalThreadsInvoluntaryContextSwitches: 5 (5)
- TotalThreadsTotalWallClockTime: 3s402ms
- TotalThreadsSysTime: 44.000ms
- TotalThreadsUserTime: 520.000ms
- TotalThreadsVoluntaryContextSwitches: 8.61K (8610)
Fragment Instance Lifecycle Timings:
- ExecTime: 1s499ms
- ExecTreeExecTime: 261.945ms
- OpenTime: 17.589ms
- ExecTreeOpenTime: 60.144us
- PrepareTime: 85.641ms
- ExecTreePrepareTime: 60.010ms
DataStreamSender (dst_id=4):(Total: 609.857ms, non-child: 609.857ms, % non-child: 100.00%)
- BytesSent: 86.12 MB (90300894)
- NetworkThroughput(*): 348.81 MB/sec
- OverallThroughput: 141.21 MB/sec
- PeakMemoryUsage: 42.66 KB (43680)
- RowsReturned: 3.76M (3755601)
- SerializeBatchTime: 362.207ms
- TransmitDataRPCTime: 246.892ms
- UncompressedRowBatchSize: 182.66 MB (191535651)
CodeGen:(Total: 42.262ms, non-child: 42.262ms, % non-child: 100.00%)
- CodegenTime: 584.211us
- CompileTime: 6.417ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 7 (7)
- NumInstructions: 105 (105)
- OptimizationTime: 10.283ms
- PeakMemoryUsage: 52.50 KB (53760)
- PrepareTime: 24.765ms
HDFS_SCAN_NODE (id=0):(Total: 380.459ms, non-child: 380.459ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:1/252.49 MB 4:1/33.54 MB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 2 out of 2
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0%
File Formats: PARQUET/SNAPPY:2
BytesRead(500.000ms): 0, 14.72 MB, 14.72 MB
- FooterProcessingTime: (Avg: 228.376ms ; Min: 228.241ms ; Max: 228.511ms ; Number of samples: 2)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.33
- BytesRead: 14.72 MB (15435328)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 14.72 MB (15435328)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 14.72 MB (15435328)
- CachedFileHandlesHitCount: 0 (0)
- CachedFileHandlesMissCount: 4 (4)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 64.810ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 2 (2)
- NumRowGroups: 2 (2)
- NumScannerThreadsStarted: 2 (2)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 20.06 MB (21030426)
- PerReadThreadRawHdfsThroughput: 121.54 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 247.077ms
- RowBatchQueuePutWaitTime: 1s220ms
- RowsRead: 3.76M (3755601)
- RowsReturned: 3.76M (3755601)
- RowsReturnedRate: 9.87 M/sec
- ScanRangesComplete: 2 (2)
- ScannerThreadsInvoluntaryContextSwitches: 2 (2)
- ScannerThreadsTotalWallClockTime: 1s886ms
- MaterializeTupleTime(*): 32.460ms
- ScannerThreadsSysTime: 8.000ms
- ScannerThreadsUserTime: 108.000ms
- ScannerThreadsVoluntaryContextSwitches: 3.57K (3568)
- TotalRawHdfsReadTime(*): 121.116ms
- TotalReadThroughput: 9.81 MB/sec
Instance 1542725b414611ca:d430e72b00000002 (host=hadoop956.hz.163.org:22000):(Total: 1s513ms, non-child: 517.696ms, % non-child: 34.22%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 2:1/252.48 MB
MemoryUsage(500.000ms): 17.80 MB, 17.80 MB, 9.83 MB
ThreadUsage(500.000ms): 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 17.84 MB (18707399)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 112.81 MB (118286517)
- RowsProduced: 3.32M (3318580)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 745.047ms
- TotalStorageWaitTime: 272.354ms
- TotalThreadsInvoluntaryContextSwitches: 5 (5)
- TotalThreadsTotalWallClockTime: 2s785ms
- TotalThreadsSysTime: 36.000ms
- TotalThreadsUserTime: 516.000ms
- TotalThreadsVoluntaryContextSwitches: 7.68K (7682)
Fragment Instance Lifecycle Timings:
- ExecTime: 1s399ms
- ExecTreeExecTime: 288.439ms
- OpenTime: 16.891ms
- ExecTreeOpenTime: 42.344us
- PrepareTime: 96.469ms
- ExecTreePrepareTime: 71.650ms
DataStreamSender (dst_id=4):(Total: 524.217ms, non-child: 524.217ms, % non-child: 100.00%)
- BytesSent: 76.09 MB (79790373)
- NetworkThroughput(*): 399.49 MB/sec
- OverallThroughput: 145.16 MB/sec
- PeakMemoryUsage: 42.66 KB (43680)
- RowsReturned: 3.32M (3318580)
- SerializeBatchTime: 332.990ms
- TransmitDataRPCTime: 190.480ms
- UncompressedRowBatchSize: 161.41 MB (169247580)
CodeGen:(Total: 40.819ms, non-child: 40.819ms, % non-child: 100.00%)
- CodegenTime: 587.076us
- CompileTime: 6.357ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 7 (7)
- NumInstructions: 105 (105)
- OptimizationTime: 9.721ms
- PeakMemoryUsage: 52.50 KB (53760)
- PrepareTime: 23.983ms
HDFS_SCAN_NODE (id=0):(Total: 430.300ms, non-child: 430.300ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 2:1/252.48 MB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0%
File Formats: PARQUET/SNAPPY:1
BytesRead(500.000ms): 12.87 MB, 12.87 MB, 12.87 MB
- FooterProcessingTime: (Avg: 226.238ms ; Min: 226.238ms ; Max: 226.238ms ; Number of samples: 1)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.00
- BytesRead: 12.87 MB (13495185)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 12.87 MB (13495185)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 12.87 MB (13495185)
- CachedFileHandlesHitCount: 0 (0)
- CachedFileHandlesMissCount: 2 (2)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 78.645ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 1 (1)
- NumRowGroups: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 17.79 MB (18655293)
- PerReadThreadRawHdfsThroughput: 199.83 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 275.187ms
- RowBatchQueuePutWaitTime: 968.648ms
- RowsRead: 3.32M (3318580)
- RowsReturned: 3.32M (3318580)
- RowsReturnedRate: 7.71 M/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 2 (2)
- ScannerThreadsTotalWallClockTime: 1s369ms
- MaterializeTupleTime(*): 37.023ms
- ScannerThreadsSysTime: 4.000ms
- ScannerThreadsUserTime: 136.000ms
- ScannerThreadsVoluntaryContextSwitches: 3.13K (3127)
- TotalRawHdfsReadTime(*): 64.406ms
- TotalReadThroughput: 8.58 MB/sec
Instance 1542725b414611ca:d430e72b00000003 (host=hadoop957.hz.163.org:22000):(Total: 1s326ms, non-child: 510.208ms, % non-child: 38.48%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 5:1/34.12 MB 1:1/189.45 MB
MemoryUsage(500.000ms): 306.88 KB, 11.80 MB, 1.29 MB
ThreadUsage(500.000ms): 3, 2, 1
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 14.11 MB (14800292)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 105.64 MB (110776490)
- RowsProduced: 2.93M (2925819)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 739.368ms
- TotalStorageWaitTime: 536.269ms
- TotalThreadsInvoluntaryContextSwitches: 1 (1)
- TotalThreadsTotalWallClockTime: 2s844ms
- TotalThreadsSysTime: 52.000ms
- TotalThreadsUserTime: 312.000ms
- TotalThreadsVoluntaryContextSwitches: 7.08K (7078)
Fragment Instance Lifecycle Timings:
- ExecTime: 1s211ms
- ExecTreeExecTime: 264.971ms
- OpenTime: 16.855ms
- ExecTreeOpenTime: 37.369us
- PrepareTime: 97.642ms
- ExecTreePrepareTime: 73.295ms
DataStreamSender (dst_id=4):(Total: 364.579ms, non-child: 364.579ms, % non-child: 100.00%)
- BytesSent: 67.09 MB (70350825)
- NetworkThroughput(*): 370.18 MB/sec
- OverallThroughput: 184.03 MB/sec
- PeakMemoryUsage: 42.66 KB (43680)
- RowsReturned: 2.93M (2925819)
- SerializeBatchTime: 182.850ms
- TransmitDataRPCTime: 181.240ms
- UncompressedRowBatchSize: 142.30 MB (149216769)
CodeGen:(Total: 40.376ms, non-child: 40.376ms, % non-child: 100.00%)
- CodegenTime: 572.232us
- CompileTime: 6.232ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980120)
- NumFunctions: 7 (7)
- NumInstructions: 105 (105)
- OptimizationTime: 9.751ms
- PeakMemoryUsage: 52.50 KB (53760)
- PrepareTime: 23.636ms
HDFS_SCAN_NODE (id=0):(Total: 410.872ms, non-child: 410.872ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 5:1/34.12 MB 1:1/189.45 MB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 2 out of 2
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0%
File Formats: PARQUET/SNAPPY:2
BytesRead(500.000ms): 0, 11.54 MB, 11.54 MB
- FooterProcessingTime: (Avg: 233.447ms ; Min: 233.378ms ; Max: 233.515ms ; Number of samples: 2)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.00
- BytesRead: 11.54 MB (12100762)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 11.54 MB (12100762)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 11.54 MB (12100762)
- CachedFileHandlesHitCount: 0 (0)
- CachedFileHandlesMissCount: 4 (4)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 69.306ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 2 (2)
- NumRowGroups: 2 (2)
- NumScannerThreadsStarted: 2 (2)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 14.06 MB (14738970)
- PerReadThreadRawHdfsThroughput: 163.35 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 256.278ms
- RowBatchQueuePutWaitTime: 964.664ms
- RowsRead: 2.93M (2925819)
- RowsReturned: 2.93M (2925819)
- RowsReturnedRate: 7.12 M/sec
- ScanRangesComplete: 2 (2)
- ScannerThreadsInvoluntaryContextSwitches: 1 (1)
- ScannerThreadsTotalWallClockTime: 1s616ms
- MaterializeTupleTime(*): 32.765ms
- ScannerThreadsSysTime: 12.000ms
- ScannerThreadsUserTime: 108.000ms
- ScannerThreadsVoluntaryContextSwitches: 2.75K (2753)
- TotalRawHdfsReadTime(*): 70.648ms
- TotalReadThroughput: 7.69 MB/sec