Query (id=68440bc099e1b843:8baf451500000000) Summary Session ID: 7e403fd95565e4aa:4a25223ed01ca49b Session Type: BEESWAX Start Time: 2018-04-17 07:13:26.525986000 End Time: 2018-04-17 07:14:13.615981000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.11.0-cdh5.14.0 RELEASE (build d68206561bce6b26762d62c01a78e6cd27aa7690) User: etl Connected User: etl Delegated User: Network Address: ::ffff:10.1.28.130:42007 Default Db: caastle_bi Sql Statement: select distinct input.customer_key, input.shipping_date_key, input.at_home_count, input.return_count, input.customer_uuid, emailTable.customer_email_address from (select customer_uuid as customer_uuid,customer_email_address as customer_email_address from table2 as dcl inner join table1 as dt on dcl.tenant_key = dt.tenant_key where dt.tenant_id = 1) as emailTable inner join (select a1.customer_uuid, a1.customer_key, a1.shipping_date_key, a1.at_home_count, a2.return_count from (select customer_uuid, E1.customer_key, shipping_date_key, count(*) as at_home_count from db1.table3 E1 join db1.table4 D1 ON E1.customer_key = D1.customer_key where closet_state_key = 1 group by 1,2,3 ) a1 join (select customer_key, shipping_date_key, count(*) as return_count from db1.table3 b1 join db1.table5 dates on dates.date_key = b1.customer_actual_return_date_key where to_date(date_key_date) = to_date(now() - interval 1 day) and closet_state_key = 0 and reason = 'returned' group by 1,2) a2 on a1.customer_key = a2.customer_key and a1.shipping_date_key = a2.shipping_date_key ) as input on input.customer_uuid = emailTable.customer_uuid Coordinator: ip-10-1-28-130.ec2.internal:22000 Query Options (set by configuration): MEM_LIMIT=10737418240 Query Options (set by configuration and planner): MEM_LIMIT=10737418240,MT_DOP=0 Plan: ---------------- Max Per-Host Resource Reservation: Memory=237.75MB Per-Host Resource Estimates: Memory=256.61GB F13:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=0B mem-reservation=0B PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B | 37:EXCHANGE [UNPARTITIONED] | mem-estimate=0B mem-reservation=0B | tuple-ids=21 row-size=118B cardinality=1061892932 | F12:PLAN FRAGMENT [HASH(input.customer_key,input.shipping_date_key,input.at_home_count,input.return_count,input.customer_uuid,emailTable.customer_email_address)] hosts=6 instances=6 Per-Host Resources: mem-estimate=127.90GB mem-reservation=34.00MB 36:AGGREGATE [FINALIZE] | group by: input.customer_key, input.shipping_date_key, input.at_home_count, input.return_count, input.customer_uuid, emailTable.customer_email_address | mem-estimate=127.90GB mem-reservation=34.00MB spill-buffer=2.00MB | tuple-ids=21 row-size=118B cardinality=1061892932 | 35:EXCHANGE [HASH(input.customer_key,input.shipping_date_key,input.at_home_count,input.return_count,input.customer_uuid,emailTable.customer_email_address)] | mem-estimate=0B mem-reservation=0B | tuple-ids=21 row-size=118B cardinality=1061892932 | F04:PLAN FRAGMENT [HASH(customer_uuid,E1.customer_key,shipping_date_key)] hosts=6 instances=6 Per-Host Resources: mem-estimate=128.08GB mem-reservation=110.50MB 19:AGGREGATE [STREAMING] | group by: E1.customer_key, shipping_date_key, count(*), count(*), customer_uuid, table4.customer_email_address | mem-estimate=127.90GB mem-reservation=34.00MB spill-buffer=2.00MB | tuple-ids=21 row-size=118B cardinality=1061892932 | 18:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: customer_uuid = table4.customer_uuid | fk/pk conjuncts: none | runtime filters: RF000[bloom] <- table4.customer_uuid | mem-estimate=68.89MB mem-reservation=34.00MB spill-buffer=2.00MB | tuple-ids=10,18,0,2 row-size=235B cardinality=1061892932 | |--34:EXCHANGE [BROADCAST] | | mem-estimate=0B mem-reservation=0B | | tuple-ids=0,2 row-size=143B cardinality=460592 | | | F10:PLAN FRAGMENT [RANDOM] hosts=6 instances=6 | Per-Host Resources: mem-estimate=161.94MB mem-reservation=1.94MB | 02:HASH JOIN [INNER JOIN, BROADCAST] | | hash predicates: table4.tenant_key = dt.tenant_key | | fk/pk conjuncts: table4.tenant_key = dt.tenant_key | | runtime filters: RF014[bloom] <- dt.tenant_key | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB | | tuple-ids=0,2 row-size=143B cardinality=460592 | | | |--33:EXCHANGE [BROADCAST] | | | mem-estimate=0B mem-reservation=0B | | | tuple-ids=2 row-size=12B cardinality=3 | | | | | F11:PLAN FRAGMENT [RANDOM] hosts=1 instances=1 | | Per-Host Resources: mem-estimate=32.00MB mem-reservation=0B | | 01:SCAN HDFS [db1.table1 dt, RANDOM] | | partitions=1/1 files=1 size=4.58KB | | predicates: dt.tenant_id = 1 | | stats-rows=8 extrapolated-rows=disabled | | table stats: rows=8 size=4.58KB | | column stats: all | | parquet statistics predicates: dt.tenant_id = 1 | | parquet dictionary predicates: dt.tenant_id = 1 | | mem-estimate=32.00MB mem-reservation=0B | | tuple-ids=2 row-size=12B cardinality=3 | | | 00:SCAN HDFS [db1.table4, RANDOM] | partitions=7/7 files=107 size=2.98GB | predicates: table4.scd_end IS NULL | runtime filters: RF014[bloom] -> table4.tenant_key | stats-rows=13817760 extrapolated-rows=disabled | table stats: rows=13817760 size=2.98GB | column stats: all | mem-estimate=160.00MB mem-reservation=0B | tuple-ids=0 row-size=131B cardinality=1381776 | 17:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: E1.customer_key = customer_key, shipping_date_key = shipping_date_key | fk/pk conjuncts: none | runtime filters: RF002[bloom] <- customer_key, RF003[bloom] <- shipping_date_key | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB | tuple-ids=10,18 row-size=92B cardinality=7378345143 | |--32:EXCHANGE [BROADCAST] | | mem-estimate=0B mem-reservation=0B | | tuple-ids=18 row-size=20B cardinality=399174 | | | F09:PLAN FRAGMENT [HASH(customer_key,shipping_date_key)] hosts=6 instances=6 | Per-Host Resources: mem-estimate=10.00MB mem-reservation=8.50MB | 31:AGGREGATE [FINALIZE] | | output: count:merge(*) | | group by: customer_key, shipping_date_key | | mem-estimate=10.00MB mem-reservation=8.50MB spill-buffer=512.00KB | | tuple-ids=18 row-size=20B cardinality=399174 | | | 30:EXCHANGE [HASH(customer_key,shipping_date_key)] | | mem-estimate=0B mem-reservation=0B | | tuple-ids=18 row-size=20B cardinality=399174 | | | F05:PLAN FRAGMENT [RANDOM] hosts=6 instances=6 | Per-Host Resources: mem-estimate=93.88MB mem-reservation=12.88MB | 16:AGGREGATE [STREAMING] | | output: count(*) | | group by: customer_key, shipping_date_key | | mem-estimate=10.00MB mem-reservation=9.00MB spill-buffer=512.00KB | | tuple-ids=18 row-size=20B cardinality=399174 | | | 15:HASH JOIN [INNER JOIN, BROADCAST] | | hash predicates: customer_actual_return_date_key = dates.date_key | | fk/pk conjuncts: customer_actual_return_date_key = dates.date_key | | runtime filters: RF010[bloom] <- dates.date_key | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB | | tuple-ids=12,17 row-size=72B cardinality=399174 | | | |--29:EXCHANGE [BROADCAST] | | | mem-estimate=0B mem-reservation=0B | | | tuple-ids=17 row-size=20B cardinality=1000 | | | | | F08:PLAN FRAGMENT [RANDOM] hosts=1 instances=1 | | Per-Host Resources: mem-estimate=32.00MB mem-reservation=0B | | 14:SCAN HDFS [db1.table5 dates, RANDOM] | | partitions=1/1 files=1 size=521.09KB | | predicates: to_date(date_key_date) = '2018-04-16' | | stats-rows=10000 extrapolated-rows=disabled | | table stats: rows=10000 size=521.09KB | | column stats: all | | parquet dictionary predicates: to_date(date_key_date) = '2018-04-16' | | mem-estimate=32.00MB mem-reservation=0B | | tuple-ids=17 row-size=20B cardinality=1000 | | | 13:HASH JOIN [LEFT SEMI JOIN, BROADCAST] | | hash predicates: date_key = max(date_key) | | runtime filters: RF012[bloom] <- max(date_key) | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB | | tuple-ids=12 row-size=52B cardinality=831080 | | | |--28:EXCHANGE [BROADCAST] | | | mem-estimate=0B mem-reservation=0B | | | tuple-ids=14 row-size=4B cardinality=1 | | | | | F07:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | | Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B | | 27:AGGREGATE [FINALIZE] | | | output: max:merge(date_key) | | | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB | | | tuple-ids=14 row-size=4B cardinality=1 | | | | | 26:EXCHANGE [UNPARTITIONED] | | | mem-estimate=0B mem-reservation=0B | | | tuple-ids=14 row-size=4B cardinality=1 | | | | | F06:PLAN FRAGMENT [RANDOM] hosts=6 instances=6 | | Per-Host Resources: mem-estimate=11.00MB mem-reservation=0B | | 12:AGGREGATE | | | output: max(date_key) | | | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB | | | tuple-ids=14 row-size=4B cardinality=1 | | | | | 11:SCAN HDFS [db1.table6, RANDOM] | | partitions=1955/1955 files=61513 size=101.44GB | | stats-rows=8153177297 extrapolated-rows=disabled | | table stats: rows=8153177297 size=unavailable | | column stats: all | | mem-estimate=1.00MB mem-reservation=0B | | tuple-ids=13 row-size=4B cardinality=8153177297 | | | 10:SCAN HDFS [db1.table6, RANDOM] | partitions=1955/1955 files=61513 size=101.44GB | predicates: db1.table6.closet_state_key = 0, db1.table6.reason = 'returned' | runtime filters: RF010[bloom] -> customer_actual_return_date_key, RF012[bloom] -> date_key | stats-rows=8153177297 extrapolated-rows=disabled | table stats: rows=8153177297 size=unavailable | column stats: all | parquet statistics predicates: db1.table6.closet_state_key = 0, db1.table6.reason = 'returned' | parquet dictionary predicates: db1.table6.closet_state_key = 0, db1.table6.reason = 'returned' | mem-estimate=80.00MB mem-reservation=0B | tuple-ids=12 row-size=52B cardinality=1176809777 | 25:AGGREGATE [FINALIZE] | output: count:merge(*) | group by: customer_uuid, E1.customer_key, shipping_date_key | mem-estimate=108.72MB mem-reservation=34.00MB spill-buffer=2.00MB | tuple-ids=10 row-size=72B cardinality=1439473 | 24:EXCHANGE [HASH(customer_uuid,E1.customer_key,shipping_date_key)] | mem-estimate=0B mem-reservation=0B | tuple-ids=10 row-size=72B cardinality=1439473 | F00:PLAN FRAGMENT [RANDOM] hosts=6 instances=6 Per-Host Resources: mem-estimate=224.97MB mem-reservation=68.00MB 09:AGGREGATE [STREAMING] | output: count(*) | group by: customer_uuid, customer_key, shipping_date_key | mem-estimate=108.72MB mem-reservation=34.00MB spill-buffer=2.00MB | tuple-ids=10 row-size=72B cardinality=1439473 | 08:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: D1.customer_key = customer_key | fk/pk conjuncts: none | runtime filters: RF006[bloom] <- customer_key | mem-estimate=36.24MB mem-reservation=34.00MB spill-buffer=2.00MB | tuple-ids=9,4 row-size=84B cardinality=1439473 | |--23:EXCHANGE [BROADCAST] | | mem-estimate=0B mem-reservation=0B | | tuple-ids=4 row-size=24B cardinality=1439473 | | | F01:PLAN FRAGMENT [RANDOM] hosts=6 instances=6 | Per-Host Resources: mem-estimate=49.94MB mem-reservation=1.94MB | 06:HASH JOIN [LEFT SEMI JOIN, BROADCAST] | | hash predicates: date_key = max(date_key) | | runtime filters: RF008[bloom] <- max(date_key) | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB | | tuple-ids=4 row-size=24B cardinality=1439473 | | | |--22:EXCHANGE [BROADCAST] | | | mem-estimate=0B mem-reservation=0B | | | tuple-ids=6 row-size=4B cardinality=1 | | | | | F03:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | | Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B | | 21:AGGREGATE [FINALIZE] | | | output: max:merge(date_key) | | | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB | | | tuple-ids=6 row-size=4B cardinality=1 | | | | | 20:EXCHANGE [UNPARTITIONED] | | | mem-estimate=0B mem-reservation=0B | | | tuple-ids=6 row-size=4B cardinality=1 | | | | | F02:PLAN FRAGMENT [RANDOM] hosts=6 instances=6 | | Per-Host Resources: mem-estimate=11.00MB mem-reservation=0B | | 05:AGGREGATE | | | output: max(date_key) | | | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB | | | tuple-ids=6 row-size=4B cardinality=1 | | | | | 04:SCAN HDFS [db1.table6, RANDOM] | | partitions=1955/1955 files=61513 size=101.44GB | | stats-rows=8153177297 extrapolated-rows=disabled | | table stats: rows=8153177297 size=unavailable | | column stats: all | | mem-estimate=1.00MB mem-reservation=0B | | tuple-ids=5 row-size=4B cardinality=8153177297 | | | 03:SCAN HDFS [db1.table6, RANDOM] | partitions=1955/1955 files=61513 size=101.44GB | predicates: db1.table6.closet_state_key = 1 | runtime filters: RF002[bloom] -> db1.table6.customer_key, RF003[bloom] -> db1.table6.shipping_date_key, RF008[bloom] -> date_key | stats-rows=8153177297 extrapolated-rows=disabled | table stats: rows=8153177297 size=unavailable | column stats: all | parquet statistics predicates: db1.table6.closet_state_key = 1 | parquet dictionary predicates: db1.table6.closet_state_key = 1 | mem-estimate=48.00MB mem-reservation=0B | tuple-ids=4 row-size=24B cardinality=2038294324 | 07:SCAN HDFS [db1.table4 d1, RANDOM] partitions=7/7 files=107 size=2.98GB runtime filters: RF000[bloom] -> d1.customer_uuid, RF002[bloom] -> d1.customer_key, RF006[bloom] -> D1.customer_key stats-rows=13817760 extrapolated-rows=disabled table stats: rows=13817760 size=2.98GB column stats: all mem-estimate=80.00MB mem-reservation=0B tuple-ids=9 row-size=60B cardinality=13817760 ---------------- Estimated Per-Host Mem: 275537778009 Per Host Min Reservation: ip-10-1-16-98.ec2.internal:22000(237.75 MB) ip-10-1-18-116.ec2.internal:22000(237.75 MB) ip-10-1-20-86.ec2.internal:22000(237.75 MB) ip-10-1-21-97.ec2.internal:22000(237.75 MB) ip-10-1-27-82.ec2.internal:22000(237.75 MB) ip-10-1-28-130.ec2.internal:22000(237.75 MB) Request Pool: root.applications Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 37:EXCHANGE 1 45s157ms 45s157ms 3 1.06B 0 0 UNPARTITIONED 36:AGGREGATE 6 999.997us 1.000ms 3 1.06B 34.08 MB 127.90 GB FINALIZE 35:EXCHANGE 6 44s756ms 45s028ms 3 1.06B 0 0 HASH(input.customer_key,input.shipping_date_key,input.at_home_count,input.return_count,input.customer_uuid,emailTable.customer_email_address) 19:AGGREGATE 6 333.327us 999.993us 3 1.06B 34.10 MB 127.90 GB STREAMING 18:HASH JOIN 6 833.333us 3.000ms 3 1.06B 34.28 MB 68.89 MB INNER JOIN, BROADCAST |--34:EXCHANGE 6 892.670ms 1s414ms 11.41K 460.59K 0 0 BROADCAST | 02:HASH JOIN 6 48.666ms 227.998ms 11.41K 460.59K 10.80 MB 1.94 MB INNER JOIN, BROADCAST | |--33:EXCHANGE 6 542.168ms 811.002ms 4 3 0 0 BROADCAST | | 01:SCAN HDFS 1 75.997ms 75.997ms 4 3 165.02 KB 32.00 MB db1.table1 dt | 00:SCAN HDFS 6 313.664ms 680.980ms 11.41K 1.38M 69.30 MB 160.00 MB db1.table4 17:HASH JOIN 6 5.166ms 6.999ms 3 7.38B 8.73 MB 8.50 MB INNER JOIN, BROADCAST |--32:EXCHANGE 6 43s865ms 44s400ms 2.21K 399.17K 0 0 BROADCAST | 31:AGGREGATE 6 2.666ms 6.000ms 2.21K 399.17K 8.55 MB 10.00 MB FINALIZE | 30:EXCHANGE 6 44s846ms 45s034ms 2.89K 399.17K 0 0 HASH(customer_key,shipping_date_key) | 16:AGGREGATE 6 0.000ns 0.000ns 2.89K 399.17K 10.23 MB 10.00 MB STREAMING | 15:HASH JOIN 6 166.661us 999.970us 3.30K 399.17K 2.87 MB 1.94 MB INNER JOIN, BROADCAST | |--29:EXCHANGE 6 192.834ms 444.001ms 1 1.00K 0 0 BROADCAST | | 14:SCAN HDFS 1 63.999ms 63.999ms 1 1.00K 257.21 KB 32.00 MB db1.table5 dates | 13:HASH JOIN 6 1.666ms 3.000ms 3.30K 831.08K 2.02 MB 1.94 MB LEFT SEMI JOIN, BROADCAST | |--28:EXCHANGE 6 43s794ms 44s229ms 1 1 0 0 BROADCAST | | 27:AGGREGATE 1 0.000ns 0.000ns 1 1 40.00 KB 10.00 MB FINALIZE | | 26:EXCHANGE 1 44s779ms 44s779ms 6 1 0 0 UNPARTITIONED | | 12:AGGREGATE 6 5s886ms 6s310ms 6 1 20.00 KB 10.00 MB | | 11:SCAN HDFS 6 12s048ms 37s978ms 8.15B 8.15B 1.85 MB 1.00 MB db1.table6 | 10:SCAN HDFS 6 451.826ms 1s567ms 3.30K 1.18B 9.63 MB 80.00 MB db1.table6 25:AGGREGATE 6 2.666ms 4.000ms 642 1.44M 34.07 MB 108.72 MB FINALIZE 24:EXCHANGE 6 36s000ms 43s547ms 642 1.44M 0 0 HASH(customer_uuid,E1.customer_key,shipping_date_key) 09:AGGREGATE 6 10.666ms 20.000ms 642 1.44M 50.01 MB 108.72 MB STREAMING 08:HASH JOIN 6 35.666ms 46.998ms 1.72K 1.44M 48.04 MB 36.24 MB INNER JOIN, BROADCAST |--23:EXCHANGE 6 42s986ms 43s394ms 140.81K 1.44M 0 0 BROADCAST | 06:HASH JOIN 6 2.166ms 3.999ms 140.81K 1.44M 2.69 MB 1.94 MB LEFT SEMI JOIN, BROADCAST | |--22:EXCHANGE 6 42s699ms 43s069ms 1 1 0 0 BROADCAST | | 21:AGGREGATE 1 0.000ns 0.000ns 1 1 40.00 KB 10.00 MB FINALIZE | | 20:EXCHANGE 1 43s370ms 43s370ms 6 1 0 0 UNPARTITIONED | | 05:AGGREGATE 6 5s774ms 6s003ms 6 1 20.00 KB 10.00 MB | | 04:SCAN HDFS 6 12s171ms 36s951ms 8.15B 8.15B 1.79 MB 1.00 MB db1.table6 | 03:SCAN HDFS 6 422.993ms 1s401ms 140.81K 2.04B 11.26 MB 48.00 MB db1.table6 07:SCAN HDFS 6 217.665ms 460.986ms 500 13.82M 68.08 MB 80.00 MB db1.table4 d1 Errors: - ComputeScanRangeAssignmentTimer: 358ms (358996914) - InactiveTotalTime: 0ns (0) - TotalTime: 0ns (0) Planner Timeline Analysis finished: 15ms (15425261) Value transfer graph computed: 15ms (15582003) Single node plan created: 415ms (415094761) Runtime filters computed: 415ms (415922079) Distributed plan created: 416ms (416028433) Lineage info computed: 416ms (416170465) Planning finished: 424ms (424028516) Query Timeline Query submitted: 0ns (0) Planning finished: 1.02s (1022991208) Submit for admission: 1.57s (1566986533) Completed admission: 1.57s (1566986533) Ready to start on 6 backends: 1.59s (1587986352) All 6 execution backends (59 fragment instances) started: 1.85s (1850984092) First dynamic filter received: 3.86s (3863966791) Rows available: 46.85s (46853597312) First row fetched: 47.07s (47074595413) Unregister query: 47.09s (47089595284) ImpalaServer - ClientFetchWaitTimer: 226ms (226998049) - InactiveTotalTime: 0ns (0) - RowMaterializationTimer: 8ms (8999923) - TotalTime: 0ns (0) Execution Profile 68440bc099e1b843:8baf451500000000 Number of filters: 8 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ---------------------------------------------------------------------------------------------------------------------- 14 2 0 LOCAL false 0 (6) N/A N/A true 12 13 10 LOCAL true 0 (6) N/A N/A true 10 15 10 LOCAL false 0 (6) N/A N/A true 8 6 3 LOCAL true 0 (6) N/A N/A true 6 8 7 LOCAL false 0 (6) N/A N/A true 3 17 3 REMOTE false 1 (3) N/A N/A true 2 17 7, 3 REMOTE, REMOTE false, false 1 (3) N/A N/A true 0 18 7 REMOTE false 1 (3) N/A N/A true Backend startup latencies: Count: 6, min / max: 41ms / 222ms, 25th %-ile: 49ms, 50th %-ile: 51ms, 75th %-ile: 89ms, 90th %-ile: 89ms, 95th %-ile: 222ms, 99.9th %-ile: 222ms Final filter table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ---------------------------------------------------------------------------------------------------------------------- 14 2 0 LOCAL false 0 (6) N/A N/A true 12 13 10 LOCAL true 0 (6) N/A N/A true 10 15 10 LOCAL false 0 (6) N/A N/A true 8 6 3 LOCAL true 0 (6) N/A N/A true 6 8 7 LOCAL false 0 (6) N/A N/A true 3 17 3 REMOTE false 0 (3) 46s835ms 46s835ms false 2 17 7, 3 REMOTE, REMOTE false, false 0 (3) 46s834ms 46s834ms false 0 18 7 REMOTE false 0 (3) 3s863ms 3s863ms false Per Node Peak Memory Usage: ip-10-1-16-98.ec2.internal:22000(310.26 MB) ip-10-1-18-116.ec2.internal:22000(314.61 MB) ip-10-1-20-86.ec2.internal:22000(312.01 MB) ip-10-1-21-97.ec2.internal:22000(314.40 MB) ip-10-1-27-82.ec2.internal:22000(303.36 MB) ip-10-1-28-130.ec2.internal:22000(290.68 MB) - FiltersReceived: 3 (3) - FinalizationTimer: 0ns (0) - InactiveTotalTime: 0ns (0) - NumBackends: 6 (6) - NumFragmentInstances: 59 (59) - NumFragments: 14 (14) - TotalTime: 45.22s (45224611311) Averaged Fragment F13 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:45s426ms max:45s426ms mean: 45s426ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 (0.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 20.0 KiB (20496) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 3 (3) - TotalNetworkReceiveTime: 45.16s (45157611888) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 45.37s (45372610040) - TotalThreadsSysTime: 0ns (0) - TotalThreadsUserTime: 197.00us (197000) - TotalThreadsVoluntaryContextSwitches: 5 (5) - TotalTime: 45.42s (45419609636) Fragment Instance Lifecycle Timings - ExecTime: 220ms (220998101) - ExecTreeExecTime: 5ms (5999949) - InactiveTotalTime: 0ns (0) - OpenTime: 45.15s (45151611939) - ExecTreeOpenTime: 45.15s (45151611939) - PrepareTime: 46ms (46999596) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) PLAN_ROOT_SINK - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - TotalTime: 0ns (0) CodeGen - CodegenTime: 0ns (0) - CompileTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - PrepareTime: 40ms (40999647) - TotalTime: 40ms (40999647) EXCHANGE_NODE (id=37) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 3 (3) - RowsReturnedRate: 0 per second (0) - TotalTime: 45.16s (45157611888) DataStreamReceiver - BytesReceived: 332 B (332) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 45.15s (45151611939) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 4.0 KiB (4112) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Coordinator Fragment F13 Instance 68440bc099e1b843:8baf451500000000 (host=ip-10-1-28-130.ec2.internal:22000) - AverageThreadTokens: 0.00 (0.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 20.0 KiB (20496) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 3 (3) - TotalNetworkReceiveTime: 45.16s (45157611888) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 45.37s (45372610040) - TotalThreadsSysTime: 0ns (0) - TotalThreadsUserTime: 197.00us (197000) - TotalThreadsVoluntaryContextSwitches: 5 (5) - TotalTime: 45.42s (45419609636) Fragment Instance Lifecycle Timings - ExecTime: 220ms (220998101) - ExecTreeExecTime: 5ms (5999949) - InactiveTotalTime: 0ns (0) - OpenTime: 45.15s (45151611939) - ExecTreeOpenTime: 45.15s (45151611939) - PrepareTime: 46ms (46999596) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) PLAN_ROOT_SINK - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - TotalTime: 0ns (0) CodeGen - CodegenTime: 0ns (0) - CompileTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - PrepareTime: 40ms (40999647) - TotalTime: 40ms (40999647) EXCHANGE_NODE (id=37) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 3 (3) - RowsReturnedRate: 0 per second (0) - TotalTime: 45.16s (45157611888) DataStreamReceiver - BytesReceived: 332 B (332) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 45.15s (45151611939) - InactiveTotalTime: 45.16s (45157611888) - PeakMemoryUsage: 4.0 KiB (4112) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Averaged Fragment F12 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:45s021ms max:45s210ms mean: 45s159ms stddev:65.011ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 6 - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.1 MiB (35751050) - PeakReservation: 34.0 MiB (35651584) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 307.6 MiB (322494863) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 44.76s (44756560791) - TotalNetworkSendTime: 666.67us (666672) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 45.09s (45089056924) - TotalThreadsSysTime: 6ms (6136333) - TotalThreadsUserTime: 155ms (155005833) - TotalThreadsVoluntaryContextSwitches: 17 (17) - TotalTime: 45.15s (45151389101) Fragment Instance Lifecycle Timings - ExecTime: 666.67us (666672) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 45.09s (45088390252) - ExecTreeOpenTime: 44.76s (44757560789) - PrepareTime: 62ms (62332177) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=37) - BytesSent: 55 B (55) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 23.7 KiB/s (24221) - PeakMemoryUsage: 2.1 KiB (2144) - RowsReturned: 0 (0) - SerializeBatchTime: 0ns (0) - TotalTime: 10ms (10499726) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 63 B (63) CodeGen - CodegenTime: 3ms (3166658) - CompileTime: 120ms (120998943) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 47 (47) - NumInstructions: 1,038 (1038) - OptimizationTime: 184ms (184997851) - PeakMemoryUsage: 519.0 KiB (531456) - PrepareTime: 67ms (67831967) - TotalTime: 374ms (374662086) AGGREGATION_NODE (id=36) - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 512 (512) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 50 (50) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35737728) - RowsRepartitioned: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 44.76s (44757560789) EXCHANGE_NODE (id=35) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.76s (44756560791) DataStreamReceiver - BytesReceived: 55 B (55) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 44.75s (44753227455) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 2.0 KiB (2052) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 3.0 MiB (3145728) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 3.0 MiB (3145728) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment F12 Instance 68440bc099e1b843:8baf45150000003a (host=ip-10-1-28-130.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.1 MiB (35748998) - PeakReservation: 34.0 MiB (35651584) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 45.03s (45028612997) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 45.17s (45170611777) - TotalThreadsSysTime: 3ms (3942000) - TotalThreadsUserTime: 134ms (134415000) - TotalThreadsVoluntaryContextSwitches: 19 (19) - TotalTime: 45.20s (45202611501) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 45.17s (45170611777) - ExecTreeOpenTime: 45.03s (45029612988) - PrepareTime: 31ms (31999724) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=37) - BytesSent: 0 B (0) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 2.1 KiB (2144) - RowsReturned: 0 (0) - SerializeBatchTime: 0ns (0) - TotalTime: 999.99us (999991) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 0 B (0) CodeGen - CodegenTime: 1ms (1999983) - CompileTime: 35ms (35999691) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 47 (47) - NumInstructions: 1,038 (1038) - OptimizationTime: 103ms (103999106) - PeakMemoryUsage: 519.0 KiB (531456) - PrepareTime: 29ms (29999743) - TotalTime: 170ms (170998531) AGGREGATION_NODE (id=36) ExecOption: Codegen Enabled - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35735680) - RowsRepartitioned: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 45.03s (45029612988) EXCHANGE_NODE (id=35) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 45.03s (45028612997) DataStreamReceiver - BytesReceived: 0 B (0) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 45.03s (45028612997) - InactiveTotalTime: 45.03s (45028612997) - PeakMemoryUsage: 0 B (0) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 2.0 MiB (2097152) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 2.0 MiB (2097152) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000037 (host=ip-10-1-20-86.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.1 MiB (35753102) - PeakReservation: 34.0 MiB (35651584) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 312.0 MiB (327162809) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 44.55s (44553680764) - TotalNetworkSendTime: 1ms (1999986) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 19 (19) - TotalThreadsTotalWallClockTime: 45.17s (45172676329) - TotalThreadsSysTime: 1ms (1990000) - TotalThreadsUserTime: 181ms (181626000) - TotalThreadsVoluntaryContextSwitches: 7 (7) - TotalTime: 45.20s (45198676142) Fragment Instance Lifecycle Timings - ExecTime: 1ms (1999986) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 45.17s (45170676343) - ExecTreeOpenTime: 44.55s (44554680756) - PrepareTime: 25ms (25999813) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=37) - BytesSent: 112 B (112) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 2.1 KiB (2144) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 0ns (0) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 129 B (129) CodeGen - CodegenTime: 3ms (3999971) - CompileTime: 392ms (392997184) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 47 (47) - NumInstructions: 1,038 (1038) - OptimizationTime: 219ms (219998424) - PeakMemoryUsage: 519.0 KiB (531456) - PrepareTime: 24ms (24999821) - TotalTime: 638ms (638995422) AGGREGATION_NODE (id=36) ExecOption: Codegen Enabled - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 1,024 (1024) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35739776) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 44.55s (44554680756) EXCHANGE_NODE (id=35) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.55s (44553680764) DataStreamReceiver - BytesReceived: 112 B (112) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 44.54s (44538680871) - InactiveTotalTime: 44.55s (44553680764) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 4.0 MiB (4194304) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 4.0 MiB (4194304) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000039 (host=ip-10-1-16-98.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.1 MiB (35748998) - PeakReservation: 34.0 MiB (35651584) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 310.3 MiB (325335426) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 45.02s (45023153856) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 2 (2) - TotalThreadsTotalWallClockTime: 45.18s (45176154378) - TotalThreadsSysTime: 2ms (2866000) - TotalThreadsUserTime: 147ms (147959000) - TotalThreadsVoluntaryContextSwitches: 7 (7) - TotalTime: 45.20s (45195154443) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 45.18s (45176154378) - ExecTreeOpenTime: 45.02s (45024153859) - PrepareTime: 19ms (19000065) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=37) - BytesSent: 0 B (0) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 2.1 KiB (2144) - RowsReturned: 0 (0) - SerializeBatchTime: 0ns (0) - TotalTime: 1ms (1000003) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 0 B (0) CodeGen - CodegenTime: 2ms (2000007) - CompileTime: 38ms (38000130) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 47 (47) - NumInstructions: 1,038 (1038) - OptimizationTime: 112ms (112000382) - PeakMemoryUsage: 519.0 KiB (531456) - PrepareTime: 19ms (19000065) - TotalTime: 169ms (169000577) AGGREGATION_NODE (id=36) ExecOption: Codegen Enabled - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35735680) - RowsRepartitioned: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 45.02s (45024153859) EXCHANGE_NODE (id=35) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 45.02s (45023153856) DataStreamReceiver - BytesReceived: 0 B (0) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 45.02s (45023153856) - InactiveTotalTime: 45.02s (45023153856) - PeakMemoryUsage: 0 B (0) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 2.0 MiB (2097152) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 2.0 MiB (2097152) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000038 (host=ip-10-1-21-97.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.1 MiB (35753102) - PeakReservation: 34.0 MiB (35651584) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 314.4 MiB (329677036) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 44.99s (44987101637) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 45.12s (45121101940) - TotalThreadsSysTime: 5ms (5211000) - TotalThreadsUserTime: 125ms (125012000) - TotalThreadsVoluntaryContextSwitches: 13 (13) - TotalTime: 45.15s (45152102010) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 45.12s (45121101940) - ExecTreeOpenTime: 44.99s (44988101640) - PrepareTime: 31ms (31000070) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=37) - BytesSent: 112 B (112) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 36.5 KiB/s (37333) - PeakMemoryUsage: 2.1 KiB (2144) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 3ms (3000007) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 129 B (129) CodeGen - CodegenTime: 3ms (3000007) - CompileTime: 32ms (32000072) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 47 (47) - NumInstructions: 1,038 (1038) - OptimizationTime: 99ms (99000223) - PeakMemoryUsage: 519.0 KiB (531456) - PrepareTime: 27ms (27000061) - TotalTime: 158ms (158000356) AGGREGATION_NODE (id=36) ExecOption: Codegen Enabled - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 1,024 (1024) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35739776) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 44.99s (44988101640) EXCHANGE_NODE (id=35) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.99s (44987101637) DataStreamReceiver - BytesReceived: 112 B (112) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 44.99s (44987101637) - InactiveTotalTime: 44.99s (44987101637) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 4.0 MiB (4194304) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 4.0 MiB (4194304) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000035 (host=ip-10-1-18-116.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.1 MiB (35753102) - PeakReservation: 34.0 MiB (35651584) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 314.6 MiB (329895427) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 44.95s (44953104542) - TotalNetworkSendTime: 2ms (2000049) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 45.13s (45127108818) - TotalThreadsSysTime: 20ms (20960000) - TotalThreadsUserTime: 148ms (148366000) - TotalThreadsVoluntaryContextSwitches: 4 (4) - TotalTime: 45.15s (45149109358) Fragment Instance Lifecycle Timings - ExecTime: 2ms (2000049) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 45.12s (45125108769) - ExecTreeOpenTime: 44.95s (44954104567) - PrepareTime: 22ms (22000540) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=37) - BytesSent: 108 B (108) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 105.5 KiB/s (107997) - PeakMemoryUsage: 2.1 KiB (2144) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 1ms (1000024) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 125 B (125) CodeGen - CodegenTime: 4ms (4000098) - CompileTime: 60ms (60001474) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 47 (47) - NumInstructions: 1,038 (1038) - OptimizationTime: 108ms (108002654) - PeakMemoryUsage: 519.0 KiB (531456) - PrepareTime: 20ms (20000492) - TotalTime: 189ms (189004645) AGGREGATION_NODE (id=36) ExecOption: Codegen Enabled - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 1,024 (1024) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35739776) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 44.95s (44954104567) EXCHANGE_NODE (id=35) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.95s (44953104542) DataStreamReceiver - BytesReceived: 108 B (108) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 44.95s (44948104419) - InactiveTotalTime: 44.95s (44953104542) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 4.0 MiB (4194304) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 4.0 MiB (4194304) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000036 (host=ip-10-1-27-82.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.1 MiB (35748998) - PeakReservation: 34.0 MiB (35651584) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 303.4 MiB (318098845) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 43.99s (43993710954) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 17 (17) - TotalThreadsTotalWallClockTime: 44.77s (44766688306) - TotalThreadsSysTime: 1ms (1849000) - TotalThreadsUserTime: 192ms (192657000) - TotalThreadsVoluntaryContextSwitches: 53 (53) - TotalTime: 45.01s (45010681157) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 44.77s (44766688306) - ExecTreeOpenTime: 43.99s (43994710925) - PrepareTime: 243ms (243992851) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=37) - BytesSent: 0 B (0) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 2.1 KiB (2144) - RowsReturned: 0 (0) - SerializeBatchTime: 0ns (0) - TotalTime: 56ms (56998331) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 0 B (0) CodeGen - CodegenTime: 3ms (3999883) - CompileTime: 166ms (166995107) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 47 (47) - NumInstructions: 1,038 (1038) - OptimizationTime: 466ms (466986317) - PeakMemoryUsage: 519.0 KiB (531456) - PrepareTime: 285ms (285991620) - TotalTime: 921ms (921972986) AGGREGATION_NODE (id=36) ExecOption: Codegen Enabled - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35735680) - RowsRepartitioned: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 43.99s (43994710925) EXCHANGE_NODE (id=35) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.99s (43993710954) DataStreamReceiver - BytesReceived: 0 B (0) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 43.99s (43993710954) - InactiveTotalTime: 43.99s (43993710954) - PeakMemoryUsage: 0 B (0) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 2.0 MiB (2097152) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 2.0 MiB (2097152) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Averaged Fragment F04 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:45s024ms max:45s193ms mean: 45s147ms stddev:58.005ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 6 - AverageThreadTokens: 1.83 (1.8315018315018314) - BloomFilterBytes: 1.5 MiB (1572864) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 112.3 MiB (117775928) - PeakReservation: 110.5 MiB (115867648) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 307.6 MiB (322494863) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 1.3m (80758673958) - TotalNetworkSendTime: 500.00us (499996) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 55 (55) - TotalThreadsTotalWallClockTime: 1.4m (82605337028) - TotalThreadsSysTime: 16ms (16172666) - TotalThreadsUserTime: 967ms (967899666) - TotalThreadsVoluntaryContextSwitches: 169 (169) - TotalTime: 45.13s (45130389377) Fragment Instance Lifecycle Timings - ExecTime: 4ms (4166628) - ExecTreeExecTime: 666.66us (666661) - InactiveTotalTime: 0ns (0) - OpenTime: 44.99s (44989058921) - ExecTreeOpenTime: 43.87s (43870897927) - PrepareTime: 137ms (137163826) - ExecTreePrepareTime: 1ms (1166671) - TotalTime: 0ns (0) DataStreamSender (dst_id=35) - BytesSent: 55 B (55) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 8.6 KiB/s (8771) - PeakMemoryUsage: 14.6 KiB (14912) - RowsReturned: 0 (0) - SerializeBatchTime: 0ns (0) - TotalTime: 33ms (33332460) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 63 B (63) CodeGen - CodegenTime: 6ms (6499966) - CompileTime: 334ms (334998830) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 143 (143) - NumInstructions: 6,565 (6565) - OptimizationTime: 779ms (779828852) - PeakMemoryUsage: 3.2 MiB (3361536) - PrepareTime: 59ms (59498949) - TotalTime: 1.17s (1174826630) AGGREGATION_NODE (id=19) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 512 (512) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 50 (50) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35760256) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - StreamingTime: 0ns (0) - TotalTime: 43.87s (43872731260) HASH_JOIN_NODE (id=18) - BuildRows: 11,410 (11410) - BuildTime: 7ms (7833352) - InactiveTotalTime: 0ns (0) - LocalTime: 833.33us (833333) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 34.3 MiB (35941698) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.87s (43873064599) Buffer pool - AllocTime: 1ms (1666677) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=18) - BuildRowsPartitionTime: 5ms (5833352) - BuildRowsPartitioned: 11,410 (11410) - HashBuckets: 29,696 (29696) - HashCollisions: 0 (0) - HashTablesBuildTime: 1ms (1166668) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 37.1 KiB (38016) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=34) - ConvertRowBatchTime: 333.33us (333334) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 11,410 (11410) - RowsReturnedRate: 9637 per second (9637) - TotalTime: 892ms (892670477) DataStreamReceiver - BytesReceived: 834.8 KiB (854827) - DeserializeRowBatchTimer: 6ms (6666566) - FirstBatchArrivalWaitTime: 66ms (66833861) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 531.1 KiB (543885) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=17) - BuildRows: 2,214 (2214) - BuildTime: 2ms (2499976) - InactiveTotalTime: 0ns (0) - LocalTime: 5ms (5166638) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 8.7 MiB (9154816) - ProbeRows: 107 (107) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 166.66us (166661) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.87s (43872231266) Hash Join Builder (join_node_id=17) - BuildRowsPartitionTime: 333.34us (333336) - BuildRowsPartitioned: 2,214 (2214) - HashBuckets: 4,096 (4096) - HashCollisions: 0 (0) - HashTablesBuildTime: 166.66us (166665) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 7 (7) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 30.1 KiB (30848) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=32) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 2,214 (2214) - RowsReturnedRate: 50 per second (50) - TotalTime: 43.86s (43865731305) DataStreamReceiver - BytesReceived: 33.3 KiB (34101) - DeserializeRowBatchTimer: 1ms (1666666) - FirstBatchArrivalWaitTime: 43.81s (43807898086) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.9 KiB (35754) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) AGGREGATION_NODE (id=25) - BuildTime: 666.66us (666665) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 10 (10) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35723218) - RowsRepartitioned: 0 (0) - RowsReturned: 107 (107) - RowsReturnedRate: 9502 per second (9502) - SpilledPartitions: 0 (0) - TotalTime: 36.00s (36003272171) EXCHANGE_NODE (id=24) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 107 (107) - RowsReturnedRate: 1 per second (1) - TotalTime: 36.00s (36000605510) DataStreamReceiver - BytesReceived: 5.7 KiB (5828) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 35.71s (35711271347) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 7.6 KiB (7785) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 166.66us (166661) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.5 MiB (8912896) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 166.67us (166670) - CumulativeAllocationBytes: 3.0 MiB (3145728) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 3.0 MiB (3145728) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment F04 Instance 68440bc099e1b843:8baf451500000033 (host=ip-10-1-16-98.ec2.internal:22000) - AverageThreadTokens: 2.00 (2.0) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 111.8 MiB (117251335) - PeakReservation: 110.5 MiB (115867648) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 310.3 MiB (325335426) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1.5m (89360305368) - TotalNetworkSendTime: 2ms (2000006) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 2 (2) - TotalThreadsTotalWallClockTime: 1.5m (90971310872) - TotalThreadsSysTime: 14ms (14483000) - TotalThreadsUserTime: 738ms (738133000) - TotalThreadsVoluntaryContextSwitches: 166 (166) - TotalTime: 45.19s (45188154419) Fragment Instance Lifecycle Timings - ExecTime: 5ms (5000017) - ExecTreeExecTime: 1ms (1000004) - InactiveTotalTime: 0ns (0) - OpenTime: 45.14s (45145154272) - ExecTreeOpenTime: 44.40s (44403151737) - PrepareTime: 38ms (38000130) - ExecTreePrepareTime: 2ms (2000007) - TotalTime: 0ns (0) DataStreamSender (dst_id=35) - BytesSent: 108 B (108) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 35.2 KiB/s (35999) - PeakMemoryUsage: 16.6 KiB (16960) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 3ms (3000010) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 125 B (125) CodeGen - CodegenTime: 4ms (4000014) - CompileTime: 244ms (244000834) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 143 (143) - NumInstructions: 6,552 (6552) - OptimizationTime: 495ms (495001691) - PeakMemoryUsage: 3.2 MiB (3354624) - PrepareTime: 24ms (24000082) - TotalTime: 763ms (763002607) AGGREGATION_NODE (id=19) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 1,024 (1024) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35760256) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - StreamingTime: 0ns (0) - TotalTime: 44.41s (44406151748) HASH_JOIN_NODE (id=18) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 11,410 (11410) - BuildTime: 5ms (5000016) - InactiveTotalTime: 0ns (0) - LocalTime: 0ns (0) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 34.3 MiB (35941698) - ProbeRows: 1 (1) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.41s (44406151747) Buffer pool - AllocTime: 1ms (1000003) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=18) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 4ms (4000013) - BuildRowsPartitioned: 11,410 (11410) - HashBuckets: 29,696 (29696) - HashCollisions: 0 (0) - HashTablesBuildTime: 1ms (1000003) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 37.1 KiB (38016) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=34) - ConvertRowBatchTime: 1ms (1000004) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 11,410 (11410) - RowsReturnedRate: 8069 per second (8069) - TotalTime: 1.41s (1414004833) DataStreamReceiver - BytesReceived: 834.8 KiB (854827) - DeserializeRowBatchTimer: 1ms (1000003) - FirstBatchArrivalWaitTime: 316ms (316001080) - InactiveTotalTime: 1.41s (1413004829) - PeakMemoryUsage: 142.6 KiB (146062) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=17) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 2,214 (2214) - BuildTime: 1ms (1000004) - InactiveTotalTime: 0ns (0) - LocalTime: 4ms (4000013) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 8.7 MiB (9154816) - ProbeRows: 89 (89) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.41s (44406151747) Hash Join Builder (join_node_id=17) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 2,214 (2214) - HashBuckets: 4,096 (4096) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 7 (7) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 30.1 KiB (30848) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=32) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 2,214 (2214) - RowsReturnedRate: 49 per second (49) - TotalTime: 44.40s (44400151727) DataStreamReceiver - BytesReceived: 33.3 KiB (34101) - DeserializeRowBatchTimer: 1ms (1000004) - FirstBatchArrivalWaitTime: 44.34s (44336151508) - InactiveTotalTime: 44.40s (44400151727) - PeakMemoryUsage: 12.6 KiB (12920) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) AGGREGATION_NODE (id=25) ExecOption: Codegen Enabled - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 10 (10) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35724416) - RowsRepartitioned: 0 (0) - RowsReturned: 89 (89) - RowsReturnedRate: 2 per second (2) - SpilledPartitions: 0 (0) - TotalTime: 43.55s (43550148822) EXCHANGE_NODE (id=24) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 89 (89) - RowsReturnedRate: 2 per second (2) - TotalTime: 43.55s (43547148812) DataStreamReceiver - BytesReceived: 4.8 KiB (4917) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 43.20s (43197147616) - InactiveTotalTime: 43.55s (43547148812) - PeakMemoryUsage: 4.6 KiB (4728) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.5 MiB (8912896) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 4.0 MiB (4194304) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 4.0 MiB (4194304) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000034 (host=ip-10-1-28-130.ec2.internal:22000) - AverageThreadTokens: 2.00 (2.0) - BloomFilterBytes: 3.0 MiB (3145728) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 113.8 MiB (119349279) - PeakReservation: 110.5 MiB (115867648) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 1.5m (88302241076) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 85 (85) - TotalThreadsTotalWallClockTime: 1.5m (90216224627) - TotalThreadsSysTime: 25ms (25057000) - TotalThreadsUserTime: 1.02s (1019724000) - TotalThreadsVoluntaryContextSwitches: 180 (180) - TotalTime: 45.18s (45182611673) Fragment Instance Lifecycle Timings - ExecTime: 1ms (1999983) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 45.10s (45096612412) - ExecTreeOpenTime: 44.04s (44045621445) - PrepareTime: 83ms (83999278) - ExecTreePrepareTime: 999.99us (999991) - TotalTime: 0ns (0) DataStreamSender (dst_id=35) - BytesSent: 0 B (0) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 12.6 KiB (12864) - RowsReturned: 0 (0) - SerializeBatchTime: 0ns (0) - TotalTime: 10ms (10999905) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 0 B (0) CodeGen - CodegenTime: 11ms (11999897) - CompileTime: 340ms (340997069) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 145 (145) - NumInstructions: 6,601 (6601) - OptimizationTime: 706ms (706993924) - PeakMemoryUsage: 3.2 MiB (3379712) - PrepareTime: 39ms (39999656) - TotalTime: 1.09s (1088990640) AGGREGATION_NODE (id=19) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35760256) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - StreamingTime: 0ns (0) - TotalTime: 44.05s (44046621436) HASH_JOIN_NODE (id=18) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 11,410 (11410) - BuildTime: 8ms (8999924) - InactiveTotalTime: 0ns (0) - LocalTime: 0ns (0) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 34.3 MiB (35941698) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.05s (44047621427) Buffer pool - AllocTime: 999.99us (999992) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=18) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 5ms (5999950) - BuildRowsPartitioned: 11,410 (11410) - HashBuckets: 29,696 (29696) - HashCollisions: 0 (0) - HashTablesBuildTime: 999.99us (999992) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 37.1 KiB (38016) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=34) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 11,410 (11410) - RowsReturnedRate: 10723 per second (10723) - TotalTime: 1.06s (1063990854) DataStreamReceiver - BytesReceived: 834.8 KiB (854827) - DeserializeRowBatchTimer: 2ms (2999973) - FirstBatchArrivalWaitTime: 0ns (0) - InactiveTotalTime: 1.06s (1062990862) - PeakMemoryUsage: 365.8 KiB (374569) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=17) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 2,214 (2214) - BuildTime: 4ms (4999957) - InactiveTotalTime: 0ns (0) - LocalTime: 5ms (5999948) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 8.7 MiB (9154816) - ProbeRows: 119 (119) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.05s (44047621427) Hash Join Builder (join_node_id=17) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 2 of 2 Runtime Filters Published - BuildRowsPartitionTime: 999.99us (999992) - BuildRowsPartitioned: 2,214 (2214) - HashBuckets: 4,096 (4096) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 7 (7) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 30.1 KiB (30848) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=32) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 2,214 (2214) - RowsReturnedRate: 50 per second (50) - TotalTime: 44.04s (44039621497) DataStreamReceiver - BytesReceived: 33.3 KiB (34101) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 43.99s (43987621944) - InactiveTotalTime: 44.04s (44039621497) - PeakMemoryUsage: 25.1 KiB (25696) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) AGGREGATION_NODE (id=25) ExecOption: Codegen Enabled - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 12 (12) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35724919) - RowsRepartitioned: 0 (0) - RowsReturned: 119 (119) - RowsReturnedRate: 2 per second (2) - SpilledPartitions: 0 (0) - TotalTime: 43.20s (43201628698) EXCHANGE_NODE (id=24) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 119 (119) - RowsReturnedRate: 2 per second (2) - TotalTime: 43.20s (43198628725) DataStreamReceiver - BytesReceived: 6.3 KiB (6404) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 42.85s (42851631707) - InactiveTotalTime: 43.20s (43198628725) - PeakMemoryUsage: 5.3 KiB (5455) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.5 MiB (8912896) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 2.0 MiB (2097152) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 2.0 MiB (2097152) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000031 (host=ip-10-1-20-86.ec2.internal:22000) - AverageThreadTokens: 1.99 (1.989010989010989) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 111.8 MiB (117251775) - PeakReservation: 110.5 MiB (115867648) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 312.0 MiB (327162809) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 1.4m (86901377334) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 98 (98) - TotalThreadsTotalWallClockTime: 1.5m (89325359966) - TotalThreadsSysTime: 17ms (17614000) - TotalThreadsUserTime: 1.09s (1093395000) - TotalThreadsVoluntaryContextSwitches: 125 (125) - TotalTime: 45.17s (45173676321) Fragment Instance Lifecycle Timings - ExecTime: 2ms (2999979) - ExecTreeExecTime: 999.99us (999993) - InactiveTotalTime: 0ns (0) - OpenTime: 45.14s (45140676557) - ExecTreeOpenTime: 43.58s (43576687764) - PrepareTime: 29ms (29999785) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=35) - BytesSent: 0 B (0) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 12.6 KiB (12864) - RowsReturned: 0 (0) - SerializeBatchTime: 0ns (0) - TotalTime: 999.99us (999993) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 0 B (0) CodeGen - CodegenTime: 5ms (5999958) - CompileTime: 393ms (393997176) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 143 (143) - NumInstructions: 6,557 (6557) - OptimizationTime: 1.17s (1166991639) - PeakMemoryUsage: 3.2 MiB (3357184) - PrepareTime: 18ms (18999863) - TotalTime: 1.58s (1579988678) AGGREGATION_NODE (id=19) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35760256) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - StreamingTime: 0ns (0) - TotalTime: 43.58s (43577687757) HASH_JOIN_NODE (id=18) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 11,410 (11410) - BuildTime: 5ms (5999958) - InactiveTotalTime: 0ns (0) - LocalTime: 0ns (0) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 34.3 MiB (35941698) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.58s (43576687764) Buffer pool - AllocTime: 999.99us (999993) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=18) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 2ms (2999979) - BuildRowsPartitioned: 11,410 (11410) - HashBuckets: 29,696 (29696) - HashCollisions: 0 (0) - HashTablesBuildTime: 999.99us (999993) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 37.1 KiB (38016) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=34) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 11,410 (11410) - RowsReturnedRate: 18953 per second (18953) - TotalTime: 601ms (601995686) DataStreamReceiver - BytesReceived: 834.8 KiB (854827) - DeserializeRowBatchTimer: 999.99us (999992) - FirstBatchArrivalWaitTime: 0ns (0) - InactiveTotalTime: 601ms (601995686) - PeakMemoryUsage: 853.6 KiB (874067) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=17) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 2,214 (2214) - BuildTime: 1ms (1999986) - InactiveTotalTime: 0ns (0) - LocalTime: 4ms (4999964) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 8.7 MiB (9154816) - ProbeRows: 98 (98) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.58s (43576687764) Hash Join Builder (join_node_id=17) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 2,214 (2214) - HashBuckets: 4,096 (4096) - HashCollisions: 0 (0) - HashTablesBuildTime: 999.99us (999993) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 7 (7) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 30.1 KiB (30848) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=32) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 2,214 (2214) - RowsReturnedRate: 50 per second (50) - TotalTime: 43.57s (43571687800) DataStreamReceiver - BytesReceived: 33.3 KiB (34101) - DeserializeRowBatchTimer: 2ms (2999979) - FirstBatchArrivalWaitTime: 43.52s (43522688151) - InactiveTotalTime: 43.56s (43561687872) - PeakMemoryUsage: 100.6 KiB (102976) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) AGGREGATION_NODE (id=25) ExecOption: Codegen Enabled - BuildTime: 999.99us (999993) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 9 (9) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35720320) - RowsRepartitioned: 0 (0) - RowsReturned: 98 (98) - RowsReturnedRate: 2 per second (2) - SpilledPartitions: 0 (0) - TotalTime: 42.73s (42729693833) EXCHANGE_NODE (id=24) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 98 (98) - RowsReturnedRate: 2 per second (2) - TotalTime: 42.73s (42727693848) DataStreamReceiver - BytesReceived: 5.4 KiB (5479) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 42.38s (42381696327) - InactiveTotalTime: 42.73s (42727693848) - PeakMemoryUsage: 4.7 KiB (4792) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.5 MiB (8912896) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 2.0 MiB (2097152) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 2.0 MiB (2097152) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf45150000002f (host=ip-10-1-18-116.ec2.internal:22000) - AverageThreadTokens: 2.00 (2.0) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 111.8 MiB (117251775) - PeakReservation: 110.5 MiB (115867648) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 314.6 MiB (329895427) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 1.5m (88655178345) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 48 (48) - TotalThreadsTotalWallClockTime: 1.5m (90468222893) - TotalThreadsSysTime: 14ms (14515000) - TotalThreadsUserTime: 906ms (906805000) - TotalThreadsVoluntaryContextSwitches: 147 (147) - TotalTime: 45.14s (45141109161) Fragment Instance Lifecycle Timings - ExecTime: 2ms (2000049) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 45.10s (45100108154) - ExecTreeOpenTime: 44.17s (44174085401) - PrepareTime: 39ms (39000958) - ExecTreePrepareTime: 1ms (1000024) - TotalTime: 0ns (0) DataStreamSender (dst_id=35) - BytesSent: 0 B (0) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 12.6 KiB (12864) - RowsReturned: 0 (0) - SerializeBatchTime: 0ns (0) - TotalTime: 1ms (1000025) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 0 B (0) CodeGen - CodegenTime: 5ms (5000123) - CompileTime: 311ms (311007641) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 143 (143) - NumInstructions: 6,557 (6557) - OptimizationTime: 612ms (612015038) - PeakMemoryUsage: 3.2 MiB (3357184) - PrepareTime: 26ms (26000640) - TotalTime: 950ms (950023343) AGGREGATION_NODE (id=19) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35760256) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - StreamingTime: 0ns (0) - TotalTime: 44.18s (44175085425) HASH_JOIN_NODE (id=18) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 11,410 (11410) - BuildTime: 14ms (14000344) - InactiveTotalTime: 0ns (0) - LocalTime: 1ms (1000024) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 34.3 MiB (35941698) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.18s (44175085425) Buffer pool - AllocTime: 4ms (4000099) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=18) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 11ms (11000271) - BuildRowsPartitioned: 11,410 (11410) - HashBuckets: 29,696 (29696) - HashCollisions: 0 (0) - HashTablesBuildTime: 2ms (2000049) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 37.1 KiB (38016) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=34) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 11,410 (11410) - RowsReturnedRate: 9661 per second (9661) - TotalTime: 1.18s (1181029019) DataStreamReceiver - BytesReceived: 834.8 KiB (854827) - DeserializeRowBatchTimer: 2ms (2000049) - FirstBatchArrivalWaitTime: 85ms (85002089) - InactiveTotalTime: 1.18s (1181029019) - PeakMemoryUsage: 264.0 KiB (270374) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=17) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 2,214 (2214) - BuildTime: 1ms (1000024) - InactiveTotalTime: 0ns (0) - LocalTime: 4ms (4000098) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 8.7 MiB (9154816) - ProbeRows: 92 (92) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.17s (44174085401) Hash Join Builder (join_node_id=17) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 1ms (1000024) - BuildRowsPartitioned: 2,214 (2214) - HashBuckets: 4,096 (4096) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 7 (7) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 30.1 KiB (30848) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=32) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 2,214 (2214) - RowsReturnedRate: 50 per second (50) - TotalTime: 44.17s (44170085303) DataStreamReceiver - BytesReceived: 33.3 KiB (34101) - DeserializeRowBatchTimer: 3ms (3000073) - FirstBatchArrivalWaitTime: 44.11s (44107083755) - InactiveTotalTime: 44.17s (44170085303) - PeakMemoryUsage: 29.4 KiB (30144) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) AGGREGATION_NODE (id=25) ExecOption: Codegen Enabled - BuildTime: 1ms (1000025) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 9 (9) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35720320) - RowsRepartitioned: 0 (0) - RowsReturned: 92 (92) - RowsReturnedRate: 2 per second (2) - SpilledPartitions: 0 (0) - TotalTime: 43.31s (43306064073) EXCHANGE_NODE (id=24) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 92 (92) - RowsReturnedRate: 2 per second (2) - TotalTime: 43.30s (43304064023) DataStreamReceiver - BytesReceived: 4.8 KiB (4947) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 42.96s (42960055571) - InactiveTotalTime: 43.30s (43304064023) - PeakMemoryUsage: 4.7 KiB (4800) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.5 MiB (8912896) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 1ms (1000024) - CumulativeAllocationBytes: 2.0 MiB (2097152) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 2.0 MiB (2097152) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000032 (host=ip-10-1-21-97.ec2.internal:22000) - AverageThreadTokens: 2.00 (2.0) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 111.8 MiB (117251351) - PeakReservation: 110.5 MiB (115867648) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 314.4 MiB (329677036) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1.5m (88396199703) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 38 (38) - TotalThreadsTotalWallClockTime: 1.5m (90278203960) - TotalThreadsSysTime: 12ms (12617000) - TotalThreadsUserTime: 1.01s (1007515000) - TotalThreadsVoluntaryContextSwitches: 181 (181) - TotalTime: 45.14s (45139101980) Fragment Instance Lifecycle Timings - ExecTime: 4ms (4000009) - ExecTreeExecTime: 1ms (1000002) - InactiveTotalTime: 0ns (0) - OpenTime: 45.09s (45093101876) - ExecTreeOpenTime: 44.08s (44079099588) - PrepareTime: 42ms (42000095) - ExecTreePrepareTime: 3ms (3000007) - TotalTime: 0ns (0) DataStreamSender (dst_id=35) - BytesSent: 112 B (112) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 15.6 KiB/s (15999) - PeakMemoryUsage: 16.6 KiB (16960) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 7ms (7000016) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 129 B (129) CodeGen - CodegenTime: 5ms (5000012) - CompileTime: 360ms (360000812) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 143 (143) - NumInstructions: 6,552 (6552) - OptimizationTime: 652ms (652001471) - PeakMemoryUsage: 3.2 MiB (3354624) - PrepareTime: 23ms (23000051) - TotalTime: 1.04s (1035002334) AGGREGATION_NODE (id=19) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 1,024 (1024) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35760256) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - StreamingTime: 0ns (0) - TotalTime: 44.08s (44083099597) HASH_JOIN_NODE (id=18) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 11,410 (11410) - BuildTime: 8ms (8000018) - InactiveTotalTime: 0ns (0) - LocalTime: 3ms (3000006) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 34.3 MiB (35941698) - ProbeRows: 1 (1) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.09s (44086099604) Buffer pool - AllocTime: 2ms (2000005) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=18) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 7ms (7000016) - BuildRowsPartitioned: 11,410 (11410) - HashBuckets: 29,696 (29696) - HashCollisions: 0 (0) - HashTablesBuildTime: 1ms (1000002) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 37.1 KiB (38016) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=34) - ConvertRowBatchTime: 1ms (1000002) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 11,410 (11410) - RowsReturnedRate: 10420 per second (10420) - TotalTime: 1.10s (1095002471) DataStreamReceiver - BytesReceived: 834.8 KiB (854827) - DeserializeRowBatchTimer: 11ms (11000024) - FirstBatchArrivalWaitTime: 0ns (0) - InactiveTotalTime: 1.09s (1094002469) - PeakMemoryUsage: 142.6 KiB (146062) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=17) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 2,214 (2214) - BuildTime: 2ms (2000004) - InactiveTotalTime: 0ns (0) - LocalTime: 5ms (5000011) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 8.7 MiB (9154816) - ProbeRows: 130 (130) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.08s (44083099598) Hash Join Builder (join_node_id=17) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 2,214 (2214) - HashBuckets: 4,096 (4096) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 7 (7) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 30.1 KiB (30848) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=32) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 2,214 (2214) - RowsReturnedRate: 50 per second (50) - TotalTime: 44.08s (44076099582) DataStreamReceiver - BytesReceived: 33.3 KiB (34101) - DeserializeRowBatchTimer: 1ms (1000002) - FirstBatchArrivalWaitTime: 44.02s (44018099450) - InactiveTotalTime: 44.08s (44075099579) - PeakMemoryUsage: 16.7 KiB (17096) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) AGGREGATION_NODE (id=25) ExecOption: Codegen Enabled - BuildTime: 1ms (1000002) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 11 (11) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35724919) - RowsRepartitioned: 0 (0) - RowsReturned: 130 (130) - RowsReturnedRate: 3 per second (3) - SpilledPartitions: 0 (0) - TotalTime: 43.23s (43230097662) EXCHANGE_NODE (id=24) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 130 (130) - RowsReturnedRate: 3 per second (3) - TotalTime: 43.23s (43226097652) DataStreamReceiver - BytesReceived: 6.8 KiB (6959) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 42.88s (42877096861) - InactiveTotalTime: 43.23s (43226097652) - PeakMemoryUsage: 5.4 KiB (5543) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.5 MiB (8912896) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 4.0 MiB (4194304) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 4.0 MiB (4194304) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000030 (host=ip-10-1-27-82.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 112.8 MiB (118300055) - PeakReservation: 110.5 MiB (115867648) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 303.4 MiB (318098845) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 42.94s (42936741925) - TotalNetworkSendTime: 999.97us (999971) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 63 (63) - TotalThreadsTotalWallClockTime: 44.37s (44372699850) - TotalThreadsSysTime: 12ms (12750000) - TotalThreadsUserTime: 1.04s (1041826000) - TotalThreadsVoluntaryContextSwitches: 218 (218) - TotalTime: 44.96s (44957682709) Fragment Instance Lifecycle Timings - ExecTime: 8ms (8999736) - ExecTreeExecTime: 999.97us (999971) - InactiveTotalTime: 0ns (0) - OpenTime: 44.36s (44358700260) - ExecTreeOpenTime: 42.95s (42946741631) - PrepareTime: 589ms (589982713) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=35) - BytesSent: 112 B (112) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 632 B/s (632) - PeakMemoryUsage: 16.6 KiB (16960) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 176ms (176994814) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 129 B (129) CodeGen - CodegenTime: 6ms (6999795) - CompileTime: 359ms (359989453) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 144 (144) - NumInstructions: 6,574 (6574) - OptimizationTime: 1.04s (1045969352) - PeakMemoryUsage: 3.2 MiB (3365888) - PrepareTime: 224ms (224993406) - TotalTime: 1.63s (1631952182) AGGREGATION_NODE (id=19) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 1,024 (1024) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35760256) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - StreamingTime: 0ns (0) - TotalTime: 42.95s (42947741602) HASH_JOIN_NODE (id=18) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 11,410 (11410) - BuildTime: 4ms (4999854) - InactiveTotalTime: 0ns (0) - LocalTime: 999.97us (999970) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 34.3 MiB (35941698) - ProbeRows: 1 (1) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 42.95s (42946741631) Buffer pool - AllocTime: 999.97us (999971) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=18) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 3ms (3999883) - BuildRowsPartitioned: 11,410 (11410) - HashBuckets: 29,696 (29696) - HashCollisions: 0 (0) - HashTablesBuildTime: 999.97us (999971) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 37.1 KiB (38016) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=34) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 11,410 (11410) - RowsReturnedRate: 0 per second (0) - TotalTime: 0ns (0) DataStreamReceiver - BytesReceived: 834.8 KiB (854827) - DeserializeRowBatchTimer: 21ms (21999356) - FirstBatchArrivalWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.4 MiB (1452178) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=17) ExecOption: Probe Side Codegen Enabled - BuildRows: 2,214 (2214) - BuildTime: 3ms (3999882) - InactiveTotalTime: 0ns (0) - LocalTime: 6ms (6999795) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 8.7 MiB (9154816) - ProbeRows: 114 (114) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 999.97us (999971) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 42.94s (42945741661) Hash Join Builder (join_node_id=17) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 2 of 2 Runtime Filters Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 2,214 (2214) - HashBuckets: 4,096 (4096) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 7 (7) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 30.1 KiB (30848) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=32) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 2,214 (2214) - RowsReturnedRate: 51 per second (51) - TotalTime: 42.94s (42936741925) DataStreamReceiver - BytesReceived: 33.3 KiB (34101) - DeserializeRowBatchTimer: 1ms (1999941) - FirstBatchArrivalWaitTime: 42.88s (42875743712) - InactiveTotalTime: 42.94s (42936741925) - PeakMemoryUsage: 25.1 KiB (25696) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) AGGREGATION_NODE (id=25) ExecOption: Codegen Enabled - BuildTime: 999.97us (999970) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 10 (10) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35724416) - RowsRepartitioned: 0 (0) - RowsReturned: 114 (114) - RowsReturnedRate: 57001 per second (57001) - SpilledPartitions: 0 (0) - TotalTime: 1ms (1999941) EXCHANGE_NODE (id=24) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 114 (114) - RowsReturnedRate: 0 per second (0) - TotalTime: 0ns (0) DataStreamReceiver - BytesReceived: 6.1 KiB (6264) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 20.9 KiB (21392) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 999.97us (999970) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.5 MiB (8912896) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 4.0 MiB (4194304) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 4.0 MiB (4194304) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Averaged Fragment F00 split sizes: min: 484.83 MB, max: 564.27 MB, avg: 508.95 MB, stddev: 27.24 MB completion times: min:43s946ms max:44s157ms mean: 44s045ms stddev:67.548ms execution rates: min:11.01 MB/sec max:12.82 MB/sec mean:11.56 MB/sec stddev:633.92 KB/sec num instances: 6 - AverageThreadTokens: 1.98 (1.9809517589690004) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 119.7 MiB (125551005) - PeakReservation: 74.0 MiB (77594624) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 307.6 MiB (322494863) - RowsProduced: 107 (107) - TotalNetworkReceiveTime: 42.98s (42985733381) - TotalNetworkSendTime: 7ms (7166675) - TotalStorageWaitTime: 100ms (100833174) - TotalThreadsInvoluntaryContextSwitches: 58 (58) - TotalThreadsTotalWallClockTime: 1.5m (87243293814) - TotalThreadsSysTime: 67ms (67255500) - TotalThreadsUserTime: 717ms (717115333) - TotalThreadsVoluntaryContextSwitches: 303 (303) - TotalTime: 44.02s (44019558170) Fragment Instance Lifecycle Timings - ExecTime: 146ms (146332804) - ExecTreeExecTime: 135ms (135166100) - InactiveTotalTime: 0ns (0) - OpenTime: 43.74s (43735728350) - ExecTreeOpenTime: 43.11s (43114899036) - PrepareTime: 136ms (136497045) - ExecTreePrepareTime: 166.67us (166667) - TotalTime: 0ns (0) DataStreamSender (dst_id=24) - BytesSent: 5.7 KiB (5828) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 3.6 MiB/s (3791821) - PeakMemoryUsage: 44.7 KiB (45792) - RowsReturned: 107 (107) - SerializeBatchTime: 166.67us (166667) - TotalTime: 25ms (25666036) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 8.5 KiB (8667) CodeGen - CodegenTime: 5ms (5166645) - CompileTime: 157ms (157999200) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 118 (118) - NumInstructions: 3,826 (3826) - OptimizationTime: 460ms (460330118) - PeakMemoryUsage: 1.9 MiB (1958912) - PrepareTime: 70ms (70165341) - TotalTime: 692ms (692161249) AGGREGATION_NODE (id=9) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 10,752 (10752) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 44.8 MiB (46998692) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 107 (107) - RowsReturnedRate: 2 per second (2) - StreamingTime: 7ms (7833372) - TotalTime: 43.25s (43250231804) HASH_JOIN_NODE (id=8) - BuildRows: 140,814 (140814) - BuildTime: 30ms (30833256) - InactiveTotalTime: 0ns (0) - LocalTime: 35ms (35666561) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 48.0 MiB (50375488) - ProbeRows: 83 (83) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 287 (287) - RowsReturnedRate: 6 per second (6) - TotalTime: 43.24s (43239565124) Hash Join Builder (join_node_id=8) - BuildRowsPartitionTime: 19ms (19999929) - BuildRowsPartitioned: 140,814 (140814) - HashBuckets: 262,144 (262144) - HashCollisions: 0 (0) - HashTablesBuildTime: 10ms (10166662) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 21.1 KiB (21632) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=23) - ConvertRowBatchTime: 1ms (1000006) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 140,814 (140814) - RowsReturnedRate: 3275 per second (3275) - TotalTime: 42.99s (42986900054) DataStreamReceiver - BytesReceived: 1.6 MiB (1727467) - DeserializeRowBatchTimer: 2ms (2999987) - FirstBatchArrivalWaitTime: 42.65s (42652734195) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 121.0 KiB (123904) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=7) - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.001984126984126984) - BytesRead: 88.4 MiB (92725764) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 88.4 MiB (92725764) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 88.4 MiB (92725764) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 53 (53) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 9 (9) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 53.2 MiB (55804879) - PerReadThreadRawHdfsThroughput: 2.4 GiB/s (2561482503) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 194ms (194831885) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,302,960 (2302960) - RowsReturned: 83 (83) - RowsReturnedRate: 431 per second (431) - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 10 (10) - ScannerThreadsTotalWallClockTime: 342ms (342999340) - MaterializeTupleTime(*): 167ms (167666304) - ScannerThreadsSysTime: 42ms (42696666) - ScannerThreadsUserTime: 193ms (193805500) - ScannerThreadsVoluntaryContextSwitches: 105 (105) - TotalRawHdfsReadTime(*): 36ms (36499914) - TotalReadThroughput: 39.2 KiB/s (40126) - TotalTime: 217ms (217665161) Filter 0 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,302,960 (2302960) - Rows rejected: 2,300,154 (2300154) - Rows total: 2,302,960 (2302960) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 2 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 2,806 (2806) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 6 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,806 (2806) - Rows rejected: 2,722 (2722) - Rows total: 2,806 (2806) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 3ms (3166608) - CumulativeAllocationBytes: 40.0 MiB (41943040) - CumulativeAllocations: 20 (20) - InactiveTotalTime: 0ns (0) - PeakReservation: 40.0 MiB (41943040) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 40.0 MiB (41943040) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 500.01us (500008) - CumulativeAllocationBytes: 23.0 MiB (24117248) - CumulativeAllocations: 11 (11) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 23.0 MiB (24117248) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment F00 Instance 68440bc099e1b843:8baf45150000000e (host=ip-10-1-18-116.ec2.internal:22000) Hdfs split stats (:<# splits>/): 5:1/45.48 MB 7:1/22.04 MB 2:1/28.41 MB 6:1/20.19 MB 10:2/18.58 MB 8:1/8.00 MB 4:2/43.54 MB 0:1/66.32 MB 11:2/41.67 MB 3:2/40.87 MB 1:2/72.29 MB 9:1/91.71 MB Filter 0 arrival: 2s202ms Filter 6 arrival: 43s783ms - AverageThreadTokens: 1.98 (1.9772727272727273) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 146.1 MiB (153174045) - PeakReservation: 74.0 MiB (77594624) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 314.6 MiB (329895427) - RowsProduced: 302 (302) - TotalNetworkReceiveTime: 43.26s (43262062995) - TotalNetworkSendTime: 18ms (18000443) - TotalStorageWaitTime: 171ms (171004204) - TotalThreadsInvoluntaryContextSwitches: 74 (74) - TotalThreadsTotalWallClockTime: 1.5m (87855158688) - TotalThreadsSysTime: 83ms (83263000) - TotalThreadsUserTime: 731ms (731464000) - TotalThreadsVoluntaryContextSwitches: 269 (269) - TotalTime: 44.09s (44094083435) Fragment Instance Lifecycle Timings - ExecTime: 268ms (268006585) - ExecTreeExecTime: 243ms (243005971) - InactiveTotalTime: 0ns (0) - OpenTime: 43.80s (43795076089) - ExecTreeOpenTime: 43.34s (43337064835) - PrepareTime: 31ms (31000761) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=24) - BytesSent: 15.7 KiB (16084) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 15.3 MiB/s (16083597) - PeakMemoryUsage: 60.7 KiB (62176) - RowsReturned: 302 (302) - SerializeBatchTime: 0ns (0) - TotalTime: 1ms (1000025) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 23.9 KiB (24462) CodeGen - CodegenTime: 5ms (5000123) - CompileTime: 137ms (137003366) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 118 (118) - NumInstructions: 3,826 (3826) - OptimizationTime: 319ms (319007838) - PeakMemoryUsage: 1.9 MiB (1958912) - PrepareTime: 23ms (23000567) - TotalTime: 480ms (480011796) AGGREGATION_NODE (id=9) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 3 (3) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 47.9 MiB (50253671) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 302 (302) - RowsReturnedRate: 6 per second (6) - StreamingTime: 16ms (16000393) - TotalTime: 43.58s (43580070806) HASH_JOIN_NODE (id=8) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 140,814 (140814) - BuildTime: 42ms (42001030) - InactiveTotalTime: 0ns (0) - LocalTime: 44ms (44001078) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 48.0 MiB (50375488) - ProbeRows: 239 (239) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 821 (821) - RowsReturnedRate: 18 per second (18) - TotalTime: 43.56s (43560070314) Hash Join Builder (join_node_id=8) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 27ms (27000661) - BuildRowsPartitioned: 140,814 (140814) - HashBuckets: 262,144 (262144) - HashCollisions: 0 (0) - HashTablesBuildTime: 14ms (14000344) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 21.1 KiB (21632) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=23) - ConvertRowBatchTime: 2ms (2000049) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 140,814 (140814) - RowsReturnedRate: 3254 per second (3254) - TotalTime: 43.26s (43264063044) DataStreamReceiver - BytesReceived: 1.6 MiB (1727467) - DeserializeRowBatchTimer: 4ms (4000097) - FirstBatchArrivalWaitTime: 42.94s (42939055055) - InactiveTotalTime: 43.26s (43262062995) - PeakMemoryUsage: 66.0 KiB (67584) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=7) Hdfs split stats (:<# splits>/): 5:1/45.48 MB 7:1/22.04 MB 2:1/28.41 MB 6:1/20.19 MB 10:2/18.58 MB 8:1/8.00 MB 4:2/43.54 MB 0:1/66.32 MB 11:2/41.67 MB 3:2/40.87 MB 1:2/72.29 MB 9:1/91.71 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 17 out of 17 Runtime filters: Not all filters arrived (arrived: [0, 6], missing [2]), waited for 20ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:17 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 89.8 MiB (94116749) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 89.8 MiB (94116749) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 89.8 MiB (94116749) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 51 (51) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 64.1 MiB (67185554) - PerReadThreadRawHdfsThroughput: 2.2 GiB/s (2413190758) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 230ms (230005651) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,379,429 (2379429) - RowsReturned: 239 (239) - RowsReturnedRate: 948 per second (948) - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 26 (26) - ScannerThreadsTotalWallClockTime: 484ms (484011892) - MaterializeTupleTime(*): 223ms (223005483) - ScannerThreadsSysTime: 56ms (56073000) - ScannerThreadsUserTime: 243ms (243509000) - ScannerThreadsVoluntaryContextSwitches: 80 (80) - TotalRawHdfsReadTime(*): 39ms (39000957) - TotalReadThroughput: 0 B/s (0) - TotalTime: 252ms (252006192) Filter 0 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,379,429 (2379429) - Rows rejected: 2,377,418 (2377418) - Rows total: 2,379,429 (2379429) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 2 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 2,011 (2011) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 6 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,011 (2011) - Rows rejected: 1,772 (1772) - Rows total: 2,011 (2011) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 2ms (2000050) - CumulativeAllocationBytes: 40.0 MiB (41943040) - CumulativeAllocations: 20 (20) - InactiveTotalTime: 0ns (0) - PeakReservation: 40.0 MiB (41943040) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 40.0 MiB (41943040) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 2ms (2000050) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf45150000000f (host=ip-10-1-27-82.ec2.internal:22000) Hdfs split stats (:<# splits>/): 2:1/20.95 MB 1:1/10.72 MB 8:2/30.20 MB 9:4/50.55 MB 5:2/40.20 MB 0:2/39.94 MB 6:3/42.46 MB 7:1/25.99 MB 3:3/162.40 MB 10:3/78.53 MB Filter 0 arrival: 2s016ms Filter 6 arrival: 43s540ms - AverageThreadTokens: 1.97 (1.9655172413793103) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 119.0 MiB (124729548) - PeakReservation: 74.0 MiB (77594624) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 303.4 MiB (318098845) - RowsProduced: 6 (6) - TotalNetworkReceiveTime: 42.00s (41997769437) - TotalNetworkSendTime: 12ms (12999619) - TotalStorageWaitTime: 151ms (151995542) - TotalThreadsInvoluntaryContextSwitches: 84 (84) - TotalThreadsTotalWallClockTime: 1.4m (85913482676) - TotalThreadsSysTime: 73ms (73320000) - TotalThreadsUserTime: 885ms (885917000) - TotalThreadsVoluntaryContextSwitches: 297 (297) - TotalTime: 44.04s (44041709548) Fragment Instance Lifecycle Timings - ExecTime: 354ms (354989598) - ExecTreeExecTime: 340ms (340990008) - InactiveTotalTime: 0ns (0) - OpenTime: 43.07s (43069738028) - ExecTreeOpenTime: 42.18s (42177764163) - PrepareTime: 610ms (610982098) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=24) - BytesSent: 422 B (422) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 3.2 KiB/s (3322) - PeakMemoryUsage: 36.7 KiB (37600) - RowsReturned: 6 (6) - SerializeBatchTime: 0ns (0) - TotalTime: 126ms (126996279) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 486 B (486) CodeGen - CodegenTime: 5ms (5999824) - CompileTime: 219ms (219993554) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 118 (118) - NumInstructions: 3,826 (3826) - OptimizationTime: 669ms (669980370) - PeakMemoryUsage: 1.9 MiB (1958912) - PrepareTime: 279ms (279991795) - TotalTime: 1.18s (1185965250) AGGREGATION_NODE (id=9) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 5,120 (5120) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 22 (22) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 49.2 MiB (51551567) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 6 (6) - RowsReturnedRate: 0 per second (0) - StreamingTime: 6ms (6999795) - TotalTime: 42.52s (42518754171) HASH_JOIN_NODE (id=8) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 140,814 (140814) - BuildTime: 43ms (43998710) - InactiveTotalTime: 0ns (0) - LocalTime: 46ms (46998622) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 48.0 MiB (50375488) - ProbeRows: 6 (6) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 9 (9) - RowsReturnedRate: 0 per second (0) - TotalTime: 42.50s (42502754640) Hash Join Builder (join_node_id=8) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 32ms (32999032) - BuildRowsPartitioned: 140,814 (140814) - HashBuckets: 262,144 (262144) - HashCollisions: 0 (0) - HashTablesBuildTime: 9ms (9999707) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 21.1 KiB (21632) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=23) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 140,814 (140814) - RowsReturnedRate: 3352 per second (3352) - TotalTime: 42.00s (41997769437) DataStreamReceiver - BytesReceived: 1.6 MiB (1727467) - DeserializeRowBatchTimer: 5ms (5999825) - FirstBatchArrivalWaitTime: 41.68s (41677778812) - InactiveTotalTime: 41.99s (41987769730) - PeakMemoryUsage: 396.0 KiB (405504) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=7) Hdfs split stats (:<# splits>/): 2:1/20.95 MB 1:1/10.72 MB 8:2/30.20 MB 9:4/50.55 MB 5:2/40.20 MB 0:2/39.94 MB 6:3/42.46 MB 7:1/25.99 MB 3:3/162.40 MB 10:3/78.53 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 22 out of 22 Runtime filters: Not all filters arrived (arrived: [0, 6], missing [2]), waited for 21ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:22 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.01 (0.011904761904761904) - BytesRead: 88.8 MiB (93087143) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 88.8 MiB (93087143) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 88.8 MiB (93087143) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 66 (66) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 10 (10) - NumRowGroups: 22 (22) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 41.0 MiB (43030177) - PerReadThreadRawHdfsThroughput: 2.2 GiB/s (2386919714) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 437ms (437987167) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,410,638 (2410638) - RowsReturned: 6 (6) - RowsReturnedRate: 13 per second (13) - ScanRangesComplete: 22 (22) - ScannerThreadsInvoluntaryContextSwitches: 28 (28) - ScannerThreadsTotalWallClockTime: 446ms (446986903) - MaterializeTupleTime(*): 210ms (210993825) - ScannerThreadsSysTime: 50ms (50454000) - ScannerThreadsUserTime: 244ms (244201000) - ScannerThreadsVoluntaryContextSwitches: 83 (83) - TotalRawHdfsReadTime(*): 38ms (38998858) - TotalReadThroughput: 235.1 KiB/s (240757) - TotalTime: 460ms (460986493) Filter 0 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,410,638 (2410638) - Rows rejected: 2,403,066 (2403066) - Rows total: 2,410,638 (2410638) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 2 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 7,572 (7572) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 6 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 7,572 (7572) - Rows rejected: 7,566 (7566) - Rows total: 7,572 (7572) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 12ms (12999620) - CumulativeAllocationBytes: 40.0 MiB (41943040) - CumulativeAllocations: 20 (20) - InactiveTotalTime: 0ns (0) - PeakReservation: 40.0 MiB (41943040) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 40.0 MiB (41943040) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 12.0 MiB (12582912) - CumulativeAllocations: 6 (6) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 12.0 MiB (12582912) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000012 (host=ip-10-1-16-98.ec2.internal:22000) Hdfs split stats (:<# splits>/): 4:1/15.87 MB 7:1/26.19 MB 5:1/52.06 MB 3:2/68.83 MB 9:1/60.53 MB 11:1/21.82 MB 8:2/45.19 MB 6:3/60.65 MB 0:3/133.69 MB Filter 0 arrival: 2s206ms Filter 6 arrival: 43s828ms - AverageThreadTokens: 1.99 (1.9886363636363635) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 133.4 MiB (139922165) - PeakReservation: 74.0 MiB (77594624) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 310.3 MiB (325335426) - RowsProduced: 273 (273) - TotalNetworkReceiveTime: 43.39s (43393148286) - TotalNetworkSendTime: 3ms (3000011) - TotalStorageWaitTime: 80ms (80000275) - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1.5m (87664299571) - TotalThreadsSysTime: 50ms (50524000) - TotalThreadsUserTime: 556ms (556796000) - TotalThreadsVoluntaryContextSwitches: 288 (288) - TotalTime: 44.02s (44020150428) Fragment Instance Lifecycle Timings - ExecTime: 141ms (141000482) - ExecTreeExecTime: 129ms (129000441) - InactiveTotalTime: 0ns (0) - OpenTime: 43.84s (43840149813) - ExecTreeOpenTime: 43.46s (43457148504) - PrepareTime: 39ms (39000133) - ExecTreePrepareTime: 1ms (1000003) - TotalTime: 0ns (0) DataStreamSender (dst_id=24) - BytesSent: 14.4 KiB (14737) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 3.5 MiB/s (3684238) - PeakMemoryUsage: 60.7 KiB (62176) - RowsReturned: 273 (273) - SerializeBatchTime: 1ms (1000003) - TotalTime: 4ms (4000013) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 21.6 KiB (22113) CodeGen - CodegenTime: 5ms (5000017) - CompileTime: 111ms (111000380) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 118 (118) - NumInstructions: 3,826 (3826) - OptimizationTime: 268ms (268000915) - PeakMemoryUsage: 1.9 MiB (1958912) - PrepareTime: 30ms (30000103) - TotalTime: 410ms (410001402) AGGREGATION_NODE (id=9) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 3 (3) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 50.0 MiB (52436895) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 273 (273) - RowsReturnedRate: 6 per second (6) - StreamingTime: 10ms (10000035) - TotalTime: 43.59s (43587148948) HASH_JOIN_NODE (id=8) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 140,814 (140814) - BuildTime: 20ms (20000068) - InactiveTotalTime: 0ns (0) - LocalTime: 23ms (23000079) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 48.0 MiB (50375488) - ProbeRows: 205 (205) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 728 (728) - RowsReturnedRate: 16 per second (16) - TotalTime: 43.58s (43577148913) Hash Join Builder (join_node_id=8) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 11ms (11000037) - BuildRowsPartitioned: 140,814 (140814) - HashBuckets: 262,144 (262144) - HashCollisions: 0 (0) - HashTablesBuildTime: 9ms (9000031) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 21.1 KiB (21632) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=23) - ConvertRowBatchTime: 1ms (1000003) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 140,814 (140814) - RowsReturnedRate: 3244 per second (3244) - TotalTime: 43.39s (43394148289) DataStreamReceiver - BytesReceived: 1.6 MiB (1727467) - DeserializeRowBatchTimer: 2ms (2000007) - FirstBatchArrivalWaitTime: 43.05s (43051147117) - InactiveTotalTime: 43.39s (43393148286) - PeakMemoryUsage: 66.0 KiB (67584) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=7) Hdfs split stats (:<# splits>/): 4:1/15.87 MB 7:1/26.19 MB 5:1/52.06 MB 3:2/68.83 MB 9:1/60.53 MB 11:1/21.82 MB 8:2/45.19 MB 6:3/60.65 MB 0:3/133.69 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 15 out of 15 Runtime filters: Not all filters arrived (arrived: [0, 6], missing [2]), waited for 20ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:15 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 83.9 MiB (88008492) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 83.9 MiB (88008492) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 83.9 MiB (88008492) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 45 (45) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 9 (9) - NumRowGroups: 15 (15) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 57.3 MiB (60086033) - PerReadThreadRawHdfsThroughput: 2.6 GiB/s (2838973997) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 139ms (139000474) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,198,214 (2198214) - RowsReturned: 205 (205) - RowsReturnedRate: 1281 per second (1281) - ScanRangesComplete: 15 (15) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 268ms (268000915) - MaterializeTupleTime(*): 133ms (133000450) - ScannerThreadsSysTime: 28ms (28282000) - ScannerThreadsUserTime: 157ms (157882000) - ScannerThreadsVoluntaryContextSwitches: 89 (89) - TotalRawHdfsReadTime(*): 31ms (31000105) - TotalReadThroughput: 0 B/s (0) - TotalTime: 160ms (160000545) Filter 0 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,198,214 (2198214) - Rows rejected: 2,193,997 (2193997) - Rows total: 2,198,214 (2198214) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 2 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 4,217 (4217) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 6 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 4,217 (4217) - Rows rejected: 4,012 (4012) - Rows total: 4,217 (4217) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 40.0 MiB (41943040) - CumulativeAllocations: 20 (20) - InactiveTotalTime: 0ns (0) - PeakReservation: 40.0 MiB (41943040) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 40.0 MiB (41943040) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 34.0 MiB (35651584) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 34.0 MiB (35651584) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000013 (host=ip-10-1-28-130.ec2.internal:22000) Hdfs split stats (:<# splits>/): 6:1/6.50 MB 8:3/96.75 MB 9:2/72.33 MB 7:3/82.19 MB 2:1/19.76 MB 11:1/6.05 MB 4:1/5.13 MB 3:2/21.53 MB 5:2/11.51 MB 1:1/8.41 MB 10:4/58.84 MB 0:4/175.26 MB Filter 0 arrival: 2s311ms Filter 6 arrival: 43s833ms - AverageThreadTokens: 1.99 (1.9885057471264367) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 82.0 MiB (85961286) - PeakReservation: 74.0 MiB (77594624) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 43.20s (43196628743) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 80ms (80999304) - TotalThreadsInvoluntaryContextSwitches: 34 (34) - TotalThreadsTotalWallClockTime: 1.5m (87445248443) - TotalThreadsSysTime: 51ms (51340000) - TotalThreadsUserTime: 727ms (727788000) - TotalThreadsVoluntaryContextSwitches: 366 (366) - TotalTime: 44.02s (44016621694) Fragment Instance Lifecycle Timings - ExecTime: 2ms (2999974) - ExecTreeExecTime: 999.99us (999991) - InactiveTotalTime: 0ns (0) - OpenTime: 43.94s (43940622348) - ExecTreeOpenTime: 43.39s (43389627083) - PrepareTime: 72ms (72999372) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=24) - BytesSent: 0 B (0) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 20.7 KiB (21216) - RowsReturned: 0 (0) - SerializeBatchTime: 0ns (0) - TotalTime: 12ms (12999888) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 0 B (0) CodeGen - CodegenTime: 6ms (6999939) - CompileTime: 155ms (155998660) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 118 (118) - NumInstructions: 3,826 (3826) - OptimizationTime: 391ms (391996631) - PeakMemoryUsage: 1.9 MiB (1958912) - PrepareTime: 41ms (41999640) - TotalTime: 592ms (592994905) AGGREGATION_NODE (id=9) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.1 MiB (35712128) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - StreamingTime: 999.99us (999991) - TotalTime: 43.39s (43390627074) HASH_JOIN_NODE (id=8) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 140,814 (140814) - BuildTime: 20ms (20999819) - InactiveTotalTime: 0ns (0) - LocalTime: 28ms (28999749) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 48.0 MiB (50375488) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.39s (43389627083) Hash Join Builder (join_node_id=8) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 10ms (10999905) - BuildRowsPartitioned: 140,814 (140814) - HashBuckets: 262,144 (262144) - HashCollisions: 0 (0) - HashTablesBuildTime: 8ms (8999923) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 21.1 KiB (21632) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=23) - ConvertRowBatchTime: 999.99us (999991) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 140,814 (140814) - RowsReturnedRate: 3259 per second (3259) - TotalTime: 43.20s (43197628734) DataStreamReceiver - BytesReceived: 1.6 MiB (1727467) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 42.86s (42855631673) - InactiveTotalTime: 43.20s (43196628743) - PeakMemoryUsage: 66.0 KiB (67584) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=7) Hdfs split stats (:<# splits>/): 6:1/6.50 MB 8:3/96.75 MB 9:2/72.33 MB 7:3/82.19 MB 2:1/19.76 MB 11:1/6.05 MB 4:1/5.13 MB 3:2/21.53 MB 5:2/11.51 MB 1:1/8.41 MB 10:4/58.84 MB 0:4/175.26 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 25 out of 25 Runtime filters: Not all filters arrived (arrived: [0, 6], missing [2]), waited for 20ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:25 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 82.6 MiB (86577288) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 82.6 MiB (86577288) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 82.6 MiB (86577288) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 75 (75) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 25 (25) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 39.9 MiB (41793122) - PerReadThreadRawHdfsThroughput: 2.6 GiB/s (2792839526) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 141ms (141998781) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,324,748 (2324748) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - ScanRangesComplete: 25 (25) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 282ms (282997568) - MaterializeTupleTime(*): 150ms (150998703) - ScannerThreadsSysTime: 32ms (32676000) - ScannerThreadsUserTime: 170ms (170468000) - ScannerThreadsVoluntaryContextSwitches: 158 (158) - TotalRawHdfsReadTime(*): 30ms (30999736) - TotalReadThroughput: 0 B/s (0) - TotalTime: 162ms (162998600) Filter 0 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,324,748 (2324748) - Rows rejected: 2,322,388 (2322388) - Rows total: 2,324,748 (2324748) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 2 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 2,360 (2360) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 6 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,360 (2360) - Rows rejected: 2,360 (2360) - Rows total: 2,360 (2360) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 999.99us (999991) - CumulativeAllocationBytes: 40.0 MiB (41943040) - CumulativeAllocations: 20 (20) - InactiveTotalTime: 0ns (0) - PeakReservation: 40.0 MiB (41943040) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 40.0 MiB (41943040) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 2.0 MiB (2097152) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 2.0 MiB (2097152) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000010 (host=ip-10-1-20-86.ec2.internal:22000) Hdfs split stats (:<# splits>/): 6:1/538.83 KB 2:1/5.46 MB 3:2/52.71 MB 11:2/53.01 MB 1:2/33.44 MB 9:1/1020.42 KB 0:2/22.05 MB 8:3/196.55 MB 4:2/153.78 MB Filter 0 arrival: 2s270ms Filter 6 arrival: 43s829ms - AverageThreadTokens: 1.98 (1.9772727272727273) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 91.1 MiB (95514334) - PeakReservation: 74.0 MiB (77594624) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 312.0 MiB (327162809) - RowsProduced: 48 (48) - TotalNetworkReceiveTime: 42.82s (42825693147) - TotalNetworkSendTime: 3ms (3999971) - TotalStorageWaitTime: 57ms (57999583) - TotalThreadsInvoluntaryContextSwitches: 125 (125) - TotalThreadsTotalWallClockTime: 1.5m (87102375891) - TotalThreadsSysTime: 75ms (75331000) - TotalThreadsUserTime: 715ms (715059000) - TotalThreadsVoluntaryContextSwitches: 297 (297) - TotalTime: 44.01s (44008684668) Fragment Instance Lifecycle Timings - ExecTime: 6ms (6999950) - ExecTreeExecTime: 2ms (2999979) - InactiveTotalTime: 0ns (0) - OpenTime: 43.98s (43976684897) - ExecTreeOpenTime: 43.01s (43014691790) - PrepareTime: 24ms (24999821) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=24) - BytesSent: 2.8 KiB (2873) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 2.7 MiB/s (2873020) - PeakMemoryUsage: 44.7 KiB (45792) - RowsReturned: 48 (48) - SerializeBatchTime: 0ns (0) - TotalTime: 999.99us (999993) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 3.8 KiB (3888) CodeGen - CodegenTime: 4ms (4999965) - CompileTime: 157ms (157998868) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 118 (118) - NumInstructions: 3,826 (3826) - OptimizationTime: 801ms (801994254) - PeakMemoryUsage: 1.9 MiB (1958912) - PrepareTime: 16ms (16999876) - TotalTime: 976ms (976992998) AGGREGATION_NODE (id=9) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 15,360 (15360) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 3 (3) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 38.2 MiB (40066079) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 48 (48) - RowsReturnedRate: 1 per second (1) - StreamingTime: 999.99us (999993) - TotalTime: 43.02s (43017691769) HASH_JOIN_NODE (id=8) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 140,814 (140814) - BuildTime: 22ms (22999832) - InactiveTotalTime: 0ns (0) - LocalTime: 33ms (33999755) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 48.0 MiB (50375488) - ProbeRows: 39 (39) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 130 (130) - RowsReturnedRate: 3 per second (3) - TotalTime: 43.02s (43016691776) Hash Join Builder (join_node_id=8) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 14ms (14999890) - BuildRowsPartitioned: 140,814 (140814) - HashBuckets: 262,144 (262144) - HashCollisions: 0 (0) - HashTablesBuildTime: 7ms (7999942) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 21.1 KiB (21632) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=23) - ConvertRowBatchTime: 999.99us (999993) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 140,814 (140814) - RowsReturnedRate: 3287 per second (3287) - TotalTime: 42.83s (42826693140) DataStreamReceiver - BytesReceived: 1.6 MiB (1727467) - DeserializeRowBatchTimer: 1ms (1999984) - FirstBatchArrivalWaitTime: 42.48s (42484695588) - InactiveTotalTime: 42.82s (42824693154) - PeakMemoryUsage: 66.0 KiB (67584) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=7) Hdfs split stats (:<# splits>/): 6:1/538.83 KB 2:1/5.46 MB 3:2/52.71 MB 11:2/53.01 MB 1:2/33.44 MB 9:1/1020.42 KB 0:2/22.05 MB 8:3/196.55 MB 4:2/153.78 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 16 out of 16 Runtime filters: Not all filters arrived (arrived: [0, 6], missing [2]), waited for 21ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:16 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 90.2 MiB (94626186) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 90.2 MiB (94626186) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 90.2 MiB (94626186) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 50 (50) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 9 (9) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 49.0 MiB (51346170) - PerReadThreadRawHdfsThroughput: 2.4 GiB/s (2557483080) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 132ms (132999045) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,215,077 (2215077) - RowsReturned: 39 (39) - RowsReturnedRate: 250 per second (250) - ScanRangesComplete: 16 (16) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 268ms (268998072) - MaterializeTupleTime(*): 136ms (136999017) - ScannerThreadsSysTime: 42ms (42201000) - ScannerThreadsUserTime: 162ms (162898000) - ScannerThreadsVoluntaryContextSwitches: 104 (104) - TotalRawHdfsReadTime(*): 36ms (36999731) - TotalReadThroughput: 0 B/s (0) - TotalTime: 155ms (155998881) Filter 0 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,215,077 (2215077) - Rows rejected: 2,214,699 (2214699) - Rows total: 2,215,077 (2215077) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 2 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 378 (378) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 6 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 378 (378) - Rows rejected: 339 (339) - Rows total: 378 (378) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 1ms (1999985) - CumulativeAllocationBytes: 40.0 MiB (41943040) - CumulativeAllocations: 20 (20) - InactiveTotalTime: 0ns (0) - PeakReservation: 40.0 MiB (41943040) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 40.0 MiB (41943040) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 32.0 MiB (33554432) - CumulativeAllocations: 16 (16) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 32.0 MiB (33554432) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000011 (host=ip-10-1-21-97.ec2.internal:22000) Hdfs split stats (:<# splits>/): 3:2/40.27 MB 11:2/62.14 MB 10:1/26.26 MB 1:1/26.16 MB 8:1/30.70 MB 4:3/186.89 MB 9:2/112.62 MB Filter 0 arrival: 2s238ms Filter 6 arrival: 43s783ms - AverageThreadTokens: 1.99 (1.9885057471264367) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 146.9 MiB (154004654) - PeakReservation: 74.0 MiB (77594624) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 314.4 MiB (329677036) - RowsProduced: 13 (13) - TotalNetworkReceiveTime: 43.24s (43239097678) - TotalNetworkSendTime: 5ms (5000011) - TotalStorageWaitTime: 63ms (63000141) - TotalThreadsInvoluntaryContextSwitches: 27 (27) - TotalThreadsTotalWallClockTime: 1.5m (87479197616) - TotalThreadsSysTime: 69ms (69755000) - TotalThreadsUserTime: 685ms (685668000) - TotalThreadsVoluntaryContextSwitches: 304 (304) - TotalTime: 43.94s (43936099251) Fragment Instance Lifecycle Timings - ExecTime: 104ms (104000236) - ExecTreeExecTime: 94ms (94000213) - InactiveTotalTime: 0ns (0) - OpenTime: 43.79s (43792098925) - ExecTreeOpenTime: 43.31s (43313097844) - PrepareTime: 40ms (40000090) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=24) - BytesSent: 854 B (854) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 104.2 KiB/s (106749) - PeakMemoryUsage: 44.7 KiB (45792) - RowsReturned: 13 (13) - SerializeBatchTime: 0ns (0) - TotalTime: 8ms (8000018) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 1.0 KiB (1053) CodeGen - CodegenTime: 3ms (3000006) - CompileTime: 166ms (166000374) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 118 (118) - NumInstructions: 3,826 (3826) - OptimizationTime: 311ms (311000702) - PeakMemoryUsage: 1.9 MiB (1958912) - PrepareTime: 29ms (29000066) - TotalTime: 507ms (507001144) AGGREGATION_NODE (id=9) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 11,264 (11264) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 5 (5) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 49.6 MiB (51971816) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 13 (13) - RowsReturnedRate: 0 per second (0) - StreamingTime: 12ms (12000028) - TotalTime: 43.41s (43407098057) HASH_JOIN_NODE (id=8) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 140,814 (140814) - BuildTime: 35ms (35000078) - InactiveTotalTime: 0ns (0) - LocalTime: 37ms (37000083) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 48.0 MiB (50375488) - ProbeRows: 11 (11) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 35 (35) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.39s (43391098021) Hash Join Builder (join_node_id=8) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 23ms (23000050) - BuildRowsPartitioned: 140,814 (140814) - HashBuckets: 262,144 (262144) - HashCollisions: 0 (0) - HashTablesBuildTime: 11ms (11000025) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 21.1 KiB (21632) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=23) - ConvertRowBatchTime: 1ms (1000002) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 140,814 (140814) - RowsReturnedRate: 3256 per second (3256) - TotalTime: 43.24s (43241097682) DataStreamReceiver - BytesReceived: 1.6 MiB (1727467) - DeserializeRowBatchTimer: 4ms (4000009) - FirstBatchArrivalWaitTime: 42.91s (42908096925) - InactiveTotalTime: 43.24s (43238097676) - PeakMemoryUsage: 66.0 KiB (67584) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=7) Hdfs split stats (:<# splits>/): 3:2/40.27 MB 11:2/62.14 MB 10:1/26.26 MB 1:1/26.16 MB 8:1/30.70 MB 4:3/186.89 MB 9:2/112.62 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 12 out of 12 Runtime filters: Not all filters arrived (arrived: [0, 6], missing [2]), waited for 20ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% File Formats: PARQUET/NONE:12 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 95.3 MiB (99938727) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 95.3 MiB (99938727) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 95.3 MiB (99938727) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 36 (36) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 7 (7) - NumRowGroups: 12 (12) - NumScannerThreadsStarted: 3 (3) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 68.1 MiB (71388219) - PerReadThreadRawHdfsThroughput: 2.2 GiB/s (2379487947) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 87ms (87000197) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,289,654 (2289654) - RowsReturned: 11 (11) - RowsReturnedRate: 96 per second (96) - ScanRangesComplete: 12 (12) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 307ms (307000695) - MaterializeTupleTime(*): 151ms (151000348) - ScannerThreadsSysTime: 46ms (46494000) - ScannerThreadsUserTime: 183ms (183875000) - ScannerThreadsVoluntaryContextSwitches: 116 (116) - TotalRawHdfsReadTime(*): 42ms (42000098) - TotalReadThroughput: 0 B/s (0) - TotalTime: 114ms (114000258) Filter 0 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,289,654 (2289654) - Rows rejected: 2,289,356 (2289356) - Rows total: 2,289,654 (2289654) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 2 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 298 (298) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 6 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 298 (298) - Rows rejected: 287 (287) - Rows total: 298 (298) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 1ms (1000002) - CumulativeAllocationBytes: 40.0 MiB (41943040) - CumulativeAllocations: 20 (20) - InactiveTotalTime: 0ns (0) - PeakReservation: 40.0 MiB (41943040) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 40.0 MiB (41943040) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 1ms (1000002) - CumulativeAllocationBytes: 24.0 MiB (25165824) - CumulativeAllocations: 12 (12) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 24.0 MiB (25165824) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Averaged Fragment F01 split sizes: min: 15.06 GB, max: 17.50 GB, avg: 16.91 GB, stddev: 938.99 MB completion times: min:43s440ms max:43s755ms mean: 43s562ms stddev:113.081ms execution rates: min:352.40 MB/sec max:412.20 MB/sec mean:397.47 MB/sec stddev:22.29 MB/sec num instances: 6 - AverageThreadTokens: 1.98 (1.9845852267664614) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 7.6 MiB (7989937) - PeakReservation: 1.9 MiB (2031616) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 303.8 MiB (318598954) - RowsProduced: 23,469 (23469) - TotalNetworkReceiveTime: 42.70s (42699899800) - TotalNetworkSendTime: 31ms (31333106) - TotalStorageWaitTime: 130ms (130498790) - TotalThreadsInvoluntaryContextSwitches: 36 (36) - TotalThreadsTotalWallClockTime: 1.4m (86232127346) - TotalThreadsSysTime: 14ms (14659666) - TotalThreadsUserTime: 431ms (431178333) - TotalThreadsVoluntaryContextSwitches: 114 (114) - TotalTime: 43.53s (43531392306) Fragment Instance Lifecycle Timings - ExecTime: 128ms (128498378) - ExecTreeExecTime: 90ms (90998632) - InactiveTotalTime: 0ns (0) - OpenTime: 43.19s (43189897534) - ExecTreeOpenTime: 42.76s (42755899581) - PrepareTime: 212ms (212829726) - ExecTreePrepareTime: 138ms (138164259) - TotalTime: 0ns (0) DataStreamSender (dst_id=23) - BytesSent: 1.6 MiB (1727467) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 461.8 MiB/s (484235401) - OverallThroughput: 106.2 MiB/s (111336683) - PeakMemoryUsage: 30.7 KiB (31440) - RowsReturned: 23,469 (23469) - SerializeBatchTime: 2ms (2333313) - TotalTime: 25ms (25499461) - TransmitDataRPCTime: 1ms (1999982) - UncompressedRowBatchSize: 4.4 MiB (4646862) CodeGen - CodegenTime: 2ms (2499989) - CompileTime: 133ms (133832804) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 104 (104) - NumInstructions: 3,191 (3191) - OptimizationTime: 295ms (295831890) - PeakMemoryUsage: 1.6 MiB (1633792) - PrepareTime: 44ms (44499370) - TotalTime: 474ms (474830730) HASH_JOIN_NODE (id=6) - BuildRows: 1 (1) - BuildTime: 1ms (1333331) - InactiveTotalTime: 0ns (0) - LocalTime: 2ms (2166660) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.2 MiB (2329856) - ProbeRows: 23,469 (23469) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 166.66us (166665) - RowsReturned: 23,469 (23469) - RowsReturnedRate: 544 per second (544) - TotalTime: 43.12s (43123226731) Hash Join Builder (join_node_id=6) - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=22) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 42.70s (42699899800) DataStreamReceiver - BytesReceived: 13 B (13) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 42.70s (42699399804) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=3) - AverageHdfsReadThreadConcurrency: 0.00 (0.003875968992248062) - AverageScannerThreadConcurrency: 0.00 (0.003875968992248062) - BytesRead: 5.8 MiB (6065633) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 5.8 MiB (6065633) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 5.8 MiB (6065633) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 24 (24) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 3 (3) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 6 (6) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 12 (12) - PeakMemoryUsage: 4.4 MiB (4597634) - PerReadThreadRawHdfsThroughput: 574.3 MiB/s (602191862) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 95ms (95498582) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 740,203 (740203) - RowsReturned: 23,469 (23469) - RowsReturnedRate: 74817 per second (74817) - ScanRangesComplete: 10,252 (10252) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 212ms (212498301) - MaterializeTupleTime(*): 42ms (42666403) - ScannerThreadsSysTime: 10ms (10340333) - ScannerThreadsUserTime: 55ms (55617666) - ScannerThreadsVoluntaryContextSwitches: 46 (46) - TotalRawHdfsReadTime(*): 105ms (105165775) - TotalReadThroughput: 3.5 KiB/s (3611) - TotalTime: 422ms (422993598) Filter 2 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 740,203 (740203) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 3 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 740,203 (740203) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 8 (1.00 MB) - Files processed: 10,252 (10252) - Files rejected: 10,240 (10240) - Files total: 10,252 (10252) - InactiveTotalTime: 0ns (0) - RowGroups processed: 25 (25) - RowGroups rejected: 0 (0) - RowGroups total: 25 (25) - Rows processed: 714,855 (714855) - Rows rejected: 0 (0) - Rows total: 740,203 (740203) - Splits processed: 12 (12) - Splits rejected: 0 (0) - Splits total: 12 (12) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment F01 Instance 68440bc099e1b843:8baf451500000009 (host=ip-10-1-21-97.ec2.internal:22000) Hdfs split stats (:<# splits>/): 7:838/1.15 GB 2:876/1.46 GB 5:798/1.29 GB 6:772/1.33 GB 11:836/1.50 GB 10:837/1.65 GB 1:769/1.21 GB 4:739/1.10 GB 8:781/1003.47 MB 9:778/1.35 GB 3:748/1.10 GB 0:774/970.37 MB Filter 8 arrival: 43s369ms - AverageThreadTokens: 2.00 (2.0) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 12.8 MiB (13423150) - PeakReservation: 1.9 MiB (2031616) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 301.4 MiB (316058244) - RowsProduced: 68,088 (68088) - TotalNetworkReceiveTime: 42.87s (42872096843) - TotalNetworkSendTime: 90ms (90000205) - TotalStorageWaitTime: 434ms (434000984) - TotalThreadsInvoluntaryContextSwitches: 53 (53) - TotalThreadsTotalWallClockTime: 1.5m (87163196902) - TotalThreadsSysTime: 48ms (48779000) - TotalThreadsUserTime: 576ms (576008000) - TotalThreadsVoluntaryContextSwitches: 249 (249) - TotalTime: 43.74s (43744098816) Fragment Instance Lifecycle Timings - ExecTime: 320ms (320000726) - ExecTreeExecTime: 219ms (219000496) - InactiveTotalTime: 0ns (0) - OpenTime: 43.32s (43323097862) - ExecTreeOpenTime: 42.93s (42929096973) - PrepareTime: 101ms (101000228) - ExecTreePrepareTime: 59ms (59000133) - TotalTime: 0ns (0) DataStreamSender (dst_id=23) - BytesSent: 4.8 MiB (5011440) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 1.6 GiB/s (1670476659) - OverallThroughput: 341.4 MiB/s (357959207) - PeakMemoryUsage: 30.7 KiB (31440) - RowsReturned: 68,088 (68088) - SerializeBatchTime: 5ms (5000011) - TotalTime: 14ms (14000031) - TransmitDataRPCTime: 3ms (3000006) - UncompressedRowBatchSize: 12.9 MiB (13481424) CodeGen - CodegenTime: 3ms (3000007) - CompileTime: 106ms (106000239) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 104 (104) - NumInstructions: 3,191 (3191) - OptimizationTime: 286ms (286000645) - PeakMemoryUsage: 1.6 MiB (1633792) - PrepareTime: 30ms (30000068) - TotalTime: 423ms (423000954) HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 2ms (2000005) - InactiveTotalTime: 0ns (0) - LocalTime: 3ms (3000007) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.7 MiB (2817280) - ProbeRows: 68,088 (68088) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 68,088 (68088) - RowsReturnedRate: 1573 per second (1573) - TotalTime: 43.27s (43266097735) Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=22) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 42.87s (42872096843) DataStreamReceiver - BytesReceived: 13 B (13) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 42.87s (42872096843) - InactiveTotalTime: 42.87s (42872096843) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=3) Hdfs split stats (:<# splits>/): 7:838/1.15 GB 2:876/1.46 GB 5:798/1.29 GB 6:772/1.33 GB 11:836/1.50 GB 10:837/1.65 GB 1:769/1.21 GB 4:739/1.10 GB 8:781/1003.47 MB 9:778/1.35 GB 3:748/1.10 GB 0:774/970.37 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 27 out of 27 Runtime filters: Not all filters arrived (arrived: [8], missing [2, 3]), waited for 40ms Hdfs Read Thread Concurrency Bucket: 0:98.84% 1:0% 2:1.163% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% File Formats: PARQUET/NONE:27 PARQUET/Unknown(Skipped):9519 - AverageHdfsReadThreadConcurrency: 0.02 (0.023255813953488372) - AverageScannerThreadConcurrency: 0.02 (0.023255813953488372) - BytesRead: 16.3 MiB (17060562) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 16.3 MiB (17060562) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 16.3 MiB (17060562) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 60 (60) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 3 (3) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 27 (27) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 16 (16) - PeakMemoryUsage: 9.4 MiB (9827412) - PerReadThreadRawHdfsThroughput: 39.6 MiB/s (41509788) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 224ms (224000508) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,153,398 (2153398) - RowsReturned: 68,088 (68088) - RowsReturnedRate: 173251 per second (173251) - ScanRangesComplete: 9,546 (9546) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 646ms (646001466) - MaterializeTupleTime(*): 128ms (128000292) - ScannerThreadsSysTime: 38ms (38360000) - ScannerThreadsUserTime: 179ms (179553000) - ScannerThreadsVoluntaryContextSwitches: 102 (102) - TotalRawHdfsReadTime(*): 411ms (411000934) - TotalReadThroughput: 21.2 KiB/s (21669) - TotalTime: 393ms (393000890) Filter 2 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 2,153,398 (2153398) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 3 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 2,153,398 (2153398) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 8 (1.00 MB) - Files processed: 9,546 (9546) - Files rejected: 9,519 (9519) - Files total: 9,546 (9546) - InactiveTotalTime: 0ns (0) - RowGroups processed: 74 (74) - RowGroups rejected: 0 (0) - RowGroups total: 74 (74) - Rows processed: 2,129,613 (2129613) - Rows rejected: 0 (0) - Rows total: 2,153,398 (2153398) - Splits processed: 27 (27) - Splits rejected: 0 (0) - Splits total: 27 (27) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000008 (host=ip-10-1-20-86.ec2.internal:22000) Hdfs split stats (:<# splits>/): 0:760/1.06 GB 7:835/1.59 GB 2:822/1.65 GB 8:808/1.40 GB 1:799/1.41 GB 6:827/1.55 GB 10:850/1.47 GB 11:821/1.48 GB 9:747/1.19 GB 5:753/1.39 GB 3:707/1.01 GB 4:775/1.22 GB Filter 8 arrival: 43s415ms - AverageThreadTokens: 1.98 (1.9770114942528736) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 14.8 MiB (15472174) - PeakReservation: 1.9 MiB (2031616) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 312.0 MiB (327162809) - RowsProduced: 54,011 (54011) - TotalNetworkReceiveTime: 42.65s (42650694399) - TotalNetworkSendTime: 58ms (58999577) - TotalStorageWaitTime: 40ms (40999707) - TotalThreadsInvoluntaryContextSwitches: 75 (75) - TotalThreadsTotalWallClockTime: 1.4m (86278381799) - TotalThreadsSysTime: 20ms (20705000) - TotalThreadsUserTime: 556ms (556813000) - TotalThreadsVoluntaryContextSwitches: 205 (205) - TotalTime: 43.56s (43560687878) Fragment Instance Lifecycle Timings - ExecTime: 93ms (93999326) - ExecTreeExecTime: 24ms (24999822) - InactiveTotalTime: 0ns (0) - OpenTime: 43.36s (43365689276) - ExecTreeOpenTime: 42.70s (42701694034) - PrepareTime: 100ms (100999276) - ExecTreePrepareTime: 66ms (66999520) - TotalTime: 0ns (0) DataStreamSender (dst_id=23) - BytesSent: 3.8 MiB (3980748) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 542.3 MiB/s (568682428) - OverallThroughput: 271.2 MiB/s (284341194) - PeakMemoryUsage: 30.7 KiB (31440) - RowsReturned: 54,011 (54011) - SerializeBatchTime: 5ms (5999957) - TotalTime: 13ms (13999899) - TransmitDataRPCTime: 6ms (6999949) - UncompressedRowBatchSize: 10.2 MiB (10694178) CodeGen - CodegenTime: 1ms (1999986) - CompileTime: 265ms (265998094) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 104 (104) - NumInstructions: 3,191 (3191) - OptimizationTime: 395ms (395997162) - PeakMemoryUsage: 1.6 MiB (1633792) - PrepareTime: 25ms (25999813) - TotalTime: 688ms (688995062) HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LocalTime: 3ms (3999972) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.4 MiB (2503936) - ProbeRows: 54,011 (54011) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 999.99us (999993) - RowsReturned: 54,011 (54011) - RowsReturnedRate: 1260 per second (1260) - TotalTime: 42.86s (42860692896) Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=22) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 42.65s (42650694399) DataStreamReceiver - BytesReceived: 13 B (13) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 42.65s (42648694413) - InactiveTotalTime: 42.65s (42650694399) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=3) Hdfs split stats (:<# splits>/): 0:760/1.06 GB 7:835/1.59 GB 2:822/1.65 GB 8:808/1.40 GB 1:799/1.41 GB 6:827/1.55 GB 10:850/1.47 GB 11:821/1.48 GB 9:747/1.19 GB 5:753/1.39 GB 3:707/1.01 GB 4:775/1.22 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 28 out of 28 Runtime filters: Not all filters arrived (arrived: [8], missing [2, 3]), waited for 40ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:28 PARQUET/Unknown(Skipped):9476 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 13.6 MiB (14223848) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 13.6 MiB (14223848) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 13.6 MiB (14223848) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 52 (52) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 3 (3) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 28 (28) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 20 (20) - PeakMemoryUsage: 11.3 MiB (11807828) - PerReadThreadRawHdfsThroughput: 2.6 GiB/s (2844789513) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 27ms (27999801) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 1,722,094 (1722094) - RowsReturned: 54,011 (54011) - RowsReturnedRate: 259670 per second (259670) - ScanRangesComplete: 9,504 (9504) - ScannerThreadsInvoluntaryContextSwitches: 5 (5) - ScannerThreadsTotalWallClockTime: 167ms (167998798) - MaterializeTupleTime(*): 84ms (84999391) - ScannerThreadsSysTime: 18ms (18793000) - ScannerThreadsUserTime: 108ms (108389000) - ScannerThreadsVoluntaryContextSwitches: 102 (102) - TotalRawHdfsReadTime(*): 4ms (4999965) - TotalReadThroughput: 0 B/s (0) - TotalTime: 207ms (207998511) Filter 2 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 1,722,094 (1722094) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 3 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 1,722,094 (1722094) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 8 (1.00 MB) - Files processed: 9,503 (9503) - Files rejected: 9,475 (9475) - Files total: 9,503 (9503) - InactiveTotalTime: 0ns (0) - RowGroups processed: 57 (57) - RowGroups rejected: 0 (0) - RowGroups total: 57 (57) - Rows processed: 1,627,487 (1627487) - Rows rejected: 0 (0) - Rows total: 1,722,094 (1722094) - Splits processed: 28 (28) - Splits rejected: 0 (0) - Splits total: 28 (28) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf45150000000b (host=ip-10-1-27-82.ec2.internal:22000) Hdfs split stats (:<# splits>/): 6:1192/1.30 GB 9:1006/1.10 GB 4:1114/1.51 GB 1:1034/1.58 GB 3:1027/1.36 GB 0:1107/1.41 GB 10:1172/1.41 GB 7:1199/1.32 GB 5:1084/1.41 GB 8:1087/1.65 GB 11:1239/1.43 GB 2:1261/2.02 GB Filter 8 arrival: 43s090ms - AverageThreadTokens: 1.97 (1.9651162790697674) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 8.4 MiB (8851972) - PeakReservation: 1.9 MiB (2031616) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 303.4 MiB (318098845) - RowsProduced: 18,140 (18140) - TotalNetworkReceiveTime: 41.80s (41798775267) - TotalNetworkSendTime: 38ms (38998856) - TotalStorageWaitTime: 287ms (287991559) - TotalThreadsInvoluntaryContextSwitches: 51 (51) - TotalThreadsTotalWallClockTime: 1.4m (84884512826) - TotalThreadsSysTime: 2ms (2881000) - TotalThreadsUserTime: 432ms (432868000) - TotalThreadsVoluntaryContextSwitches: 94 (94) - TotalTime: 43.51s (43512725048) Fragment Instance Lifecycle Timings - ExecTime: 339ms (339990038) - ExecTreeExecTime: 295ms (295991327) - InactiveTotalTime: 0ns (0) - OpenTime: 42.39s (42394757805) - ExecTreeOpenTime: 41.87s (41874773041) - PrepareTime: 777ms (777977205) - ExecTreePrepareTime: 514ms (514984911) - TotalTime: 0ns (0) DataStreamSender (dst_id=23) - BytesSent: 1.3 MiB (1332468) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 635.4 MiB/s (666253321) - OverallThroughput: 11.8 MiB/s (12338028) - PeakMemoryUsage: 30.7 KiB (31440) - RowsReturned: 18,140 (18140) - SerializeBatchTime: 2ms (2999913) - TotalTime: 107ms (107996837) - TransmitDataRPCTime: 1ms (1999942) - UncompressedRowBatchSize: 3.4 MiB (3591720) CodeGen - CodegenTime: 2ms (2999912) - CompileTime: 112ms (112996689) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 104 (104) - NumInstructions: 3,191 (3191) - OptimizationTime: 390ms (390988544) - PeakMemoryUsage: 1.6 MiB (1633792) - PrepareTime: 140ms (140995867) - TotalTime: 644ms (644981100) HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 1ms (1999942) - InactiveTotalTime: 0ns (0) - LocalTime: 1ms (1999942) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.3 MiB (2360576) - ProbeRows: 18,140 (18140) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 18,140 (18140) - RowsReturnedRate: 419 per second (419) - TotalTime: 43.20s (43200734190) Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=22) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 41.80s (41798775267) DataStreamReceiver - BytesReceived: 13 B (13) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 41.80s (41798775267) - InactiveTotalTime: 41.80s (41798775267) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=3) Hdfs split stats (:<# splits>/): 6:1192/1.30 GB 9:1006/1.10 GB 4:1114/1.51 GB 1:1034/1.58 GB 3:1027/1.36 GB 0:1107/1.41 GB 10:1172/1.41 GB 7:1199/1.32 GB 5:1084/1.41 GB 8:1087/1.65 GB 11:1239/1.43 GB 2:1261/2.02 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 6 out of 6 Runtime filters: Not all filters arrived (arrived: [8], missing [2, 3]), waited for 43ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:6 PARQUET/Unknown(Skipped):13516 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 4.2 MiB (4403098) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 4.2 MiB (4403098) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 4.2 MiB (4403098) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 15 (15) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 3 (3) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 6 (6) - NumRowGroups: 6 (6) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 3 (3) - PeakMemoryUsage: 5.1 MiB (5358634) - PerReadThreadRawHdfsThroughput: 19.6 MiB/s (20575827) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 309ms (309990917) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 556,383 (556383) - RowsReturned: 18,140 (18140) - RowsReturnedRate: 12939 per second (12939) - ScanRangesComplete: 13,522 (13522) - ScannerThreadsInvoluntaryContextSwitches: 5 (5) - ScannerThreadsTotalWallClockTime: 348ms (348989774) - MaterializeTupleTime(*): 42ms (42998739) - ScannerThreadsSysTime: 1ms (1905000) - ScannerThreadsUserTime: 43ms (43815000) - ScannerThreadsVoluntaryContextSwitches: 23 (23) - TotalRawHdfsReadTime(*): 213ms (213993730) - TotalReadThroughput: 0 B/s (0) - TotalTime: 1.40s (1401958922) Filter 2 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 556,383 (556383) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 3 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 556,383 (556383) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 8 (1.00 MB) - Files processed: 13,522 (13522) - Files rejected: 13,516 (13516) - Files total: 13,522 (13522) - InactiveTotalTime: 0ns (0) - RowGroups processed: 20 (20) - RowGroups rejected: 0 (0) - RowGroups total: 20 (20) - Rows processed: 522,687 (522687) - Rows rejected: 0 (0) - Rows total: 556,383 (556383) - Splits processed: 6 (6) - Splits rejected: 0 (0) - Splits total: 6 (6) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf45150000000a (host=ip-10-1-28-130.ec2.internal:22000) Hdfs split stats (:<# splits>/): 5:682/1.37 GB 3:728/1.12 GB 1:730/1.43 GB 8:687/1.53 GB 6:647/1.57 GB 9:739/1.71 GB 4:687/1.42 GB 0:689/1.51 GB 2:610/1.28 GB 11:628/1.62 GB 7:623/1.52 GB 10:619/1.43 GB Filter 8 arrival: 43s421ms - AverageThreadTokens: 1.99 (1.9883720930232558) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 3.1 MiB (3204570) - PeakReservation: 1.9 MiB (2031616) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 42.91s (42914631165) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 15 (15) - TotalThreadsTotalWallClockTime: 1.4m (86351257845) - TotalThreadsSysTime: 4ms (4027000) - TotalThreadsUserTime: 383ms (383016000) - TotalThreadsVoluntaryContextSwitches: 42 (42) - TotalTime: 43.47s (43472626370) Fragment Instance Lifecycle Timings - ExecTime: 3ms (3999966) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 43.34s (43341627496) - ExecTreeOpenTime: 42.96s (42962630753) - PrepareTime: 126ms (126998908) - ExecTreePrepareTime: 81ms (81999295) - TotalTime: 0ns (0) DataStreamSender (dst_id=23) - BytesSent: 0 B (0) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 30.7 KiB (31440) - RowsReturned: 0 (0) - SerializeBatchTime: 0ns (0) - TotalTime: 9ms (9999914) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 0 B (0) CodeGen - CodegenTime: 2ms (2999974) - CompileTime: 124ms (124998926) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 104 (104) - NumInstructions: 3,191 (3191) - OptimizationTime: 250ms (250997843) - PeakMemoryUsage: 1.6 MiB (1633792) - PrepareTime: 27ms (27999760) - TotalTime: 404ms (404996520) HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 999.99us (999992) - InactiveTotalTime: 0ns (0) - LocalTime: 999.99us (999992) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2074880) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.13s (43126629343) Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=22) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 42.91s (42914631165) DataStreamReceiver - BytesReceived: 13 B (13) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 42.91s (42913631174) - InactiveTotalTime: 42.91s (42914631165) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=3) Hdfs split stats (:<# splits>/): 5:682/1.37 GB 3:728/1.12 GB 1:730/1.43 GB 8:687/1.53 GB 6:647/1.57 GB 9:739/1.71 GB 4:687/1.42 GB 0:689/1.51 GB 2:610/1.28 GB 11:628/1.62 GB 7:623/1.52 GB 10:619/1.43 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 0 out of 0 Runtime filters: Not all filters arrived (arrived: [8], missing [2, 3]), waited for 41ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/Unknown(Skipped):8069 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 0 B (0) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 32.0 KiB (32768) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - ScanRangesComplete: 8,069 (8069) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 89ms (89999226) - MaterializeTupleTime(*): 0ns (0) - ScannerThreadsSysTime: 370.00us (370000) - ScannerThreadsUserTime: 0ns (0) - ScannerThreadsVoluntaryContextSwitches: 12 (12) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 212ms (212998168) Filter 2 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 3 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 8 (1.00 MB) - Files processed: 8,069 (8069) - Files rejected: 8,069 (8069) - Files total: 8,069 (8069) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf45150000000c (host=ip-10-1-16-98.ec2.internal:22000) Hdfs split stats (:<# splits>/): 5:708/1.57 GB 0:786/1.72 GB 1:738/1.48 GB 11:682/1.17 GB 3:690/1.34 GB 8:762/1.80 GB 2:739/1.28 GB 7:776/1.43 GB 6:742/1.27 GB 4:756/1.63 GB 10:736/1.60 GB 9:771/1.21 GB Filter 8 arrival: 43s413ms - AverageThreadTokens: 1.99 (1.9885057471264367) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 3.1 MiB (3237338) - PeakReservation: 1.9 MiB (2031616) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 310.3 MiB (325335426) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 43.07s (43069147179) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 1.4m (86466295477) - TotalThreadsSysTime: 4ms (4492000) - TotalThreadsUserTime: 278ms (278213000) - TotalThreadsVoluntaryContextSwitches: 25 (25) - TotalTime: 43.46s (43465148531) Fragment Instance Lifecycle Timings - ExecTime: 5ms (5000017) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 43.39s (43391148278) - ExecTreeOpenTime: 43.12s (43116147339) - PrepareTime: 68ms (68000233) - ExecTreePrepareTime: 43ms (43000147) - TotalTime: 0ns (0) DataStreamSender (dst_id=23) - BytesSent: 0 B (0) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 30.7 KiB (31440) - RowsReturned: 0 (0) - SerializeBatchTime: 0ns (0) - TotalTime: 4ms (4000014) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 0 B (0) CodeGen - CodegenTime: 2ms (2000006) - CompileTime: 88ms (88000300) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 104 (104) - NumInstructions: 3,191 (3191) - OptimizationTime: 186ms (186000636) - PeakMemoryUsage: 1.6 MiB (1633792) - PrepareTime: 15ms (15000051) - TotalTime: 290ms (290000991) HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 1ms (1000003) - InactiveTotalTime: 0ns (0) - LocalTime: 1ms (1000003) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2074880) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.20s (43202147633) Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=22) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.07s (43069147179) DataStreamReceiver - BytesReceived: 13 B (13) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 43.07s (43069147179) - InactiveTotalTime: 43.07s (43069147179) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=3) Hdfs split stats (:<# splits>/): 5:708/1.57 GB 0:786/1.72 GB 1:738/1.48 GB 11:682/1.17 GB 3:690/1.34 GB 8:762/1.80 GB 2:739/1.28 GB 7:776/1.43 GB 6:742/1.27 GB 4:756/1.63 GB 10:736/1.60 GB 9:771/1.21 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 0 out of 0 Runtime filters: Not all filters arrived (arrived: [8], missing [2, 3]), waited for 40ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/Unknown(Skipped):8886 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 0 B (0) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 0 (0) - PeakMemoryUsage: 64.0 KiB (65536) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - ScanRangesComplete: 8,886 (8886) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0ns (0) - MaterializeTupleTime(*): 0ns (0) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 0ns (0) - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 133ms (133000455) Filter 2 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 3 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 8 (1.00 MB) - Files processed: 8,886 (8886) - Files rejected: 8,886 (8886) - Files total: 8,886 (8886) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf45150000000d (host=ip-10-1-18-116.ec2.internal:22000) Hdfs split stats (:<# splits>/): 8:950/1.59 GB 2:1005/1.68 GB 9:1011/1.81 GB 11:1065/1.25 GB 0:1021/1.29 GB 10:979/1.75 GB 7:999/1.34 GB 3:972/1.46 GB 4:1002/1.44 GB 6:993/1.22 GB 1:925/1.43 GB 5:1066/1.23 GB Filter 8 arrival: 43s368ms - AverageThreadTokens: 1.99 (1.9885057471264367) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 3.6 MiB (3750421) - PeakReservation: 1.9 MiB (2031616) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 305.3 MiB (320138763) - RowsProduced: 575 (575) - TotalNetworkReceiveTime: 42.89s (42894053949) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 20ms (20000492) - TotalThreadsInvoluntaryContextSwitches: 20 (20) - TotalThreadsTotalWallClockTime: 1.4m (86249119227) - TotalThreadsSysTime: 7ms (7074000) - TotalThreadsUserTime: 360ms (360152000) - TotalThreadsVoluntaryContextSwitches: 74 (74) - TotalTime: 43.43s (43433067193) Fragment Instance Lifecycle Timings - ExecTime: 8ms (8000196) - ExecTreeExecTime: 6ms (6000147) - InactiveTotalTime: 0ns (0) - OpenTime: 43.32s (43323064491) - ExecTreeOpenTime: 42.95s (42951055350) - PrepareTime: 102ms (102002506) - ExecTreePrepareTime: 63ms (63001548) - TotalTime: 0ns (0) DataStreamSender (dst_id=23) - BytesSent: 39.2 KiB (40146) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 12.8 MiB/s (13381669) - PeakMemoryUsage: 30.7 KiB (31440) - RowsReturned: 575 (575) - SerializeBatchTime: 0ns (0) - TotalTime: 3ms (3000074) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 111.2 KiB (113850) CodeGen - CodegenTime: 2ms (2000049) - CompileTime: 105ms (105002580) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 104 (104) - NumInstructions: 3,191 (3191) - OptimizationTime: 265ms (265006512) - PeakMemoryUsage: 1.6 MiB (1633792) - PrepareTime: 27ms (27000664) - TotalTime: 397ms (397009756) HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 2ms (2000049) - InactiveTotalTime: 0ns (0) - LocalTime: 2ms (2000049) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2147584) - ProbeRows: 575 (575) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 575 (575) - RowsReturnedRate: 13 per second (13) - TotalTime: 43.08s (43083058593) Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=22) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 42.89s (42894053949) DataStreamReceiver - BytesReceived: 13 B (13) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 42.89s (42894053949) - InactiveTotalTime: 42.89s (42894053949) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=3) Hdfs split stats (:<# splits>/): 8:950/1.59 GB 2:1005/1.68 GB 9:1011/1.81 GB 11:1065/1.25 GB 0:1021/1.29 GB 10:979/1.75 GB 7:999/1.34 GB 3:972/1.46 GB 4:1002/1.44 GB 6:993/1.22 GB 1:925/1.43 GB 5:1066/1.23 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 11 out of 11 Runtime filters: Not all filters arrived (arrived: [8], missing [2, 3]), waited for 40ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:11 PARQUET/Unknown(Skipped):11977 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 689.7 KiB (706293) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 689.7 KiB (706293) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 689.7 KiB (706293) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 17 (17) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 3 (3) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 9 (9) - NumRowGroups: 11 (11) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 9 (9) - PeakMemoryUsage: 482.1 KiB (493627) - PerReadThreadRawHdfsThroughput: 673.6 MiB/s (706276049) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 11ms (11000271) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 9,344 (9344) - RowsReturned: 575 (575) - RowsReturnedRate: 3042 per second (3042) - ScanRangesComplete: 11,988 (11988) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 22ms (22000542) - MaterializeTupleTime(*): 0ns (0) - ScannerThreadsSysTime: 2ms (2614000) - ScannerThreadsUserTime: 1ms (1949000) - ScannerThreadsVoluntaryContextSwitches: 41 (41) - TotalRawHdfsReadTime(*): 1ms (1000024) - TotalReadThroughput: 0 B/s (0) - TotalTime: 189ms (189004644) Filter 2 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 9,344 (9344) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 3 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 9,344 (9344) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 8 (1.00 MB) - Files processed: 11,988 (11988) - Files rejected: 11,977 (11977) - Files total: 11,988 (11988) - InactiveTotalTime: 0ns (0) - RowGroups processed: 2 (2) - RowGroups rejected: 0 (0) - RowGroups total: 2 (2) - Rows processed: 9,344 (9344) - Rows rejected: 0 (0) - Rows total: 9,344 (9344) - Splits processed: 11 (11) - Splits rejected: 0 (0) - Splits total: 11 (11) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Averaged Fragment F03 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:43s429ms max:43s429ms mean: 43s429ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 201.6 KiB (206416) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 43.37s (43370627247) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 43.37s (43374627212) - TotalThreadsSysTime: 52.00us (52000) - TotalThreadsUserTime: 520.00us (520000) - TotalThreadsVoluntaryContextSwitches: 15 (15) - TotalTime: 43.42s (43421626808) Fragment Instance Lifecycle Timings - ExecTime: 3ms (3999965) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 43.37s (43370627247) - ExecTreeOpenTime: 43.37s (43370627247) - PrepareTime: 46ms (46999596) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=22) - BytesSent: 78 B (78) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 4.0 KiB/s (4105) - PeakMemoryUsage: 153.6 KiB (157248) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 18ms (18999837) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 78 B (78) CodeGen - CodegenTime: 0ns (0) - CompileTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - PrepareTime: 24ms (24999785) - TotalTime: 26ms (26999768) AGGREGATION_NODE (id=21) - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 40.0 KiB (40960) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 43.37s (43370627247) EXCHANGE_NODE (id=20) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 6 (6) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.37s (43370627247) DataStreamReceiver - BytesReceived: 78 B (78) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 10.16s (10157912697) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 4.0 KiB (4112) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Fragment F03 Instance 68440bc099e1b843:8baf451500000007 (host=ip-10-1-28-130.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 201.6 KiB (206416) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 43.37s (43370627247) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 43.37s (43374627212) - TotalThreadsSysTime: 52.00us (52000) - TotalThreadsUserTime: 520.00us (520000) - TotalThreadsVoluntaryContextSwitches: 15 (15) - TotalTime: 43.42s (43421626808) Fragment Instance Lifecycle Timings - ExecTime: 3ms (3999965) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 43.37s (43370627247) - ExecTreeOpenTime: 43.37s (43370627247) - PrepareTime: 46ms (46999596) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=22) - BytesSent: 78 B (78) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 4.0 KiB/s (4105) - PeakMemoryUsage: 153.6 KiB (157248) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 18ms (18999837) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 78 B (78) CodeGen - CodegenTime: 0ns (0) - CompileTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - PrepareTime: 24ms (24999785) - TotalTime: 26ms (26999768) AGGREGATION_NODE (id=21) - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 40.0 KiB (40960) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 43.37s (43370627247) EXCHANGE_NODE (id=20) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 6 (6) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.37s (43370627247) DataStreamReceiver - BytesReceived: 78 B (78) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 10.16s (10157912697) - InactiveTotalTime: 43.37s (43370627247) - PeakMemoryUsage: 4.0 KiB (4112) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Averaged Fragment F02 split sizes: min: 15.06 GB, max: 17.50 GB, avg: 16.91 GB, stddev: 938.99 MB completion times: min:10s204ms max:43s241ms mean: 18s061ms stddev:11s402ms execution rates: min:414.41 MB/sec max:1.71 GB/sec mean:1.17 GB/sec stddev:412.59 MB/sec num instances: 6 - AverageThreadTokens: 2.95 (2.946927050390048) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.8 MiB (1881365) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 303.8 MiB (318598954) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 833.33us (833332) - TotalStorageWaitTime: 21.55s (21551819386) - TotalThreadsInvoluntaryContextSwitches: 271 (271) - TotalThreadsTotalWallClockTime: 49.65s (49654793736) - TotalThreadsSysTime: 2.04s (2043821166) - TotalThreadsUserTime: 13.09s (13087358333) - TotalThreadsVoluntaryContextSwitches: 383,162 (383162) - TotalTime: 18.02s (18025826850) Fragment Instance Lifecycle Timings - ExecTime: 1ms (1166659) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 17.80s (17798831224) - ExecTreeOpenTime: 17.77s (17768831432) - PrepareTime: 225ms (225828966) - ExecTreePrepareTime: 176ms (176663122) - TotalTime: 0ns (0) DataStreamSender (dst_id=20) - BytesSent: 13 B (13) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 5.7 KiB/s (5837) - PeakMemoryUsage: 25.6 KiB (26208) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 7ms (7666487) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13 B (13) CodeGen - CodegenTime: 1ms (1333324) - CompileTime: 11ms (11499868) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 16 (16) - NumInstructions: 232 (232) - OptimizationTime: 16ms (16666594) - PeakMemoryUsage: 116.0 KiB (118784) - PrepareTime: 37ms (37166141) - TotalTime: 66ms (66332603) AGGREGATION_NODE (id=5) - BuildTime: 5.69s (5686483169) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.0 KiB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 17.94s (17945494554) HDFS_SCAN_NODE (id=4) - AverageHdfsReadThreadConcurrency: 0.04 (0.03734939759036144) - AverageScannerThreadConcurrency: 1.96 (1.9630187416331992) - BytesRead: 542.9 MiB (569282723) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 542.9 MiB (569282723) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 542.9 MiB (569282723) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 10,252 (10252) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 1.7 MiB (1830229) - PerReadThreadRawHdfsThroughput: 1.4 GiB/s (1499377677) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 10.20s (10204185518) - RowBatchQueuePutWaitTime: 7.56s (7562484812) - RowsRead: 1,686,971,481,464 (1686971481464) - RowsReturned: 1,358,862,882 (1358862882) - RowsReturnedRate: 176665071 per second (176665071) - ScanRangesComplete: 10,252 (10252) - ScannerThreadsInvoluntaryContextSwitches: 13 (13) - ScannerThreadsTotalWallClockTime: 31.85s (31854795852) - MaterializeTupleTime(*): 883ms (883997730) - ScannerThreadsSysTime: 804ms (804101000) - ScannerThreadsUserTime: 6.70s (6699698166) - ScannerThreadsVoluntaryContextSwitches: 374,429 (374429) - TotalRawHdfsReadTime(*): 386ms (386998615) - TotalReadThroughput: 36.3 MiB/s (38079828) - TotalTime: 12.17s (12171175280) Fragment F02 Instance 68440bc099e1b843:8baf451500000004 (host=ip-10-1-27-82.ec2.internal:22000) Hdfs split stats (:<# splits>/): 6:1192/1.30 GB 9:1006/1.10 GB 4:1114/1.51 GB 1:1034/1.58 GB 3:1027/1.36 GB 0:1107/1.41 GB 10:1172/1.41 GB 7:1199/1.32 GB 5:1084/1.41 GB 8:1087/1.65 GB 11:1239/1.43 GB 2:1261/2.02 GB - AverageThreadTokens: 2.33 (2.3294117647058825) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.8 MiB (1849280) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 303.4 MiB (318098845) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 999.97us (999971) - TotalStorageWaitTime: 47.59s (47589605575) - TotalThreadsInvoluntaryContextSwitches: 461 (461) - TotalThreadsTotalWallClockTime: 1.7m (99490084874) - TotalThreadsSysTime: 1.42s (1424384000) - TotalThreadsUserTime: 12.82s (12823777000) - TotalThreadsVoluntaryContextSwitches: 234,527 (234527) - TotalTime: 43.07s (43069738027) Fragment Instance Lifecycle Timings - ExecTime: 1ms (1999941) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 42.14s (42144765130) - ExecTreeOpenTime: 42.10s (42097766507) - PrepareTime: 922ms (922972956) - ExecTreePrepareTime: 748ms (748978054) - TotalTime: 0ns (0) DataStreamSender (dst_id=20) - BytesSent: 13 B (13) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 361 B/s (361) - PeakMemoryUsage: 25.6 KiB (26208) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 35ms (35998945) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13 B (13) CodeGen - CodegenTime: 1ms (1999942) - CompileTime: 28ms (28999150) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 16 (16) - NumInstructions: 232 (232) - OptimizationTime: 16ms (16999502) - PeakMemoryUsage: 116.0 KiB (118784) - PrepareTime: 113ms (113996661) - TotalTime: 160ms (160995283) AGGREGATION_NODE (id=5) ExecOption: Codegen Enabled - BuildTime: 6.34s (6342814171) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.0 KiB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 42.85s (42846744561) HDFS_SCAN_NODE (id=4) Hdfs split stats (:<# splits>/): 6:1192/1.30 GB 9:1006/1.10 GB 4:1114/1.51 GB 1:1034/1.58 GB 3:1027/1.36 GB 0:1107/1.41 GB 10:1172/1.41 GB 7:1199/1.32 GB 5:1084/1.41 GB 8:1087/1.65 GB 11:1239/1.43 GB 2:1261/2.02 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 13522 out of 13522 Hdfs Read Thread Concurrency Bucket: 0:96.39% 1:3.614% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:13522 - AverageHdfsReadThreadConcurrency: 0.02 (0.024096385542168676) - AverageScannerThreadConcurrency: 1.36 (1.3614457831325302) - BytesRead: 603.4 MiB (632702451) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 603.4 MiB (632702451) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 603.4 MiB (632702451) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 13,522 (13522) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 3 (3) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 1.7 MiB (1798144) - PerReadThreadRawHdfsThroughput: 1.1 GiB/s (1209791539) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 34.07s (34066001831) - RowBatchQueuePutWaitTime: 6.50s (6498809580) - RowsRead: 1,678,564,198,007 (1678564198007) - RowsReturned: 1,401,658,999 (1401658999) - RowsReturnedRate: 37931969 per second (37931969) - ScanRangesComplete: 13,522 (13522) - ScannerThreadsInvoluntaryContextSwitches: 49 (49) - ScannerThreadsTotalWallClockTime: 57.34s (57343319803) - MaterializeTupleTime(*): 924ms (924972877) - ScannerThreadsSysTime: 601ms (601454000) - ScannerThreadsUserTime: 5.67s (5668581000) - ScannerThreadsVoluntaryContextSwitches: 221,138 (221138) - TotalRawHdfsReadTime(*): 522ms (522984688) - TotalReadThroughput: 14.0 MiB/s (14714862) - TotalTime: 36.95s (36951917259) Instance 68440bc099e1b843:8baf451500000001 (host=ip-10-1-20-86.ec2.internal:22000) Hdfs split stats (:<# splits>/): 0:760/1.06 GB 7:835/1.59 GB 2:822/1.65 GB 8:808/1.40 GB 1:799/1.41 GB 6:827/1.55 GB 10:850/1.47 GB 11:821/1.48 GB 9:747/1.19 GB 5:753/1.39 GB 3:707/1.01 GB 4:775/1.22 GB - AverageThreadTokens: 2.94 (2.935483870967742) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.8 MiB (1861568) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 312.0 MiB (327162809) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 20.44s (20435853585) - TotalThreadsInvoluntaryContextSwitches: 259 (259) - TotalThreadsTotalWallClockTime: 45.50s (45496674007) - TotalThreadsSysTime: 2.19s (2189562000) - TotalThreadsUserTime: 12.80s (12799424000) - TotalThreadsVoluntaryContextSwitches: 402,477 (402477) - TotalTime: 15.30s (15295890401) Fragment Instance Lifecycle Timings - ExecTime: 999.99us (999992) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 15.22s (15216890968) - ExecTreeOpenTime: 15.19s (15188891169) - PrepareTime: 77ms (77999441) - ExecTreePrepareTime: 47ms (47999656) - TotalTime: 0ns (0) DataStreamSender (dst_id=20) - BytesSent: 13 B (13) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 2.1 KiB/s (2166) - PeakMemoryUsage: 25.6 KiB (26208) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 5ms (5999957) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13 B (13) CodeGen - CodegenTime: 999.99us (999993) - CompileTime: 7ms (7999943) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 16 (16) - NumInstructions: 232 (232) - OptimizationTime: 17ms (17999871) - PeakMemoryUsage: 116.0 KiB (118784) - PrepareTime: 23ms (23999827) - TotalTime: 49ms (49999641) AGGREGATION_NODE (id=5) ExecOption: Codegen Enabled - BuildTime: 5.46s (5459960872) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.0 KiB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 15.24s (15236890825) HDFS_SCAN_NODE (id=4) Hdfs split stats (:<# splits>/): 0:760/1.06 GB 7:835/1.59 GB 2:822/1.65 GB 8:808/1.40 GB 1:799/1.41 GB 6:827/1.55 GB 10:850/1.47 GB 11:821/1.48 GB 9:747/1.19 GB 5:753/1.39 GB 3:707/1.01 GB 4:775/1.22 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 9503 out of 9503 Hdfs Read Thread Concurrency Bucket: 0:93.33% 1:6.667% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:9504 - AverageHdfsReadThreadConcurrency: 0.07 (0.06666666666666667) - AverageScannerThreadConcurrency: 2.00 (2.0) - BytesRead: 508.9 MiB (533590682) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 508.9 MiB (533590682) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 508.9 MiB (533590682) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 9,503 (9503) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 1.7 MiB (1810432) - PerReadThreadRawHdfsThroughput: 1.5 GiB/s (1626812549) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 7.86s (7860943677) - RowBatchQueuePutWaitTime: 7.23s (7228948195) - RowsRead: 1,742,767,454,342 (1742767454342) - RowsReturned: 1,313,723,526 (1313723526) - RowsReturnedRate: 137376653 per second (137376653) - ScanRangesComplete: 9,504 (9504) - ScannerThreadsInvoluntaryContextSwitches: 7 (7) - ScannerThreadsTotalWallClockTime: 30.28s (30278783047) - MaterializeTupleTime(*): 835ms (835994008) - ScannerThreadsSysTime: 876ms (876299000) - ScannerThreadsUserTime: 6.69s (6690608000) - ScannerThreadsVoluntaryContextSwitches: 395,037 (395037) - TotalRawHdfsReadTime(*): 327ms (327997643) - TotalReadThroughput: 32.2 MiB/s (33748404) - TotalTime: 9.56s (9562931477) Instance 68440bc099e1b843:8baf451500000006 (host=ip-10-1-18-116.ec2.internal:22000) Hdfs split stats (:<# splits>/): 8:950/1.59 GB 2:1005/1.68 GB 9:1011/1.81 GB 11:1065/1.25 GB 0:1021/1.29 GB 10:979/1.75 GB 7:999/1.34 GB 3:972/1.46 GB 4:1002/1.44 GB 6:993/1.22 GB 1:925/1.43 GB 5:1066/1.23 GB - AverageThreadTokens: 3.00 (3.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.8 MiB (1882048) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 305.3 MiB (320138763) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 1ms (1000024) - TotalStorageWaitTime: 20.13s (20130494633) - TotalThreadsInvoluntaryContextSwitches: 227 (227) - TotalThreadsTotalWallClockTime: 45.27s (45268112283) - TotalThreadsSysTime: 1.89s (1893701000) - TotalThreadsUserTime: 13.42s (13419401000) - TotalThreadsVoluntaryContextSwitches: 370,894 (370894) - TotalTime: 15.20s (15196373390) Fragment Instance Lifecycle Timings - ExecTime: 1ms (1000024) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 15.12s (15115371400) - ExecTreeOpenTime: 15.10s (15097370958) - PrepareTime: 80ms (80001966) - ExecTreePrepareTime: 62ms (62001524) - TotalTime: 0ns (0) DataStreamSender (dst_id=20) - BytesSent: 13 B (13) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 12.7 KiB/s (12999) - PeakMemoryUsage: 25.6 KiB (26208) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 1ms (1000024) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13 B (13) CodeGen - CodegenTime: 1ms (1000025) - CompileTime: 6ms (6000147) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 16 (16) - NumInstructions: 232 (232) - OptimizationTime: 11ms (11000270) - PeakMemoryUsage: 116.0 KiB (118784) - PrepareTime: 17ms (17000419) - TotalTime: 35ms (35000860) AGGREGATION_NODE (id=5) ExecOption: Codegen Enabled - BuildTime: 5.79s (5793142332) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.0 KiB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 15.16s (15159372482) HDFS_SCAN_NODE (id=4) Hdfs split stats (:<# splits>/): 8:950/1.59 GB 2:1005/1.68 GB 9:1011/1.81 GB 11:1065/1.25 GB 0:1021/1.29 GB 10:979/1.75 GB 7:999/1.34 GB 3:972/1.46 GB 4:1002/1.44 GB 6:993/1.22 GB 1:925/1.43 GB 5:1066/1.23 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 11988 out of 11988 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:11988 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 2.00 (2.0) - BytesRead: 585.3 MiB (613760298) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 585.3 MiB (613760298) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 585.3 MiB (613760298) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 11,988 (11988) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 1.7 MiB (1830912) - PerReadThreadRawHdfsThroughput: 1.3 GiB/s (1437342493) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 7.46s (7465183437) - RowBatchQueuePutWaitTime: 7.08s (7081173985) - RowsRead: 1,759,448,578,513 (1759448578513) - RowsReturned: 1,404,322,257 (1404322257) - RowsReturnedRate: 153373498 per second (153373498) - ScanRangesComplete: 11,988 (11988) - ScannerThreadsInvoluntaryContextSwitches: 10 (10) - ScannerThreadsTotalWallClockTime: 30.15s (30151740859) - MaterializeTupleTime(*): 926ms (926022753) - ScannerThreadsSysTime: 722ms (722311000) - ScannerThreadsUserTime: 6.83s (6826338000) - ScannerThreadsVoluntaryContextSwitches: 359,961 (359961) - TotalRawHdfsReadTime(*): 427ms (427010473) - TotalReadThroughput: 37.4 MiB/s (39244093) - TotalTime: 9.16s (9156224979) Instance 68440bc099e1b843:8baf451500000002 (host=ip-10-1-21-97.ec2.internal:22000) Hdfs split stats (:<# splits>/): 7:838/1.15 GB 2:876/1.46 GB 5:798/1.29 GB 6:772/1.33 GB 11:836/1.50 GB 10:837/1.65 GB 1:769/1.21 GB 4:739/1.10 GB 8:781/1003.47 MB 9:778/1.35 GB 3:748/1.10 GB 0:774/970.37 MB - AverageThreadTokens: 3.42 (3.4166666666666665) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.8 MiB (1890240) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 301.4 MiB (316058244) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 1ms (1000002) - TotalStorageWaitTime: 18.47s (18472041641) - TotalThreadsInvoluntaryContextSwitches: 368 (368) - TotalThreadsTotalWallClockTime: 41.69s (41687094064) - TotalThreadsSysTime: 1.86s (1859878000) - TotalThreadsUserTime: 12.20s (12201472000) - TotalThreadsVoluntaryContextSwitches: 342,953 (342953) - TotalTime: 12.36s (12358027885) Fragment Instance Lifecycle Timings - ExecTime: 1ms (1000002) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 12.26s (12263027671) - ExecTreeOpenTime: 12.23s (12234027605) - PrepareTime: 94ms (94000212) - ExecTreePrepareTime: 71ms (71000160) - TotalTime: 0ns (0) DataStreamSender (dst_id=20) - BytesSent: 13 B (13) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 25.6 KiB (26208) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 0ns (0) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13 B (13) CodeGen - CodegenTime: 1ms (1000002) - CompileTime: 10ms (10000023) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 16 (16) - NumInstructions: 232 (232) - OptimizationTime: 17ms (17000038) - PeakMemoryUsage: 116.0 KiB (118784) - PrepareTime: 23ms (23000053) - TotalTime: 51ms (51000116) AGGREGATION_NODE (id=5) ExecOption: Codegen Enabled - BuildTime: 5.20s (5195011730) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.0 KiB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 12.30s (12305027765) HDFS_SCAN_NODE (id=4) Hdfs split stats (:<# splits>/): 7:838/1.15 GB 2:876/1.46 GB 5:798/1.29 GB 6:772/1.33 GB 11:836/1.50 GB 10:837/1.65 GB 1:769/1.21 GB 4:739/1.10 GB 8:781/1003.47 MB 9:778/1.35 GB 3:748/1.10 GB 0:774/970.37 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 9546 out of 9546 Hdfs Read Thread Concurrency Bucket: 0:95.83% 1:4.167% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% File Formats: PARQUET/NONE:9546 - AverageHdfsReadThreadConcurrency: 0.04 (0.041666666666666664) - AverageScannerThreadConcurrency: 2.42 (2.4166666666666665) - BytesRead: 498.0 MiB (522206654) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 498.0 MiB (522206654) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 498.0 MiB (522206654) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 9,546 (9546) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 3 (3) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 1.8 MiB (1839104) - PerReadThreadRawHdfsThroughput: 1.3 GiB/s (1419036635) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 5.27s (5272011854) - RowBatchQueuePutWaitTime: 8.26s (8264018670) - RowsRead: 1,401,382,381,206 (1401382381206) - RowsReturned: 1,213,141,654 (1213141654) - RowsReturnedRate: 175359774 per second (175359774) - ScanRangesComplete: 9,546 (9546) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 29.42s (29423066391) - MaterializeTupleTime(*): 852ms (852001936) - ScannerThreadsSysTime: 652ms (652707000) - ScannerThreadsUserTime: 6.39s (6388814000) - ScannerThreadsVoluntaryContextSwitches: 336,017 (336017) - TotalRawHdfsReadTime(*): 368ms (368000826) - TotalReadThroughput: 39.9 MiB/s (41810953) - TotalTime: 6.92s (6918015587) Instance 68440bc099e1b843:8baf451500000003 (host=ip-10-1-28-130.ec2.internal:22000) Hdfs split stats (:<# splits>/): 5:682/1.37 GB 3:728/1.12 GB 1:730/1.43 GB 8:687/1.53 GB 6:647/1.57 GB 9:739/1.71 GB 4:687/1.42 GB 0:689/1.51 GB 2:610/1.28 GB 11:628/1.62 GB 7:623/1.52 GB 10:619/1.43 GB - AverageThreadTokens: 3.00 (3.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.8 MiB (1877952) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 999.99us (999992) - TotalStorageWaitTime: 13.04s (13039887898) - TotalThreadsInvoluntaryContextSwitches: 242 (242) - TotalThreadsTotalWallClockTime: 35.66s (35656693544) - TotalThreadsSysTime: 2.08s (2082436000) - TotalThreadsUserTime: 13.54s (13537343000) - TotalThreadsVoluntaryContextSwitches: 429,771 (429771) - TotalTime: 12.04s (12036896548) Fragment Instance Lifecycle Timings - ExecTime: 999.99us (999992) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 11.92s (11919897553) - ExecTreeOpenTime: 11.89s (11888897819) - PrepareTime: 115ms (115999003) - ExecTreePrepareTime: 91ms (91999209) - TotalTime: 0ns (0) DataStreamSender (dst_id=20) - BytesSent: 13 B (13) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 12.7 KiB/s (13000) - PeakMemoryUsage: 25.6 KiB (26208) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 999.99us (999991) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13 B (13) CodeGen - CodegenTime: 1ms (1999982) - CompileTime: 8ms (8999923) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 16 (16) - NumInstructions: 232 (232) - OptimizationTime: 19ms (19999828) - PeakMemoryUsage: 116.0 KiB (118784) - PrepareTime: 21ms (21999811) - TotalTime: 51ms (51999554) AGGREGATION_NODE (id=5) ExecOption: Codegen Enabled - BuildTime: 5.73s (5730950740) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.0 KiB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 11.98s (11980897028) HDFS_SCAN_NODE (id=4) Hdfs split stats (:<# splits>/): 5:682/1.37 GB 3:728/1.12 GB 1:730/1.43 GB 8:687/1.53 GB 6:647/1.57 GB 9:739/1.71 GB 4:687/1.42 GB 0:689/1.51 GB 2:610/1.28 GB 11:628/1.62 GB 7:623/1.52 GB 10:619/1.43 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 8068 out of 8068 Hdfs Read Thread Concurrency Bucket: 0:95.83% 1:4.167% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:8069 - AverageHdfsReadThreadConcurrency: 0.04 (0.041666666666666664) - AverageScannerThreadConcurrency: 2.00 (2.0) - BytesRead: 521.6 MiB (546894230) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 521.6 MiB (546894230) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 521.6 MiB (546894230) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 8,068 (8068) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 1.7 MiB (1826816) - PerReadThreadRawHdfsThroughput: 1.4 GiB/s (1549288807) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 4.17s (4171964145) - RowBatchQueuePutWaitTime: 8.09s (8093930469) - RowsRead: 1,764,202,602,788 (1764202602788) - RowsReturned: 1,414,115,620 (1414115620) - RowsReturnedRate: 231862235 per second (231862235) - ScanRangesComplete: 8,069 (8069) - ScannerThreadsInvoluntaryContextSwitches: 10 (10) - ScannerThreadsTotalWallClockTime: 23.74s (23735795999) - MaterializeTupleTime(*): 935ms (935991965) - ScannerThreadsSysTime: 720ms (720397000) - ScannerThreadsUserTime: 7.12s (7122584000) - ScannerThreadsVoluntaryContextSwitches: 422,990 (422990) - TotalRawHdfsReadTime(*): 352ms (352996954) - TotalReadThroughput: 43.2 MiB/s (45346851) - TotalTime: 6.10s (6098947587) Instance 68440bc099e1b843:8baf451500000005 (host=ip-10-1-16-98.ec2.internal:22000) Hdfs split stats (:<# splits>/): 5:708/1.57 GB 0:786/1.72 GB 1:738/1.48 GB 11:682/1.17 GB 3:690/1.34 GB 8:762/1.80 GB 2:739/1.28 GB 7:776/1.43 GB 6:742/1.27 GB 4:756/1.63 GB 10:736/1.60 GB 9:771/1.21 GB - AverageThreadTokens: 3.00 (3.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.8 MiB (1927104) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 310.3 MiB (325335426) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 1ms (1000003) - TotalStorageWaitTime: 9.64s (9643032988) - TotalThreadsInvoluntaryContextSwitches: 69 (69) - TotalThreadsTotalWallClockTime: 30.33s (30330103645) - TotalThreadsSysTime: 2.81s (2812966000) - TotalThreadsUserTime: 13.74s (13742733000) - TotalThreadsVoluntaryContextSwitches: 518,355 (518355) - TotalTime: 10.20s (10198034849) Fragment Instance Lifecycle Timings - ExecTime: 1ms (1000003) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 10.13s (10133034627) - ExecTreeOpenTime: 10.11s (10106034535) - PrepareTime: 64ms (64000219) - ExecTreePrepareTime: 38ms (38000130) - TotalTime: 0ns (0) DataStreamSender (dst_id=20) - BytesSent: 13 B (13) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 6.3 KiB/s (6499) - PeakMemoryUsage: 25.6 KiB (26208) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 2ms (2000006) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13 B (13) CodeGen - CodegenTime: 1ms (1000004) - CompileTime: 7ms (7000024) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 16 (16) - NumInstructions: 232 (232) - OptimizationTime: 17ms (17000058) - PeakMemoryUsage: 116.0 KiB (118784) - PrepareTime: 23ms (23000079) - TotalTime: 49ms (49000167) AGGREGATION_NODE (id=5) ExecOption: Codegen Enabled - BuildTime: 5.60s (5597019171) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.0 KiB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 10.14s (10144034665) HDFS_SCAN_NODE (id=4) Hdfs split stats (:<# splits>/): 5:708/1.57 GB 0:786/1.72 GB 1:738/1.48 GB 11:682/1.17 GB 3:690/1.34 GB 8:762/1.80 GB 2:739/1.28 GB 7:776/1.43 GB 6:742/1.27 GB 4:756/1.63 GB 10:736/1.60 GB 9:771/1.21 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 8886 out of 8886 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:8886 - AverageHdfsReadThreadConcurrency: 0.05 (0.05) - AverageScannerThreadConcurrency: 2.00 (2.0) - BytesRead: 540.3 MiB (566542025) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 540.3 MiB (566542025) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 540.3 MiB (566542025) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 8,886 (8886) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 3 (3) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 1.8 MiB (1875968) - PerReadThreadRawHdfsThroughput: 1.6 GiB/s (1753994044) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 2.39s (2389008167) - RowBatchQueuePutWaitTime: 8.21s (8208027974) - RowsRead: 1,775,463,673,929 (1775463673929) - RowsReturned: 1,406,215,241 (1406215241) - RowsReturnedRate: 324086298 per second (324086298) - ScanRangesComplete: 8,886 (8886) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 20.20s (20196069015) - MaterializeTupleTime(*): 829ms (829002844) - ScannerThreadsSysTime: 1.25s (1251438000) - ScannerThreadsUserTime: 7.50s (7501264000) - ScannerThreadsVoluntaryContextSwitches: 511,434 (511434) - TotalRawHdfsReadTime(*): 323ms (323001111) - TotalReadThroughput: 51.1 MiB/s (53613809) - TotalTime: 4.34s (4339014791) Averaged Fragment F09 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:44s976ms max:45s188ms mean: 45s115ms stddev:67.617ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 6 - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 8.6 MiB (9012535) - PeakReservation: 8.5 MiB (8912896) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 307.6 MiB (322494863) - RowsProduced: 369 (369) - TotalNetworkReceiveTime: 44.85s (44846226676) - TotalNetworkSendTime: 26ms (26499930) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 44.97s (44974558895) - TotalThreadsSysTime: 1ms (1220666) - TotalThreadsUserTime: 68ms (68961333) - TotalThreadsVoluntaryContextSwitches: 50 (50) - TotalTime: 45.08s (45078390035) Fragment Instance Lifecycle Timings - ExecTime: 32ms (32499917) - ExecTreeExecTime: 833.33us (833328) - InactiveTotalTime: 0ns (0) - OpenTime: 44.94s (44942058978) - ExecTreeOpenTime: 44.85s (44848060025) - PrepareTime: 103ms (103831140) - ExecTreePrepareTime: 166.67us (166667) - TotalTime: 0ns (0) DataStreamSender (dst_id=32) - BytesSent: 33.3 KiB (34101) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 16.1 MiB/s (16834214) - OverallThroughput: 7.7 MiB/s (8059906) - PeakMemoryUsage: 36.6 KiB (37440) - RowsReturned: 369 (369) - SerializeBatchTime: 166.67us (166670) - TotalTime: 49ms (49332031) - TransmitDataRPCTime: 999.99us (999992) - UncompressedRowBatchSize: 62.7 KiB (64206) CodeGen - CodegenTime: 5ms (5499894) - CompileTime: 36ms (36832734) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 35 (35) - NumInstructions: 654 (654) - OptimizationTime: 53ms (53832934) - PeakMemoryUsage: 327.0 KiB (334848) - PrepareTime: 51ms (51999115) - TotalTime: 143ms (143664781) AGGREGATION_NODE (id=31) - BuildTime: 833.35us (833353) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 8.5 MiB (8964356) - RowsRepartitioned: 0 (0) - RowsReturned: 369 (369) - RowsReturnedRate: 7 per second (7) - SpilledPartitions: 0 (0) - TotalTime: 44.85s (44849060021) EXCHANGE_NODE (id=30) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 481 (481) - RowsReturnedRate: 10 per second (10) - TotalTime: 44.85s (44846393341) DataStreamReceiver - BytesReceived: 6.1 KiB (6250) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 44.69s (44689227090) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 6.9 KiB (7028) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 833.35us (833353) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.5 MiB (8912896) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment F09 Instance 68440bc099e1b843:8baf451500000022 (host=ip-10-1-20-86.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 8.6 MiB (9012497) - PeakReservation: 8.5 MiB (8912896) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 312.0 MiB (327162809) - RowsProduced: 373 (373) - TotalNetworkReceiveTime: 45.03s (45033677324) - TotalNetworkSendTime: 48ms (48999649) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 45.16s (45159676421) - TotalThreadsSysTime: 2ms (2046000) - TotalThreadsUserTime: 59ms (59531000) - TotalThreadsVoluntaryContextSwitches: 39 (39) - TotalTime: 45.18s (45182676256) Fragment Instance Lifecycle Timings - ExecTime: 65ms (65999527) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 45.09s (45093676894) - ExecTreeOpenTime: 45.04s (45035677310) - PrepareTime: 22ms (22999835) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=32) - BytesSent: 33.6 KiB (34410) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 32.8 MiB/s (34410240) - OverallThroughput: 16.4 MiB/s (17205120) - PeakMemoryUsage: 36.6 KiB (37440) - RowsReturned: 373 (373) - SerializeBatchTime: 0ns (0) - TotalTime: 1ms (1999986) - TransmitDataRPCTime: 999.99us (999993) - UncompressedRowBatchSize: 63.4 KiB (64902) CodeGen - CodegenTime: 999.99us (999993) - CompileTime: 14ms (14999892) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 35 (35) - NumInstructions: 654 (654) - OptimizationTime: 40ms (40999706) - PeakMemoryUsage: 327.0 KiB (334848) - PrepareTime: 19ms (19999857) - TotalTime: 76ms (76999447) AGGREGATION_NODE (id=31) ExecOption: Codegen Enabled - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 8.5 MiB (8964286) - RowsRepartitioned: 0 (0) - RowsReturned: 373 (373) - RowsReturnedRate: 8 per second (8) - SpilledPartitions: 0 (0) - TotalTime: 45.04s (45035677310) EXCHANGE_NODE (id=30) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 489 (489) - RowsReturnedRate: 10 per second (10) - TotalTime: 45.03s (45034677317) DataStreamReceiver - BytesReceived: 6.2 KiB (6331) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 44.88s (44875678456) - InactiveTotalTime: 45.03s (45033677324) - PeakMemoryUsage: 6.8 KiB (6990) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.5 MiB (8912896) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000024 (host=ip-10-1-28-130.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 8.6 MiB (9012331) - PeakReservation: 8.5 MiB (8912896) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 366 (366) - TotalNetworkReceiveTime: 44.98s (44983613384) - TotalNetworkSendTime: 19ms (19999828) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 13 (13) - TotalThreadsTotalWallClockTime: 45.09s (45089612472) - TotalThreadsSysTime: 1ms (1026000) - TotalThreadsUserTime: 79ms (79729000) - TotalThreadsVoluntaryContextSwitches: 57 (57) - TotalTime: 45.14s (45145611991) Fragment Instance Lifecycle Timings - ExecTime: 22ms (22999802) - ExecTreeExecTime: 999.99us (999992) - InactiveTotalTime: 0ns (0) - OpenTime: 45.07s (45066612670) - ExecTreeOpenTime: 44.98s (44984613375) - PrepareTime: 55ms (55999519) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=32) - BytesSent: 32.8 KiB (33624) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 16.0 MiB/s (16812134) - OverallThroughput: 2.3 MiB/s (2401734) - PeakMemoryUsage: 36.6 KiB (37440) - RowsReturned: 366 (366) - SerializeBatchTime: 0ns (0) - TotalTime: 13ms (13999881) - TransmitDataRPCTime: 1ms (1999984) - UncompressedRowBatchSize: 62.2 KiB (63684) CodeGen - CodegenTime: 2ms (2999975) - CompileTime: 21ms (21999811) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 35 (35) - NumInstructions: 654 (654) - OptimizationTime: 56ms (56999510) - PeakMemoryUsage: 327.0 KiB (334848) - PrepareTime: 42ms (42999631) - TotalTime: 122ms (122998943) AGGREGATION_NODE (id=31) ExecOption: Codegen Enabled - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 8.5 MiB (8964224) - RowsRepartitioned: 0 (0) - RowsReturned: 366 (366) - RowsReturnedRate: 8 per second (8) - SpilledPartitions: 0 (0) - TotalTime: 44.98s (44985613367) EXCHANGE_NODE (id=30) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 462 (462) - RowsReturnedRate: 10 per second (10) - TotalTime: 44.98s (44983613384) DataStreamReceiver - BytesReceived: 5.9 KiB (6026) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 44.82s (44825614742) - InactiveTotalTime: 44.98s (44983613384) - PeakMemoryUsage: 6.7 KiB (6816) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.5 MiB (8912896) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000026 (host=ip-10-1-16-98.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 8.6 MiB (9012259) - PeakReservation: 8.5 MiB (8912896) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 310.3 MiB (325335426) - RowsProduced: 354 (354) - TotalNetworkReceiveTime: 45.02s (45015153828) - TotalNetworkSendTime: 16ms (16000055) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 1 (1) - TotalThreadsTotalWallClockTime: 45.11s (45109154150) - TotalThreadsSysTime: 0ns (0) - TotalThreadsUserTime: 74ms (74347000) - TotalThreadsVoluntaryContextSwitches: 54 (54) - TotalTime: 45.14s (45137154245) Fragment Instance Lifecycle Timings - ExecTime: 21ms (21000072) - ExecTreeExecTime: 2ms (2000007) - InactiveTotalTime: 0ns (0) - OpenTime: 45.09s (45088154078) - ExecTreeOpenTime: 45.02s (45015153828) - PrepareTime: 28ms (28000095) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=32) - BytesSent: 32.0 KiB (32784) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 15.6 MiB/s (16391942) - OverallThroughput: 6.3 MiB/s (6556777) - PeakMemoryUsage: 36.6 KiB (37440) - RowsReturned: 354 (354) - SerializeBatchTime: 0ns (0) - TotalTime: 5ms (5000017) - TransmitDataRPCTime: 2ms (2000007) - UncompressedRowBatchSize: 60.2 KiB (61596) CodeGen - CodegenTime: 3ms (3000010) - CompileTime: 21ms (21000072) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 35 (35) - NumInstructions: 654 (654) - OptimizationTime: 50ms (50000171) - PeakMemoryUsage: 327.0 KiB (334848) - PrepareTime: 23ms (23000080) - TotalTime: 95ms (95000326) AGGREGATION_NODE (id=31) ExecOption: Codegen Enabled - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 7 (7) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 8.5 MiB (8964224) - RowsRepartitioned: 0 (0) - RowsReturned: 354 (354) - RowsReturnedRate: 7 per second (7) - SpilledPartitions: 0 (0) - TotalTime: 45.02s (45017153835) EXCHANGE_NODE (id=30) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 460 (460) - RowsReturnedRate: 10 per second (10) - TotalTime: 45.02s (45015153828) DataStreamReceiver - BytesReceived: 5.9 KiB (6035) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 44.86s (44857153288) - InactiveTotalTime: 45.02s (45015153828) - PeakMemoryUsage: 6.7 KiB (6872) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.5 MiB (8912896) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000027 (host=ip-10-1-18-116.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 8.6 MiB (9012937) - PeakReservation: 8.5 MiB (8912896) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 314.6 MiB (329895427) - RowsProduced: 384 (384) - TotalNetworkReceiveTime: 45.00s (45001105722) - TotalNetworkSendTime: 33ms (33000810) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 45.09s (45094108007) - TotalThreadsSysTime: 3ms (3189000) - TotalThreadsUserTime: 51ms (51048000) - TotalThreadsVoluntaryContextSwitches: 50 (50) - TotalTime: 45.12s (45116108548) Fragment Instance Lifecycle Timings - ExecTime: 37ms (37000909) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 45.06s (45057107098) - ExecTreeOpenTime: 45.01s (45007105870) - PrepareTime: 22ms (22000541) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=32) - BytesSent: 34.6 KiB (35388) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 16.9 MiB/s (17693566) - PeakMemoryUsage: 36.6 KiB (37440) - RowsReturned: 384 (384) - SerializeBatchTime: 1ms (1000024) - TotalTime: 2ms (2000049) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 65.2 KiB (66816) CodeGen - CodegenTime: 1ms (1000024) - CompileTime: 15ms (15000368) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 35 (35) - NumInstructions: 654 (654) - OptimizationTime: 34ms (34000836) - PeakMemoryUsage: 327.0 KiB (334848) - PrepareTime: 19ms (19000466) - TotalTime: 69ms (69001695) AGGREGATION_NODE (id=31) ExecOption: Codegen Enabled - BuildTime: 5ms (5000123) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 8.5 MiB (8964622) - RowsRepartitioned: 0 (0) - RowsReturned: 384 (384) - RowsReturnedRate: 8 per second (8) - SpilledPartitions: 0 (0) - TotalTime: 45.01s (45007105870) EXCHANGE_NODE (id=30) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 504 (504) - RowsReturnedRate: 11 per second (11) - TotalTime: 45.00s (45001105722) DataStreamReceiver - BytesReceived: 6.4 KiB (6557) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 44.85s (44849101987) - InactiveTotalTime: 45.00s (45001105722) - PeakMemoryUsage: 7.2 KiB (7342) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 5ms (5000123) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.5 MiB (8912896) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000023 (host=ip-10-1-21-97.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 8.6 MiB (9012866) - PeakReservation: 8.5 MiB (8912896) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 314.4 MiB (329677036) - RowsProduced: 376 (376) - TotalNetworkReceiveTime: 44.96s (44957101569) - TotalNetworkSendTime: 14ms (14000031) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 2 (2) - TotalThreadsTotalWallClockTime: 45.05s (45046101769) - TotalThreadsSysTime: 1ms (1063000) - TotalThreadsUserTime: 70ms (70653000) - TotalThreadsVoluntaryContextSwitches: 43 (43) - TotalTime: 45.09s (45088101864) Fragment Instance Lifecycle Timings - ExecTime: 19ms (19000043) - ExecTreeExecTime: 1ms (1000003) - InactiveTotalTime: 0ns (0) - OpenTime: 45.03s (45027101726) - ExecTreeOpenTime: 44.96s (44958101571) - PrepareTime: 42ms (42000095) - ExecTreePrepareTime: 1ms (1000002) - TotalTime: 0ns (0) DataStreamSender (dst_id=32) - BytesSent: 34.2 KiB (35010) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 4.2 MiB/s (4376240) - PeakMemoryUsage: 36.6 KiB (37440) - RowsReturned: 376 (376) - SerializeBatchTime: 0ns (0) - TotalTime: 8ms (8000018) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 63.9 KiB (65424) CodeGen - CodegenTime: 3ms (3000007) - CompileTime: 19ms (19000043) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 35 (35) - NumInstructions: 654 (654) - OptimizationTime: 48ms (48000108) - PeakMemoryUsage: 327.0 KiB (334848) - PrepareTime: 23ms (23000051) - TotalTime: 91ms (91000204) AGGREGATION_NODE (id=31) ExecOption: Codegen Enabled - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 8.5 MiB (8964559) - RowsRepartitioned: 0 (0) - RowsReturned: 376 (376) - RowsReturnedRate: 8 per second (8) - SpilledPartitions: 0 (0) - TotalTime: 44.96s (44960101576) EXCHANGE_NODE (id=30) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 511 (511) - RowsReturnedRate: 11 per second (11) - TotalTime: 44.96s (44957101569) DataStreamReceiver - BytesReceived: 6.5 KiB (6617) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 44.80s (44799101210) - InactiveTotalTime: 44.96s (44957101569) - PeakMemoryUsage: 7.2 KiB (7343) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.5 MiB (8912896) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000025 (host=ip-10-1-27-82.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 8.6 MiB (9012323) - PeakReservation: 8.5 MiB (8912896) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 303.4 MiB (318098845) - RowsProduced: 361 (361) - TotalNetworkReceiveTime: 44.09s (44086708229) - TotalNetworkSendTime: 26ms (26999209) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 17 (17) - TotalThreadsTotalWallClockTime: 44.35s (44348700553) - TotalThreadsSysTime: 0ns (0) - TotalThreadsUserTime: 78ms (78460000) - TotalThreadsVoluntaryContextSwitches: 58 (58) - TotalTime: 44.80s (44800687310) Fragment Instance Lifecycle Timings - ExecTime: 28ms (28999151) - ExecTreeExecTime: 999.97us (999971) - InactiveTotalTime: 0ns (0) - OpenTime: 44.32s (44319701402) - ExecTreeOpenTime: 44.09s (44087708200) - PrepareTime: 451ms (451986757) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=32) - BytesSent: 32.6 KiB (33390) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 31.8 MiB/s (33390968) - OverallThroughput: 123.0 KiB/s (126003) - PeakMemoryUsage: 36.6 KiB (37440) - RowsReturned: 361 (361) - SerializeBatchTime: 0ns (0) - TotalTime: 264ms (264992236) - TransmitDataRPCTime: 999.97us (999971) - UncompressedRowBatchSize: 61.3 KiB (62814) CodeGen - CodegenTime: 21ms (21999356) - CompileTime: 128ms (128996220) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 35 (35) - NumInstructions: 654 (654) - OptimizationTime: 92ms (92997276) - PeakMemoryUsage: 327.0 KiB (334848) - PrepareTime: 183ms (183994608) - TotalTime: 406ms (406988075) AGGREGATION_NODE (id=31) ExecOption: Codegen Enabled - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 7 (7) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 8.5 MiB (8964224) - RowsRepartitioned: 0 (0) - RowsReturned: 361 (361) - RowsReturnedRate: 8 per second (8) - SpilledPartitions: 0 (0) - TotalTime: 44.09s (44088708171) EXCHANGE_NODE (id=30) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 460 (460) - RowsReturnedRate: 10 per second (10) - TotalTime: 44.09s (44086708229) DataStreamReceiver - BytesReceived: 5.8 KiB (5939) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 43.93s (43928712859) - InactiveTotalTime: 44.09s (44086708229) - PeakMemoryUsage: 6.6 KiB (6808) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.5 MiB (8912896) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Averaged Fragment F05 split sizes: min: 15.06 GB, max: 17.50 GB, avg: 16.91 GB, stddev: 938.99 MB completion times: min:44s892ms max:45s080ms mean: 44s996ms stddev:72.866ms execution rates: min:343.14 MB/sec max:399.00 MB/sec mean:384.76 MB/sec stddev:20.81 MB/sec num instances: 6 - AverageThreadTokens: 1.83 (1.8296504369538076) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 16.2 MiB (16979881) - PeakReservation: 12.9 MiB (13500416) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 307.6 MiB (322494863) - RowsProduced: 481 (481) - TotalNetworkReceiveTime: 43.99s (43987399325) - TotalNetworkSendTime: 4ms (4833355) - TotalStorageWaitTime: 99ms (99833258) - TotalThreadsInvoluntaryContextSwitches: 38 (38) - TotalThreadsTotalWallClockTime: 1.4m (81782669581) - TotalThreadsSysTime: 17ms (17808000) - TotalThreadsUserTime: 799ms (799863666) - TotalThreadsVoluntaryContextSwitches: 214 (214) - TotalTime: 44.95s (44951223870) Fragment Instance Lifecycle Timings - ExecTime: 45ms (45500035) - ExecTreeExecTime: 9ms (9666596) - InactiveTotalTime: 0ns (0) - OpenTime: 44.67s (44673394658) - ExecTreeOpenTime: 43.92s (43915231031) - PrepareTime: 232ms (232329176) - ExecTreePrepareTime: 162ms (162496836) - TotalTime: 0ns (0) DataStreamSender (dst_id=30) - BytesSent: 6.1 KiB (6250) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 3.7 MiB/s (3891062) - PeakMemoryUsage: 60.6 KiB (62016) - RowsReturned: 481 (481) - SerializeBatchTime: 0ns (0) - TotalTime: 14ms (14666305) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13.6 KiB (13949) CodeGen - CodegenTime: 4ms (4166653) - CompileTime: 236ms (236499032) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 165 (165) - NumInstructions: 6,311 (6311) - OptimizationTime: 519ms (519497934) - PeakMemoryUsage: 3.1 MiB (3231232) - PrepareTime: 40ms (40832778) - TotalTime: 797ms (797663078) AGGREGATION_NODE (id=16) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 4 (4) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 9.5 MiB (9981397) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 481 (481) - RowsReturnedRate: 10 per second (10) - StreamingTime: 499.99us (499994) - TotalTime: 44.09s (44087394465) HASH_JOIN_NODE (id=15) - BuildRows: 1 (1) - BuildTime: 1ms (1166664) - InactiveTotalTime: 0ns (0) - LocalTime: 166.66us (166661) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.4 MiB (2506666) - ProbeRows: 549 (549) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 549 (549) - RowsReturnedRate: 12 per second (12) - TotalTime: 44.25s (44246391344) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=15) - BuildRowsPartitionTime: 166.66us (166661) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=29) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 2 per second (2) - TotalTime: 192ms (192834536) DataStreamReceiver - BytesReceived: 26 B (26) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=13) - BuildRows: 1 (1) - BuildTime: 1,000.00us (999997) - InactiveTotalTime: 0ns (0) - LocalTime: 1ms (1666664) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2102869) - ProbeRows: 549 (549) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 549 (549) - RowsReturnedRate: 12 per second (12) - TotalTime: 44.25s (44246224682) Hash Join Builder (join_node_id=13) - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=28) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.79s (43794564789) DataStreamReceiver - BytesReceived: 13 B (13) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 43.73s (43729398297) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=10) - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.17 (0.16666666666666666) - BytesRead: 7.0 MiB (7351708) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 7.0 MiB (7351708) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 7.0 MiB (7351708) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 52 (52) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 5 (5) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 6 (6) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 6 (6) - PeakMemoryUsage: 4.3 MiB (4544466) - PerReadThreadRawHdfsThroughput: 301.0 MiB/s (315570977) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 113ms (113332754) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 868,917 (868917) - RowsReturned: 549 (549) - RowsReturnedRate: 1910 per second (1910) - ScanRangesComplete: 10,252 (10252) - ScannerThreadsInvoluntaryContextSwitches: 5 (5) - ScannerThreadsTotalWallClockTime: 203ms (203500003) - MaterializeTupleTime(*): 83ms (83333452) - ScannerThreadsSysTime: 8ms (8247000) - ScannerThreadsUserTime: 82ms (82896166) - ScannerThreadsVoluntaryContextSwitches: 163 (163) - TotalRawHdfsReadTime(*): 72ms (72999271) - TotalReadThroughput: 10.4 KiB/s (10654) - TotalTime: 451ms (451826569) Filter 10 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 868,917 (868917) - Rows rejected: 868,319 (868319) - Rows total: 868,917 (868917) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 12 (1.00 MB) - Files processed: 10,252 (10252) - Files rejected: 10,240 (10240) - Files total: 10,252 (10252) - InactiveTotalTime: 0ns (0) - RowGroups processed: 8 (8) - RowGroups rejected: 0 (0) - RowGroups total: 8 (8) - Rows processed: 598 (598) - Rows rejected: 0 (0) - Rows total: 598 (598) - Splits processed: 12 (12) - Splits rejected: 0 (0) - Splits total: 12 (12) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 6.0 MiB (6291456) - CumulativeAllocations: 12 (12) - InactiveTotalTime: 0ns (0) - PeakReservation: 9.0 MiB (9437184) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 6.0 MiB (6291456) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment F05 Instance 68440bc099e1b843:8baf45150000001c (host=ip-10-1-20-86.ec2.internal:22000) Hdfs split stats (:<# splits>/): 0:760/1.06 GB 7:835/1.59 GB 2:822/1.65 GB 8:808/1.40 GB 1:799/1.41 GB 6:827/1.55 GB 10:850/1.47 GB 11:821/1.48 GB 9:747/1.19 GB 5:753/1.39 GB 3:707/1.01 GB 4:775/1.22 GB Filter 10 arrival: 1s170ms Filter 12 arrival: 44s868ms - AverageThreadTokens: 1.97 (1.9666666666666666) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 19.8 MiB (20730562) - PeakReservation: 12.9 MiB (13500416) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 312.0 MiB (327162809) - RowsProduced: 1,196 (1196) - TotalNetworkReceiveTime: 43.70s (43698686889) - TotalNetworkSendTime: 4ms (4999964) - TotalStorageWaitTime: 109ms (109999214) - TotalThreadsInvoluntaryContextSwitches: 101 (101) - TotalThreadsTotalWallClockTime: 1.5m (89040362005) - TotalThreadsSysTime: 30ms (30786000) - TotalThreadsUserTime: 1.08s (1082973000) - TotalThreadsVoluntaryContextSwitches: 213 (213) - TotalTime: 45.07s (45073677037) Fragment Instance Lifecycle Timings - ExecTime: 61ms (61999555) - ExecTreeExecTime: 54ms (54999605) - InactiveTotalTime: 0ns (0) - OpenTime: 44.90s (44901678270) - ExecTreeOpenTime: 43.84s (43842685858) - PrepareTime: 109ms (109999212) - ExecTreePrepareTime: 65ms (65999527) - TotalTime: 0ns (0) DataStreamSender (dst_id=30) - BytesSent: 15.1 KiB (15469) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 14.8 MiB/s (15469108) - PeakMemoryUsage: 108.6 KiB (111168) - RowsReturned: 1,196 (1196) - SerializeBatchTime: 0ns (0) - TotalTime: 999.99us (999993) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 33.9 KiB (34684) CodeGen - CodegenTime: 3ms (3999972) - CompileTime: 262ms (262998116) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 165 (165) - NumInstructions: 6,311 (6311) - OptimizationTime: 792ms (792994318) - PeakMemoryUsage: 3.1 MiB (3231232) - PrepareTime: 29ms (29999784) - TotalTime: 1.09s (1086992211) AGGREGATION_NODE (id=16) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 32,768 (32768) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 10.2 MiB (10729600) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 1,196 (1196) - RowsReturnedRate: 27 per second (27) - StreamingTime: 999.99us (999992) - TotalTime: 43.96s (43963684990) HASH_JOIN_NODE (id=15) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 999.99us (999992) - InactiveTotalTime: 0ns (0) - LocalTime: 0ns (0) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.9 MiB (2999552) - ProbeRows: 1,417 (1417) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 1,417 (1417) - RowsReturnedRate: 32 per second (32) - TotalTime: 44.03s (44026684540) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=15) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=29) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 0ns (0) DataStreamReceiver - BytesReceived: 26 B (26) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=13) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 999.99us (999993) - InactiveTotalTime: 0ns (0) - LocalTime: 1ms (1999986) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2116864) - ProbeRows: 1,417 (1417) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 1,417 (1417) - RowsReturnedRate: 32 per second (32) - TotalTime: 44.03s (44026684540) Hash Join Builder (join_node_id=13) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=28) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.70s (43698686889) DataStreamReceiver - BytesReceived: 13 B (13) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 43.63s (43632687362) - InactiveTotalTime: 43.70s (43698686889) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=10) Hdfs split stats (:<# splits>/): 0:760/1.06 GB 7:835/1.59 GB 2:822/1.65 GB 8:808/1.40 GB 1:799/1.41 GB 6:827/1.55 GB 10:850/1.47 GB 11:821/1.48 GB 9:747/1.19 GB 5:753/1.39 GB 3:707/1.01 GB 4:775/1.22 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 28 out of 28 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:28 PARQUET/Unknown(Skipped):9476 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 18.2 MiB (19066232) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 18.2 MiB (19066232) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 18.2 MiB (19066232) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 128 (128) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 5 (5) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 28 (28) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 8 (8) - PeakMemoryUsage: 6.5 MiB (6801596) - PerReadThreadRawHdfsThroughput: 1.5 GiB/s (1588864053) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 185ms (185998668) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,280,280 (2280280) - RowsReturned: 1,417 (1417) - RowsReturnedRate: 4307 per second (4307) - ScanRangesComplete: 9,504 (9504) - ScannerThreadsInvoluntaryContextSwitches: 16 (16) - ScannerThreadsTotalWallClockTime: 375ms (375997306) - MaterializeTupleTime(*): 205ms (205998526) - ScannerThreadsSysTime: 24ms (24899000) - ScannerThreadsUserTime: 238ms (238830000) - ScannerThreadsVoluntaryContextSwitches: 135 (135) - TotalRawHdfsReadTime(*): 11ms (11999914) - TotalReadThroughput: 0 B/s (0) - TotalTime: 328ms (328997643) Filter 10 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,280,280 (2280280) - Rows rejected: 2,278,747 (2278747) - Rows total: 2,280,280 (2280280) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 12 (1.00 MB) - Files processed: 9,503 (9503) - Files rejected: 9,475 (9475) - Files total: 9,503 (9503) - InactiveTotalTime: 0ns (0) - RowGroups processed: 20 (20) - RowGroups rejected: 0 (0) - RowGroups total: 20 (20) - Rows processed: 1,533 (1533) - Rows rejected: 0 (0) - Rows total: 1,533 (1533) - Splits processed: 28 (28) - Splits rejected: 0 (0) - Splits total: 28 (28) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 9.0 MiB (9437184) - CumulativeAllocations: 18 (18) - InactiveTotalTime: 0ns (0) - PeakReservation: 9.0 MiB (9437184) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 9.0 MiB (9437184) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000021 (host=ip-10-1-18-116.ec2.internal:22000) Hdfs split stats (:<# splits>/): 8:950/1.59 GB 2:1005/1.68 GB 9:1011/1.81 GB 11:1065/1.25 GB 0:1021/1.29 GB 10:979/1.75 GB 7:999/1.34 GB 3:972/1.46 GB 4:1002/1.44 GB 6:993/1.22 GB 1:925/1.43 GB 5:1066/1.23 GB Filter 10 arrival: 1s039ms Filter 12 arrival: 44s821ms - AverageThreadTokens: 2.00 (2.0) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 15.6 MiB (16385542) - PeakReservation: 12.9 MiB (13500416) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 314.6 MiB (329895427) - RowsProduced: 418 (418) - TotalNetworkReceiveTime: 44.31s (44314088840) - TotalNetworkSendTime: 8ms (8000196) - TotalStorageWaitTime: 213ms (213005233) - TotalThreadsInvoluntaryContextSwitches: 13 (13) - TotalThreadsTotalWallClockTime: 1.5m (89646202696) - TotalThreadsSysTime: 25ms (25124000) - TotalThreadsUserTime: 754ms (754133000) - TotalThreadsVoluntaryContextSwitches: 87 (87) - TotalTime: 45.06s (45058107123) Fragment Instance Lifecycle Timings - ExecTime: 11ms (11000271) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 44.95s (44953104542) - ExecTreeOpenTime: 44.28s (44275087882) - PrepareTime: 94ms (94002310) - ExecTreePrepareTime: 59ms (59001450) - TotalTime: 0ns (0) DataStreamSender (dst_id=30) - BytesSent: 5.3 KiB (5466) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 5.2 MiB/s (5465868) - PeakMemoryUsage: 60.6 KiB (62016) - RowsReturned: 418 (418) - SerializeBatchTime: 0ns (0) - TotalTime: 1ms (1000024) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 11.8 KiB (12122) CodeGen - CodegenTime: 4ms (4000098) - CompileTime: 216ms (216005308) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 165 (165) - NumInstructions: 6,311 (6311) - OptimizationTime: 460ms (460011303) - PeakMemoryUsage: 3.1 MiB (3231232) - PrepareTime: 24ms (24000589) - TotalTime: 701ms (701017225) AGGREGATION_NODE (id=16) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 7 (7) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 9.6 MiB (10026112) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 418 (418) - RowsReturnedRate: 9 per second (9) - StreamingTime: 0ns (0) - TotalTime: 44.33s (44334089332) HASH_JOIN_NODE (id=15) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 1ms (1000024) - InactiveTotalTime: 0ns (0) - LocalTime: 0ns (0) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.5 MiB (2612480) - ProbeRows: 451 (451) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 451 (451) - RowsReturnedRate: 10 per second (10) - TotalTime: 44.39s (44392090758) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=15) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=29) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 3 per second (3) - TotalTime: 266ms (266006536) DataStreamReceiver - BytesReceived: 26 B (26) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 0ns (0) - InactiveTotalTime: 266ms (266006536) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=13) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 1ms (1000025) - InactiveTotalTime: 0ns (0) - LocalTime: 1ms (1000025) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2116864) - ProbeRows: 451 (451) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 451 (451) - RowsReturnedRate: 10 per second (10) - TotalTime: 44.39s (44392090758) Hash Join Builder (join_node_id=13) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=28) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.05s (44048082304) DataStreamReceiver - BytesReceived: 13 B (13) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 43.99s (43986080781) - InactiveTotalTime: 44.05s (44048082304) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=10) Hdfs split stats (:<# splits>/): 8:950/1.59 GB 2:1005/1.68 GB 9:1011/1.81 GB 11:1065/1.25 GB 0:1021/1.29 GB 10:979/1.75 GB 7:999/1.34 GB 3:972/1.46 GB 4:1002/1.44 GB 6:993/1.22 GB 1:925/1.43 GB 5:1066/1.23 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 11 out of 11 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:11 PARQUET/Unknown(Skipped):11977 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 5.2 MiB (5500569) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 5.2 MiB (5500569) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 5.2 MiB (5500569) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 56 (56) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 5 (5) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 9 (9) - NumRowGroups: 11 (11) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 2 (2) - PeakMemoryUsage: 6.5 MiB (6801596) - PerReadThreadRawHdfsThroughput: 44.8 MiB/s (47012255) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 182ms (182004472) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 639,262 (639262) - RowsReturned: 451 (451) - RowsReturnedRate: 1303 per second (1303) - ScanRangesComplete: 11,988 (11988) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 366ms (366008994) - MaterializeTupleTime(*): 139ms (139003415) - ScannerThreadsSysTime: 4ms (4337000) - ScannerThreadsUserTime: 78ms (78086000) - ScannerThreadsVoluntaryContextSwitches: 46 (46) - TotalRawHdfsReadTime(*): 117ms (117002874) - TotalReadThroughput: 0 B/s (0) - TotalTime: 346ms (346008502) Filter 10 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 639,262 (639262) - Rows rejected: 638,777 (638777) - Rows total: 639,262 (639262) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 12 (1.00 MB) - Files processed: 11,988 (11988) - Files rejected: 11,977 (11977) - Files total: 11,988 (11988) - InactiveTotalTime: 0ns (0) - RowGroups processed: 9 (9) - RowGroups rejected: 0 (0) - RowGroups total: 9 (9) - Rows processed: 485 (485) - Rows rejected: 0 (0) - Rows total: 485 (485) - Splits processed: 11 (11) - Splits rejected: 0 (0) - Splits total: 11 (11) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 9.0 MiB (9437184) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000020 (host=ip-10-1-16-98.ec2.internal:22000) Hdfs split stats (:<# splits>/): 5:708/1.57 GB 0:786/1.72 GB 1:738/1.48 GB 11:682/1.17 GB 3:690/1.34 GB 8:762/1.80 GB 2:739/1.28 GB 7:776/1.43 GB 6:742/1.27 GB 4:756/1.63 GB 10:736/1.60 GB 9:771/1.21 GB Filter 10 arrival: 1s078ms Filter 12 arrival: 44s864ms - AverageThreadTokens: 2.00 (2.0) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 15.1 MiB (15839750) - PeakReservation: 12.9 MiB (13500416) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 310.3 MiB (325335426) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 44.67s (44673152658) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 12 (12) - TotalThreadsTotalWallClockTime: 1.5m (89641306326) - TotalThreadsSysTime: 6ms (6303000) - TotalThreadsUserTime: 559ms (559270000) - TotalThreadsVoluntaryContextSwitches: 26 (26) - TotalTime: 45.04s (45044153927) Fragment Instance Lifecycle Timings - ExecTime: 173ms (173000591) - ExecTreeExecTime: 1ms (1000003) - InactiveTotalTime: 0ns (0) - OpenTime: 44.79s (44792153066) - ExecTreeOpenTime: 44.24s (44237151170) - PrepareTime: 79ms (79000270) - ExecTreePrepareTime: 47ms (47000161) - TotalTime: 0ns (0) DataStreamSender (dst_id=30) - BytesSent: 0 B (0) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 36.6 KiB (37440) - RowsReturned: 0 (0) - SerializeBatchTime: 0ns (0) - TotalTime: 3ms (3000010) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 0 B (0) CodeGen - CodegenTime: 2ms (2000007) - CompileTime: 174ms (174000594) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 165 (165) - NumInstructions: 6,311 (6311) - OptimizationTime: 380ms (380001299) - PeakMemoryUsage: 3.1 MiB (3231232) - PrepareTime: 17ms (17000059) - TotalTime: 572ms (572001956) AGGREGATION_NODE (id=16) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 9.0 MiB (9480320) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - StreamingTime: 0ns (0) - TotalTime: 44.28s (44285151334) HASH_JOIN_NODE (id=15) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 2ms (2000007) - InactiveTotalTime: 0ns (0) - LocalTime: 0ns (0) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2074880) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.33s (44331151492) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=15) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=29) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 2 per second (2) - TotalTime: 444ms (444001517) DataStreamReceiver - BytesReceived: 26 B (26) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 0ns (0) - InactiveTotalTime: 444ms (444001517) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=13) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 1ms (1000004) - InactiveTotalTime: 0ns (0) - LocalTime: 2ms (2000008) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2074880) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.33s (44331151492) Hash Join Builder (join_node_id=13) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=28) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.23s (44229151141) DataStreamReceiver - BytesReceived: 13 B (13) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 44.16s (44163150916) - InactiveTotalTime: 44.23s (44229151141) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=10) Hdfs split stats (:<# splits>/): 5:708/1.57 GB 0:786/1.72 GB 1:738/1.48 GB 11:682/1.17 GB 3:690/1.34 GB 8:762/1.80 GB 2:739/1.28 GB 7:776/1.43 GB 6:742/1.27 GB 4:756/1.63 GB 10:736/1.60 GB 9:771/1.21 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 0 out of 0 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/Unknown(Skipped):8886 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 0 B (0) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 0 (0) - PeakMemoryUsage: 64.0 KiB (65536) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - ScanRangesComplete: 8,886 (8886) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0ns (0) - MaterializeTupleTime(*): 0ns (0) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 0ns (0) - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 101ms (101000346) Filter 10 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 12 (1.00 MB) - Files processed: 8,886 (8886) - Files rejected: 8,886 (8886) - Files total: 8,886 (8886) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 512.0 KiB (524288) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 9.0 MiB (9437184) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 512.0 KiB (524288) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf45150000001d (host=ip-10-1-21-97.ec2.internal:22000) Hdfs split stats (:<# splits>/): 7:838/1.15 GB 2:876/1.46 GB 5:798/1.29 GB 6:772/1.33 GB 11:836/1.50 GB 10:837/1.65 GB 1:769/1.21 GB 4:739/1.10 GB 8:781/1003.47 MB 9:778/1.35 GB 3:748/1.10 GB 0:774/970.37 MB Filter 10 arrival: 1s031ms Filter 12 arrival: 44s814ms - AverageThreadTokens: 2.00 (2.0) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 16.3 MiB (17044256) - PeakReservation: 12.9 MiB (13500416) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 314.4 MiB (329677036) - RowsProduced: 904 (904) - TotalNetworkReceiveTime: 44.34s (44337100168) - TotalNetworkSendTime: 14ms (14000032) - TotalStorageWaitTime: 101ms (101000228) - TotalThreadsInvoluntaryContextSwitches: 33 (33) - TotalThreadsTotalWallClockTime: 1.5m (89399201975) - TotalThreadsSysTime: 26ms (26855000) - TotalThreadsUserTime: 780ms (780518000) - TotalThreadsVoluntaryContextSwitches: 830 (830) - TotalTime: 44.92s (44920101483) Fragment Instance Lifecycle Timings - ExecTime: 16ms (16000036) - ExecTreeExecTime: 1ms (1000002) - InactiveTotalTime: 0ns (0) - OpenTime: 44.80s (44801101215) - ExecTreeOpenTime: 44.15s (44150099746) - PrepareTime: 103ms (103000232) - ExecTreePrepareTime: 49ms (49000111) - TotalTime: 0ns (0) DataStreamSender (dst_id=30) - BytesSent: 11.5 KiB (11730) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 2.2 MiB/s (2345994) - PeakMemoryUsage: 60.6 KiB (62016) - RowsReturned: 904 (904) - SerializeBatchTime: 0ns (0) - TotalTime: 5ms (5000011) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 25.6 KiB (26216) CodeGen - CodegenTime: 7ms (7000016) - CompileTime: 226ms (226000510) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 165 (165) - NumInstructions: 6,311 (6311) - OptimizationTime: 422ms (422000952) - PeakMemoryUsage: 3.1 MiB (3231232) - PrepareTime: 28ms (28000063) - TotalTime: 677ms (677001528) AGGREGATION_NODE (id=16) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 32,768 (32768) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 6 (6) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 10.0 MiB (10492032) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 904 (904) - RowsReturnedRate: 20 per second (20) - StreamingTime: 1ms (1000002) - TotalTime: 44.20s (44200099859) HASH_JOIN_NODE (id=15) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 1ms (1000002) - InactiveTotalTime: 0ns (0) - LocalTime: 0ns (0) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.9 MiB (3011840) - ProbeRows: 1,043 (1043) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 1,043 (1043) - RowsReturnedRate: 23 per second (23) - TotalTime: 44.25s (44247099965) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=15) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=29) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 3 per second (3) - TotalTime: 277ms (277000626) DataStreamReceiver - BytesReceived: 26 B (26) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 0ns (0) - InactiveTotalTime: 277ms (277000626) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=13) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 1ms (1000002) - InactiveTotalTime: 0ns (0) - LocalTime: 3ms (3000006) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2116864) - ProbeRows: 1,043 (1043) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 1,043 (1043) - RowsReturnedRate: 23 per second (23) - TotalTime: 44.25s (44247099965) Hash Join Builder (join_node_id=13) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=28) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.06s (44060099542) DataStreamReceiver - BytesReceived: 13 B (13) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 43.99s (43994099392) - InactiveTotalTime: 44.06s (44059099540) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=10) Hdfs split stats (:<# splits>/): 7:838/1.15 GB 2:876/1.46 GB 5:798/1.29 GB 6:772/1.33 GB 11:836/1.50 GB 10:837/1.65 GB 1:769/1.21 GB 4:739/1.10 GB 8:781/1003.47 MB 9:778/1.35 GB 3:748/1.10 GB 0:774/970.37 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 27 out of 27 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% File Formats: PARQUET/NONE:27 PARQUET/Unknown(Skipped):9519 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 13.5 MiB (14154402) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 13.5 MiB (14154402) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 13.5 MiB (14154402) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 107 (107) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 5 (5) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 27 (27) - NumScannerThreadsStarted: 3 (3) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 11 (11) - PeakMemoryUsage: 9.6 MiB (10094874) - PerReadThreadRawHdfsThroughput: 225.0 MiB/s (235906169) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 80ms (80000181) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 1,635,438 (1635438) - RowsReturned: 1,043 (1043) - RowsReturnedRate: 5607 per second (5607) - ScanRangesComplete: 9,546 (9546) - ScannerThreadsInvoluntaryContextSwitches: 5 (5) - ScannerThreadsTotalWallClockTime: 243ms (243000552) - MaterializeTupleTime(*): 105ms (105000238) - ScannerThreadsSysTime: 17ms (17010000) - ScannerThreadsUserTime: 127ms (127672000) - ScannerThreadsVoluntaryContextSwitches: 766 (766) - TotalRawHdfsReadTime(*): 60ms (60000135) - TotalReadThroughput: 0 B/s (0) - TotalTime: 186ms (186000422) Filter 10 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 1,635,438 (1635438) - Rows rejected: 1,634,284 (1634284) - Rows total: 1,635,438 (1635438) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 12 (1.00 MB) - Files processed: 9,546 (9546) - Files rejected: 9,519 (9519) - Files total: 9,546 (9546) - InactiveTotalTime: 0ns (0) - RowGroups processed: 16 (16) - RowGroups rejected: 0 (0) - RowGroups total: 16 (16) - Rows processed: 1,154 (1154) - Rows rejected: 0 (0) - Rows total: 1,154 (1154) - Splits processed: 27 (27) - Splits rejected: 0 (0) - Splits total: 27 (27) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 9.0 MiB (9437184) - CumulativeAllocations: 18 (18) - InactiveTotalTime: 0ns (0) - PeakReservation: 9.0 MiB (9437184) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 9.0 MiB (9437184) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf45150000001e (host=ip-10-1-28-130.ec2.internal:22000) Hdfs split stats (:<# splits>/): 5:682/1.37 GB 3:728/1.12 GB 1:730/1.43 GB 8:687/1.53 GB 6:647/1.57 GB 9:739/1.71 GB 4:687/1.42 GB 0:689/1.51 GB 2:610/1.28 GB 11:628/1.62 GB 7:623/1.52 GB 10:619/1.43 GB Filter 10 arrival: 1s086ms Filter 12 arrival: 44s871ms - AverageThreadTokens: 2.00 (2.0) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 15.1 MiB (15839750) - PeakReservation: 12.9 MiB (13500416) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 44.12s (44125620758) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 14 (14) - TotalThreadsTotalWallClockTime: 1.5m (88890236023) - TotalThreadsSysTime: 4ms (4794000) - TotalThreadsUserTime: 795ms (795783000) - TotalThreadsVoluntaryContextSwitches: 48 (48) - TotalTime: 44.88s (44881614260) Fragment Instance Lifecycle Timings - ExecTime: 3ms (3999965) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 44.75s (44753615361) - ExecTreeOpenTime: 43.96s (43962622159) - PrepareTime: 123ms (123998934) - ExecTreePrepareTime: 89ms (89999227) - TotalTime: 0ns (0) DataStreamSender (dst_id=30) - BytesSent: 0 B (0) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 36.6 KiB (37440) - RowsReturned: 0 (0) - SerializeBatchTime: 0ns (0) - TotalTime: 3ms (3999966) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 0 B (0) CodeGen - CodegenTime: 2ms (2999975) - CompileTime: 264ms (264997723) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 165 (165) - NumInstructions: 6,311 (6311) - OptimizationTime: 523ms (523995496) - PeakMemoryUsage: 3.1 MiB (3231232) - PrepareTime: 21ms (21999811) - TotalTime: 810ms (810993030) AGGREGATION_NODE (id=16) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 9.0 MiB (9480320) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - StreamingTime: 0ns (0) - TotalTime: 44.05s (44052621386) HASH_JOIN_NODE (id=15) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 999.99us (999991) - InactiveTotalTime: 0ns (0) - LocalTime: 0ns (0) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2074880) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.14s (44135620673) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=15) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=29) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 5 per second (5) - TotalTime: 169ms (169998539) DataStreamReceiver - BytesReceived: 26 B (26) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 0ns (0) - InactiveTotalTime: 169ms (169998539) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=13) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 999.99us (999991) - InactiveTotalTime: 0ns (0) - LocalTime: 999.99us (999991) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2074880) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.14s (44135620673) Hash Join Builder (join_node_id=13) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=28) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 43.96s (43955622219) DataStreamReceiver - BytesReceived: 13 B (13) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 43.89s (43889622786) - InactiveTotalTime: 43.96s (43955622219) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=10) Hdfs split stats (:<# splits>/): 5:682/1.37 GB 3:728/1.12 GB 1:730/1.43 GB 8:687/1.53 GB 6:647/1.57 GB 9:739/1.71 GB 4:687/1.42 GB 0:689/1.51 GB 2:610/1.28 GB 11:628/1.62 GB 7:623/1.52 GB 10:619/1.43 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 0 out of 0 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/Unknown(Skipped):8069 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 0 B (0) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 64.0 KiB (65536) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 per second (0) - ScanRangesComplete: 8,069 (8069) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 3ms (3999966) - MaterializeTupleTime(*): 0ns (0) - ScannerThreadsSysTime: 248.00us (248000) - ScannerThreadsUserTime: 0ns (0) - ScannerThreadsVoluntaryContextSwitches: 7 (7) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 180ms (180998446) Filter 10 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 12 (1.00 MB) - Files processed: 8,069 (8069) - Files rejected: 8,069 (8069) - Files total: 8,069 (8069) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 512.0 KiB (524288) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 9.0 MiB (9437184) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 512.0 KiB (524288) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Instance 68440bc099e1b843:8baf45150000001f (host=ip-10-1-27-82.ec2.internal:22000) Hdfs split stats (:<# splits>/): 6:1192/1.30 GB 9:1006/1.10 GB 4:1114/1.51 GB 1:1034/1.58 GB 3:1027/1.36 GB 0:1107/1.41 GB 10:1172/1.41 GB 7:1199/1.32 GB 5:1084/1.41 GB 8:1087/1.65 GB 11:1239/1.43 GB 2:1261/2.02 GB Filter 10 arrival: 1s700ms Filter 12 arrival: 44s476ms - AverageThreadTokens: 1.01 (1.0112359550561798) - BloomFilterBytes: 2.0 MiB (2097152) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 15.3 MiB (16039430) - PeakReservation: 12.9 MiB (13500416) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 303.4 MiB (318098845) - RowsProduced: 368 (368) - TotalNetworkReceiveTime: 42.78s (42775746641) - TotalNetworkSendTime: 1ms (1999941) - TotalStorageWaitTime: 174ms (174994873) - TotalThreadsInvoluntaryContextSwitches: 57 (57) - TotalThreadsTotalWallClockTime: 44.08s (44078708465) - TotalThreadsSysTime: 12ms (12986000) - TotalThreadsUserTime: 826ms (826505000) - TotalThreadsVoluntaryContextSwitches: 85 (85) - TotalTime: 44.73s (44729689390) Fragment Instance Lifecycle Timings - ExecTime: 6ms (6999795) - ExecTreeExecTime: 999.97us (999970) - InactiveTotalTime: 0ns (0) - OpenTime: 43.84s (43838715496) - ExecTreeOpenTime: 43.02s (43023739375) - PrepareTime: 883ms (883974099) - ExecTreePrepareTime: 663ms (663980545) - TotalTime: 0ns (0) DataStreamSender (dst_id=30) - BytesSent: 4.7 KiB (4840) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 63.9 KiB/s (65407) - PeakMemoryUsage: 60.6 KiB (62016) - RowsReturned: 368 (368) - SerializeBatchTime: 0ns (0) - TotalTime: 73ms (73997831) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 10.4 KiB (10672) CodeGen - CodegenTime: 4ms (4999853) - CompileTime: 274ms (274991943) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 165 (165) - NumInstructions: 6,311 (6311) - OptimizationTime: 537ms (537984237) - PeakMemoryUsage: 3.1 MiB (3231232) - PrepareTime: 123ms (123996367) - TotalTime: 937ms (937972518) AGGREGATION_NODE (id=16) ExecOption: Streaming Preaggregation, Codegen Enabled - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 16,384 (16384) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 9 (9) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 9.2 MiB (9680000) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 368 (368) - RowsReturnedRate: 8 per second (8) - StreamingTime: 999.97us (999970) - TotalTime: 43.69s (43688719890) HASH_JOIN_NODE (id=15) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 1 (1) - BuildTime: 999.97us (999971) - InactiveTotalTime: 0ns (0) - LocalTime: 999.97us (999970) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.2 MiB (2266368) - ProbeRows: 388 (388) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 388 (388) - RowsReturnedRate: 8 per second (8) - TotalTime: 44.34s (44345700639) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=15) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 999.97us (999971) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=29) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 0ns (0) DataStreamReceiver - BytesReceived: 26 B (26) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=13) ExecOption: Probe Side Codegen Enabled - BuildRows: 1 (1) - BuildTime: 999.97us (999971) - InactiveTotalTime: 0ns (0) - LocalTime: 999.97us (999971) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2116864) - ProbeRows: 388 (388) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 388 (388) - RowsReturnedRate: 8 per second (8) - TotalTime: 44.34s (44344700669) Hash Join Builder (join_node_id=13) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 1 (1) - HashBuckets: 1 (1) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=28) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - TotalTime: 42.78s (42775746641) DataStreamReceiver - BytesReceived: 13 B (13) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 42.71s (42710748546) - InactiveTotalTime: 42.78s (42775746641) - PeakMemoryUsage: 4.0 KiB (4104) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=10) Hdfs split stats (:<# splits>/): 6:1192/1.30 GB 9:1006/1.10 GB 4:1114/1.51 GB 1:1034/1.58 GB 3:1027/1.36 GB 0:1107/1.41 GB 10:1172/1.41 GB 7:1199/1.32 GB 5:1084/1.41 GB 8:1087/1.65 GB 11:1239/1.43 GB 2:1261/2.02 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 6 out of 6 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:6 PARQUET/Unknown(Skipped):13516 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 1.00 (1.0) - BytesRead: 5.1 MiB (5389046) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 5.1 MiB (5389046) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 5.1 MiB (5389046) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 21 (21) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 5 (5) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 6 (6) - NumRowGroups: 6 (6) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 3 (3) - PeakMemoryUsage: 3.3 MiB (3437662) - PerReadThreadRawHdfsThroughput: 20.6 MiB/s (21643389) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 231ms (231993203) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 658,522 (658522) - RowsReturned: 388 (388) - RowsReturnedRate: 247 per second (247) - ScanRangesComplete: 13,522 (13522) - ScannerThreadsInvoluntaryContextSwitches: 6 (6) - ScannerThreadsTotalWallClockTime: 231ms (231993203) - MaterializeTupleTime(*): 49ms (49998535) - ScannerThreadsSysTime: 2ms (2988000) - ScannerThreadsUserTime: 52ms (52789000) - ScannerThreadsVoluntaryContextSwitches: 28 (28) - TotalRawHdfsReadTime(*): 248ms (248992704) - TotalReadThroughput: 62.4 KiB/s (63924) - TotalTime: 1.57s (1567954057) Filter 10 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 658,522 (658522) - Rows rejected: 658,106 (658106) - Rows total: 658,522 (658522) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Filter 12 (1.00 MB) - Files processed: 13,522 (13522) - Files rejected: 13,516 (13516) - Files total: 13,522 (13522) - InactiveTotalTime: 0ns (0) - RowGroups processed: 3 (3) - RowGroups rejected: 0 (0) - RowGroups total: 3 (3) - Rows processed: 416 (416) - Rows rejected: 0 (0) - Rows total: 416 (416) - Splits processed: 6 (6) - Splits rejected: 0 (0) - Splits total: 6 (6) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 128.0 KiB (131072) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 128.0 KiB (131072) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 8.5 MiB (8912896) - CumulativeAllocations: 17 (17) - InactiveTotalTime: 0ns (0) - PeakReservation: 9.0 MiB (9437184) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.5 MiB (8912896) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Averaged Fragment F07 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:44s881ms max:44s881ms mean: 44s881ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 201.6 KiB (206416) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 44.78s (44779615137) - TotalNetworkSendTime: 3ms (3999965) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 44.85s (44847614552) - TotalThreadsSysTime: 77.00us (77000) - TotalThreadsUserTime: 651.00us (651000) - TotalThreadsVoluntaryContextSwitches: 14 (14) - TotalTime: 44.87s (44871614346) Fragment Instance Lifecycle Timings - ExecTime: 67ms (67999415) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 44.78s (44779615137) - ExecTreeOpenTime: 44.78s (44779615137) - PrepareTime: 23ms (23999794) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=28) - BytesSent: 78 B (78) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 38.1 KiB/s (39000) - PeakMemoryUsage: 153.6 KiB (157248) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 1ms (1999983) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 78 B (78) CodeGen - CodegenTime: 0ns (0) - CompileTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - PrepareTime: 20ms (20999819) - TotalTime: 21ms (21999811) AGGREGATION_NODE (id=27) - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 40.0 KiB (40960) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 44.78s (44779615137) EXCHANGE_NODE (id=26) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 6 (6) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.78s (44779615137) DataStreamReceiver - BytesReceived: 78 B (78) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 10.14s (10143912817) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 4.0 KiB (4112) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Fragment F07 Instance 68440bc099e1b843:8baf45150000001a (host=ip-10-1-28-130.ec2.internal:22000) - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 201.6 KiB (206416) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 44.78s (44779615137) - TotalNetworkSendTime: 3ms (3999965) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 44.85s (44847614552) - TotalThreadsSysTime: 77.00us (77000) - TotalThreadsUserTime: 651.00us (651000) - TotalThreadsVoluntaryContextSwitches: 14 (14) - TotalTime: 44.87s (44871614346) Fragment Instance Lifecycle Timings - ExecTime: 67ms (67999415) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 44.78s (44779615137) - ExecTreeOpenTime: 44.78s (44779615137) - PrepareTime: 23ms (23999794) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=28) - BytesSent: 78 B (78) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 38.1 KiB/s (39000) - PeakMemoryUsage: 153.6 KiB (157248) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 1ms (1999983) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 78 B (78) CodeGen - CodegenTime: 0ns (0) - CompileTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - PrepareTime: 20ms (20999819) - TotalTime: 21ms (21999811) AGGREGATION_NODE (id=27) - BuildTime: 0ns (0) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 40.0 KiB (40960) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 44.78s (44779615137) EXCHANGE_NODE (id=26) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 6 (6) - RowsReturnedRate: 0 per second (0) - TotalTime: 44.78s (44779615137) DataStreamReceiver - BytesReceived: 78 B (78) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 10.14s (10143912817) - InactiveTotalTime: 44.78s (44779615137) - PeakMemoryUsage: 4.0 KiB (4112) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Averaged Fragment F06 split sizes: min: 15.06 GB, max: 17.50 GB, avg: 16.91 GB, stddev: 938.99 MB completion times: min:10s169ms max:44s629ms mean: 18s090ms stddev:11s982ms execution rates: min:401.53 MB/sec max:1.72 GB/sec mean:1.18 GB/sec stddev:420.11 MB/sec num instances: 6 - AverageThreadTokens: 3.03 (3.0343411069217523) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.8 MiB (1895018) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 303.8 MiB (318598954) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 1ms (1166666) - TotalStorageWaitTime: 21.92s (21924815674) - TotalThreadsInvoluntaryContextSwitches: 284 (284) - TotalThreadsTotalWallClockTime: 50.71s (50709294842) - TotalThreadsSysTime: 2.20s (2199823333) - TotalThreadsUserTime: 13.39s (13392606666) - TotalThreadsVoluntaryContextSwitches: 395,095 (395095) - TotalTime: 18.04s (18039149055) Fragment Instance Lifecycle Timings - ExecTime: 1ms (1666669) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 17.84s (17839818830) - ExecTreeOpenTime: 17.80s (17798486016) - PrepareTime: 197ms (197663555) - ExecTreePrepareTime: 136ms (136164594) - TotalTime: 0ns (0) DataStreamSender (dst_id=26) - BytesSent: 13 B (13) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 4.6 KiB/s (4685) - PeakMemoryUsage: 25.6 KiB (26208) - RowsReturned: 1 (1) - SerializeBatchTime: 166.66us (166665) - TotalTime: 4ms (4666597) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13 B (13) CodeGen - CodegenTime: 1ms (1666650) - CompileTime: 16ms (16833058) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 16 (16) - NumInstructions: 232 (232) - OptimizationTime: 22ms (22833092) - PeakMemoryUsage: 116.0 KiB (118784) - PrepareTime: 24ms (24999848) - TotalTime: 65ms (65665997) AGGREGATION_NODE (id=12) - BuildTime: 5.74s (5739484803) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.0 KiB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 17.93s (17934650611) HDFS_SCAN_NODE (id=11) - AverageHdfsReadThreadConcurrency: 0.01 (0.013423645320197046) - AverageScannerThreadConcurrency: 2.05 (2.047619047619048) - BytesRead: 542.9 MiB (569282723) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 542.9 MiB (569282723) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 542.9 MiB (569282723) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 10,252 (10252) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 1.8 MiB (1843882) - PerReadThreadRawHdfsThroughput: 1.4 GiB/s (1459624841) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 10.09s (10086337598) - RowBatchQueuePutWaitTime: 8.16s (8162001775) - RowsRead: 1,686,971,481,464 (1686971481464) - RowsReturned: 1,358,862,882 (1358862882) - RowsReturnedRate: 182740926 per second (182740926) - ScanRangesComplete: 10,252 (10252) - ScannerThreadsInvoluntaryContextSwitches: 11 (11) - ScannerThreadsTotalWallClockTime: 32.87s (32867809341) - MaterializeTupleTime(*): 886ms (886497414) - ScannerThreadsSysTime: 892ms (892672166) - ScannerThreadsUserTime: 6.93s (6932907166) - ScannerThreadsVoluntaryContextSwitches: 386,697 (386697) - TotalRawHdfsReadTime(*): 407ms (407164896) - TotalReadThroughput: 36.9 MiB/s (38652101) - TotalTime: 12.05s (12048164497) Fragment F06 Instance 68440bc099e1b843:8baf451500000017 (host=ip-10-1-27-82.ec2.internal:22000) Hdfs split stats (:<# splits>/): 6:1192/1.30 GB 9:1006/1.10 GB 4:1114/1.51 GB 1:1034/1.58 GB 3:1027/1.36 GB 0:1107/1.41 GB 10:1172/1.41 GB 7:1199/1.32 GB 5:1084/1.41 GB 8:1087/1.65 GB 11:1239/1.43 GB 2:1261/2.02 GB - AverageThreadTokens: 2.32 (2.3181818181818183) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.8 MiB (1865664) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 303.4 MiB (318098845) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 999.97us (999971) - TotalStorageWaitTime: 49.34s (49337554383) - TotalThreadsInvoluntaryContextSwitches: 460 (460) - TotalThreadsTotalWallClockTime: 1.7m (102458997883) - TotalThreadsSysTime: 1.40s (1396932000) - TotalThreadsUserTime: 12.96s (12956559000) - TotalThreadsVoluntaryContextSwitches: 221,266 (221266) - TotalTime: 44.38s (44382699557) Fragment Instance Lifecycle Timings - ExecTime: 999.97us (999971) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 43.68s (43682720067) - ExecTreeOpenTime: 43.56s (43563723554) - PrepareTime: 698ms (698979519) - ExecTreePrepareTime: 484ms (484985789) - TotalTime: 0ns (0) DataStreamSender (dst_id=26) - BytesSent: 13 B (13) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 1.3 KiB/s (1300) - PeakMemoryUsage: 25.6 KiB (26208) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 9ms (9999707) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13 B (13) CodeGen - CodegenTime: 3ms (3999883) - CompileTime: 59ms (59998242) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 16 (16) - NumInstructions: 232 (232) - OptimizationTime: 56ms (56998329) - PeakMemoryUsage: 116.0 KiB (118784) - PrepareTime: 40ms (40998799) - TotalTime: 158ms (158995341) AGGREGATION_NODE (id=12) ExecOption: Codegen Enabled - BuildTime: 6.22s (6222817677) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.0 KiB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 44.05s (44048709343) HDFS_SCAN_NODE (id=11) Hdfs split stats (:<# splits>/): 6:1192/1.30 GB 9:1006/1.10 GB 4:1114/1.51 GB 1:1034/1.58 GB 3:1027/1.36 GB 0:1107/1.41 GB 10:1172/1.41 GB 7:1199/1.32 GB 5:1084/1.41 GB 8:1087/1.65 GB 11:1239/1.43 GB 2:1261/2.02 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 13522 out of 13522 Hdfs Read Thread Concurrency Bucket: 0:97.7% 1:2.299% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:13522 - AverageHdfsReadThreadConcurrency: 0.01 (0.011494252873563218) - AverageScannerThreadConcurrency: 1.33 (1.3333333333333333) - BytesRead: 603.4 MiB (632702451) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 603.4 MiB (632702451) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 603.4 MiB (632702451) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 13,522 (13522) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 3 (3) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 1.7 MiB (1814528) - PerReadThreadRawHdfsThroughput: 960.8 MiB/s (1007517503) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 35.50s (35495959962) - RowBatchQueuePutWaitTime: 6.08s (6078821870) - RowsRead: 1,678,564,198,007 (1678564198007) - RowsReturned: 1,401,658,999 (1401658999) - RowsReturnedRate: 36906268 per second (36906268) - ScanRangesComplete: 13,522 (13522) - ScannerThreadsInvoluntaryContextSwitches: 32 (32) - ScannerThreadsTotalWallClockTime: 58.78s (58775277845) - MaterializeTupleTime(*): 973ms (973971459) - ScannerThreadsSysTime: 634ms (634558000) - ScannerThreadsUserTime: 5.76s (5764274000) - ScannerThreadsVoluntaryContextSwitches: 207,726 (207726) - TotalRawHdfsReadTime(*): 627ms (627981597) - TotalReadThroughput: 13.5 MiB/s (14162708) - TotalTime: 37.98s (37978887180) Instance 68440bc099e1b843:8baf451500000014 (host=ip-10-1-20-86.ec2.internal:22000) Hdfs split stats (:<# splits>/): 0:760/1.06 GB 7:835/1.59 GB 2:822/1.65 GB 8:808/1.40 GB 1:799/1.41 GB 6:827/1.55 GB 10:850/1.47 GB 11:821/1.48 GB 9:747/1.19 GB 5:753/1.39 GB 3:707/1.01 GB 4:775/1.22 GB - AverageThreadTokens: 2.94 (2.935483870967742) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.8 MiB (1849280) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 312.0 MiB (327162809) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 999.99us (999993) - TotalStorageWaitTime: 20.34s (20340854257) - TotalThreadsInvoluntaryContextSwitches: 269 (269) - TotalThreadsTotalWallClockTime: 45.70s (45700672546) - TotalThreadsSysTime: 2.34s (2338100000) - TotalThreadsUserTime: 12.89s (12889380000) - TotalThreadsVoluntaryContextSwitches: 407,827 (407827) - TotalTime: 15.41s (15408889592) Fragment Instance Lifecycle Timings - ExecTime: 999.99us (999993) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 15.30s (15298890380) - ExecTreeOpenTime: 15.27s (15268890595) - PrepareTime: 108ms (108999219) - ExecTreePrepareTime: 45ms (45999670) - TotalTime: 0ns (0) DataStreamSender (dst_id=26) - BytesSent: 13 B (13) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 812 B/s (812) - PeakMemoryUsage: 25.6 KiB (26208) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 15ms (15999885) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13 B (13) CodeGen - CodegenTime: 999.99us (999992) - CompileTime: 10ms (10999922) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 16 (16) - NumInstructions: 232 (232) - OptimizationTime: 16ms (16999878) - PeakMemoryUsage: 116.0 KiB (118784) - PrepareTime: 23ms (23999828) - TotalTime: 52ms (52999621) AGGREGATION_NODE (id=12) ExecOption: Codegen Enabled - BuildTime: 5.50s (5505960575) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.0 KiB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 15.31s (15314890265) HDFS_SCAN_NODE (id=11) Hdfs split stats (:<# splits>/): 0:760/1.06 GB 7:835/1.59 GB 2:822/1.65 GB 8:808/1.40 GB 1:799/1.41 GB 6:827/1.55 GB 10:850/1.47 GB 11:821/1.48 GB 9:747/1.19 GB 5:753/1.39 GB 3:707/1.01 GB 4:775/1.22 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 9503 out of 9503 Hdfs Read Thread Concurrency Bucket: 0:96.67% 1:3.333% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:9504 - AverageHdfsReadThreadConcurrency: 0.03 (0.03333333333333333) - AverageScannerThreadConcurrency: 2.00 (2.0) - BytesRead: 508.9 MiB (533590682) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 508.9 MiB (533590682) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 508.9 MiB (533590682) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 9,503 (9503) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 1.7 MiB (1798144) - PerReadThreadRawHdfsThroughput: 1.4 GiB/s (1494662454) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 7.81s (7806944049) - RowBatchQueuePutWaitTime: 7.43s (7430946764) - RowsRead: 1,742,767,454,342 (1742767454342) - RowsReturned: 1,313,723,526 (1313723526) - RowsReturnedRate: 137104251 per second (137104251) - ScanRangesComplete: 9,504 (9504) - ScannerThreadsInvoluntaryContextSwitches: 6 (6) - ScannerThreadsTotalWallClockTime: 30.40s (30400782173) - MaterializeTupleTime(*): 856ms (856993862) - ScannerThreadsSysTime: 948ms (948734000) - ScannerThreadsUserTime: 6.78s (6781472000) - ScannerThreadsVoluntaryContextSwitches: 400,498 (400498) - TotalRawHdfsReadTime(*): 356ms (356997448) - TotalReadThroughput: 31.6 MiB/s (33110750) - TotalTime: 9.58s (9581931321) Instance 68440bc099e1b843:8baf451500000019 (host=ip-10-1-18-116.ec2.internal:22000) Hdfs split stats (:<# splits>/): 8:950/1.59 GB 2:1005/1.68 GB 9:1011/1.81 GB 11:1065/1.25 GB 0:1021/1.29 GB 10:979/1.75 GB 7:999/1.34 GB 3:972/1.46 GB 4:1002/1.44 GB 6:993/1.22 GB 1:925/1.43 GB 5:1066/1.23 GB - AverageThreadTokens: 3.54 (3.5357142857142856) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.9 MiB (1992640) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 305.3 MiB (320138763) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 1ms (1000024) - TotalStorageWaitTime: 21.33s (21334524206) - TotalThreadsInvoluntaryContextSwitches: 274 (274) - TotalThreadsTotalWallClockTime: 49.16s (49159207891) - TotalThreadsSysTime: 2.69s (2686169000) - TotalThreadsUserTime: 15.08s (15082263000) - TotalThreadsVoluntaryContextSwitches: 458,499 (458499) - TotalTime: 14.07s (14074345821) Fragment Instance Lifecycle Timings - ExecTime: 2ms (2000049) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 13.95s (13949342749) - ExecTreeOpenTime: 13.92s (13921342061) - PrepareTime: 123ms (123003023) - ExecTreePrepareTime: 100ms (100002457) - TotalTime: 0ns (0) DataStreamSender (dst_id=26) - BytesSent: 13 B (13) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 25.6 KiB (26208) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 0ns (0) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13 B (13) CodeGen - CodegenTime: 1ms (1000025) - CompileTime: 9ms (9000222) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 16 (16) - NumInstructions: 232 (232) - OptimizationTime: 17ms (17000417) - PeakMemoryUsage: 116.0 KiB (118784) - PrepareTime: 22ms (22000541) - TotalTime: 49ms (49001205) AGGREGATION_NODE (id=12) ExecOption: Codegen Enabled - BuildTime: 6.09s (6087149603) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.0 KiB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 14.02s (14021344518) HDFS_SCAN_NODE (id=11) Hdfs split stats (:<# splits>/): 8:950/1.59 GB 2:1005/1.68 GB 9:1011/1.81 GB 11:1065/1.25 GB 0:1021/1.29 GB 10:979/1.75 GB 7:999/1.34 GB 3:972/1.46 GB 4:1002/1.44 GB 6:993/1.22 GB 1:925/1.43 GB 5:1066/1.23 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 11988 out of 11988 Hdfs Read Thread Concurrency Bucket: 0:96.43% 1:3.571% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:11988 - AverageHdfsReadThreadConcurrency: 0.04 (0.03571428571428571) - AverageScannerThreadConcurrency: 2.54 (2.5357142857142856) - BytesRead: 585.3 MiB (613760298) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 585.3 MiB (613760298) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 585.3 MiB (613760298) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 11,988 (11988) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 3 (3) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 1.9 MiB (1941504) - PerReadThreadRawHdfsThroughput: 1.3 GiB/s (1360854139) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 5.57s (5570136869) - RowBatchQueuePutWaitTime: 10.77s (10766264565) - RowsRead: 1,759,448,578,513 (1759448578513) - RowsReturned: 1,404,322,257 (1404322257) - RowsReturnedRate: 182114869 per second (182114869) - ScanRangesComplete: 11,988 (11988) - ScannerThreadsInvoluntaryContextSwitches: 5 (5) - ScannerThreadsTotalWallClockTime: 35.21s (35207865093) - MaterializeTupleTime(*): 914ms (914022446) - ScannerThreadsSysTime: 1.05s (1053535000) - ScannerThreadsUserTime: 8.23s (8233612000) - ScannerThreadsVoluntaryContextSwitches: 449,636 (449636) - TotalRawHdfsReadTime(*): 451ms (451011082) - TotalReadThroughput: 40.8 MiB/s (42735803) - TotalTime: 7.71s (7711189447) Instance 68440bc099e1b843:8baf451500000015 (host=ip-10-1-21-97.ec2.internal:22000) Hdfs split stats (:<# splits>/): 7:838/1.15 GB 2:876/1.46 GB 5:798/1.29 GB 6:772/1.33 GB 11:836/1.50 GB 10:837/1.65 GB 1:769/1.21 GB 4:739/1.10 GB 8:781/1003.47 MB 9:778/1.35 GB 3:748/1.10 GB 0:774/970.37 MB - AverageThreadTokens: 3.42 (3.4166666666666665) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.8 MiB (1882048) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 301.4 MiB (316058244) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 3ms (3000007) - TotalStorageWaitTime: 17.81s (17811040183) - TotalThreadsInvoluntaryContextSwitches: 409 (409) - TotalThreadsTotalWallClockTime: 41.18s (41176092910) - TotalThreadsSysTime: 1.97s (1973847000) - TotalThreadsUserTime: 12.03s (12030337000) - TotalThreadsVoluntaryContextSwitches: 327,917 (327917) - TotalTime: 12.23s (12229027594) Fragment Instance Lifecycle Timings - ExecTime: 4ms (4000009) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 12.13s (12128027366) - ExecTreeOpenTime: 12.10s (12101027305) - PrepareTime: 97ms (97000219) - ExecTreePrepareTime: 72ms (72000162) - TotalTime: 0ns (0) DataStreamSender (dst_id=26) - BytesSent: 13 B (13) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 0 B/s (0) - PeakMemoryUsage: 25.6 KiB (26208) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 0ns (0) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13 B (13) CodeGen - CodegenTime: 1ms (1000002) - CompileTime: 8ms (8000018) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 16 (16) - NumInstructions: 232 (232) - OptimizationTime: 17ms (17000038) - PeakMemoryUsage: 116.0 KiB (118784) - PrepareTime: 23ms (23000052) - TotalTime: 49ms (49000111) AGGREGATION_NODE (id=12) ExecOption: Codegen Enabled - BuildTime: 5.28s (5278011915) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.0 KiB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 12.17s (12173027467) HDFS_SCAN_NODE (id=11) Hdfs split stats (:<# splits>/): 7:838/1.15 GB 2:876/1.46 GB 5:798/1.29 GB 6:772/1.33 GB 11:836/1.50 GB 10:837/1.65 GB 1:769/1.21 GB 4:739/1.10 GB 8:781/1003.47 MB 9:778/1.35 GB 3:748/1.10 GB 0:774/970.37 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 9546 out of 9546 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% File Formats: PARQUET/NONE:9546 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 2.42 (2.4166666666666665) - BytesRead: 498.0 MiB (522206654) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 498.0 MiB (522206654) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 498.0 MiB (522206654) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 9,546 (9546) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 3 (3) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 1.7 MiB (1830912) - PerReadThreadRawHdfsThroughput: 1.3 GiB/s (1446552518) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 5.06s (5064011437) - RowBatchQueuePutWaitTime: 8.57s (8568019321) - RowsRead: 1,401,382,381,206 (1401382381206) - RowsReturned: 1,213,141,654 (1213141654) - RowsReturnedRate: 181038489 per second (181038489) - ScanRangesComplete: 9,546 (9546) - ScannerThreadsInvoluntaryContextSwitches: 14 (14) - ScannerThreadsTotalWallClockTime: 29.04s (29044065535) - MaterializeTupleTime(*): 821ms (821001852) - ScannerThreadsSysTime: 800ms (800670000) - ScannerThreadsUserTime: 6.09s (6088325000) - ScannerThreadsVoluntaryContextSwitches: 321,080 (321080) - TotalRawHdfsReadTime(*): 361ms (361000826) - TotalReadThroughput: 40.3 MiB/s (42278526) - TotalTime: 6.70s (6701015110) Instance 68440bc099e1b843:8baf451500000016 (host=ip-10-1-28-130.ec2.internal:22000) Hdfs split stats (:<# splits>/): 5:682/1.37 GB 3:728/1.12 GB 1:730/1.43 GB 8:687/1.53 GB 6:647/1.57 GB 9:739/1.71 GB 4:687/1.42 GB 0:689/1.51 GB 2:610/1.28 GB 11:628/1.62 GB 7:623/1.52 GB 10:619/1.43 GB - AverageThreadTokens: 3.00 (3.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.8 MiB (1882048) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 13.04s (13039887918) - TotalThreadsInvoluntaryContextSwitches: 237 (237) - TotalThreadsTotalWallClockTime: 35.54s (35539694551) - TotalThreadsSysTime: 2.16s (2163146000) - TotalThreadsUserTime: 13.65s (13653438000) - TotalThreadsVoluntaryContextSwitches: 437,827 (437827) - TotalTime: 11.98s (11978897046) Fragment Instance Lifecycle Timings - ExecTime: 999.99us (999991) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 11.88s (11878897906) - ExecTreeOpenTime: 11.85s (11852898129) - PrepareTime: 98ms (98999149) - ExecTreePrepareTime: 74ms (74999355) - TotalTime: 0ns (0) DataStreamSender (dst_id=26) - BytesSent: 13 B (13) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 12.7 KiB/s (13000) - PeakMemoryUsage: 25.6 KiB (26208) - RowsReturned: 1 (1) - SerializeBatchTime: 999.99us (999991) - TotalTime: 999.99us (999991) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13 B (13) CodeGen - CodegenTime: 999.99us (999991) - CompileTime: 7ms (7999932) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 16 (16) - NumInstructions: 232 (232) - OptimizationTime: 16ms (16999854) - PeakMemoryUsage: 116.0 KiB (118784) - PrepareTime: 21ms (21999810) - TotalTime: 47ms (47999587) AGGREGATION_NODE (id=12) ExecOption: Codegen Enabled - BuildTime: 5.80s (5805950118) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.0 KiB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 11.93s (11927897484) HDFS_SCAN_NODE (id=11) Hdfs split stats (:<# splits>/): 5:682/1.37 GB 3:728/1.12 GB 1:730/1.43 GB 8:687/1.53 GB 6:647/1.57 GB 9:739/1.71 GB 4:687/1.42 GB 0:689/1.51 GB 2:610/1.28 GB 11:628/1.62 GB 7:623/1.52 GB 10:619/1.43 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 8068 out of 8068 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:8069 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 2.00 (2.0) - BytesRead: 521.6 MiB (546894230) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 521.6 MiB (546894230) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 521.6 MiB (546894230) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 8,068 (8068) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 1.7 MiB (1830912) - PerReadThreadRawHdfsThroughput: 1.6 GiB/s (1741716373) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 4.10s (4098964795) - RowBatchQueuePutWaitTime: 8.08s (8075930608) - RowsRead: 1,764,202,602,788 (1764202602788) - RowsReturned: 1,414,115,620 (1414115620) - RowsReturnedRate: 238229073 per second (238229073) - ScanRangesComplete: 8,069 (8069) - ScannerThreadsInvoluntaryContextSwitches: 7 (7) - ScannerThreadsTotalWallClockTime: 23.66s (23659796654) - MaterializeTupleTime(*): 924ms (924992041) - ScannerThreadsSysTime: 804ms (804295000) - ScannerThreadsUserTime: 7.18s (7175737000) - ScannerThreadsVoluntaryContextSwitches: 431,054 (431054) - TotalRawHdfsReadTime(*): 313ms (313997295) - TotalReadThroughput: 42.9 MiB/s (45004381) - TotalTime: 5.94s (5935948964) Instance 68440bc099e1b843:8baf451500000018 (host=ip-10-1-16-98.ec2.internal:22000) Hdfs split stats (:<# splits>/): 5:708/1.57 GB 0:786/1.72 GB 1:738/1.48 GB 11:682/1.17 GB 3:690/1.34 GB 8:762/1.80 GB 2:739/1.28 GB 7:776/1.43 GB 6:742/1.27 GB 4:756/1.63 GB 10:736/1.60 GB 9:771/1.21 GB - AverageThreadTokens: 3.00 (3.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.8 MiB (1898432) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 310.3 MiB (325335426) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 1ms (1000004) - TotalStorageWaitTime: 9.68s (9685033101) - TotalThreadsInvoluntaryContextSwitches: 57 (57) - TotalThreadsTotalWallClockTime: 30.22s (30221103272) - TotalThreadsSysTime: 2.64s (2640746000) - TotalThreadsUserTime: 13.74s (13743663000) - TotalThreadsVoluntaryContextSwitches: 517,239 (517239) - TotalTime: 10.16s (10161034723) Fragment Instance Lifecycle Timings - ExecTime: 1ms (1000004) - ExecTreeExecTime: 0ns (0) - InactiveTotalTime: 0ns (0) - OpenTime: 10.10s (10101034517) - ExecTreeOpenTime: 10.08s (10083034456) - PrepareTime: 59ms (59000202) - ExecTreePrepareTime: 39ms (39000133) - TotalTime: 0ns (0) DataStreamSender (dst_id=26) - BytesSent: 13 B (13) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 12.7 KiB/s (12999) - PeakMemoryUsage: 25.6 KiB (26208) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 1ms (1000004) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 13 B (13) CodeGen - CodegenTime: 2ms (2000008) - CompileTime: 5ms (5000017) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 16 (16) - NumInstructions: 232 (232) - OptimizationTime: 12ms (12000041) - PeakMemoryUsage: 116.0 KiB (118784) - PrepareTime: 18ms (18000061) - TotalTime: 36ms (36000122) AGGREGATION_NODE (id=12) ExecOption: Codegen Enabled - BuildTime: 5.54s (5537018930) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - HashBuckets: 0 (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.0 KiB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 per second (0) - SpilledPartitions: 0 (0) - TotalTime: 10.12s (10122034589) HDFS_SCAN_NODE (id=11) Hdfs split stats (:<# splits>/): 5:708/1.57 GB 0:786/1.72 GB 1:738/1.48 GB 11:682/1.17 GB 3:690/1.34 GB 8:762/1.80 GB 2:739/1.28 GB 7:776/1.43 GB 6:742/1.27 GB 4:756/1.63 GB 10:736/1.60 GB 9:771/1.21 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 8886 out of 8886 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:8886 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 2.00 (2.0) - BytesRead: 540.3 MiB (566542025) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 540.3 MiB (566542025) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 540.3 MiB (566542025) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 8,886 (8886) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 1.8 MiB (1847296) - PerReadThreadRawHdfsThroughput: 1.6 GiB/s (1706446059) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 2.48s (2482008479) - RowBatchQueuePutWaitTime: 8.05s (8052027527) - RowsRead: 1,775,463,673,929 (1775463673929) - RowsReturned: 1,406,215,241 (1406215241) - RowsReturnedRate: 321052611 per second (321052611) - ScanRangesComplete: 8,886 (8886) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 20.12s (20119068751) - MaterializeTupleTime(*): 828ms (828002828) - ScannerThreadsSysTime: 1.11s (1114241000) - ScannerThreadsUserTime: 7.55s (7554023000) - ScannerThreadsVoluntaryContextSwitches: 510,191 (510191) - TotalRawHdfsReadTime(*): 332ms (332001133) - TotalReadThroughput: 52.1 MiB/s (54620439) - TotalTime: 4.38s (4380014960) Averaged Fragment F08 split sizes: min: 521.09 KB, max: 521.09 KB, avg: 521.09 KB, stddev: 0 completion times: min:1s102ms max:1s102ms mean: 1s102ms stddev:0.000ns execution rates: min:472.44 KB/sec max:472.44 KB/sec mean:472.44 KB/sec stddev:0.60 B/sec num instances: 1 - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 481.6 KiB (493120) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 269.7 MiB (282791402) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 843ms (843992746) - TotalStorageWaitTime: 59ms (59999484) - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1.10s (1102990520) - TotalThreadsSysTime: 65.00us (65000) - TotalThreadsUserTime: 100ms (100192000) - TotalThreadsVoluntaryContextSwitches: 35 (35) - TotalTime: 1.09s (1086990658) Fragment Instance Lifecycle Timings - ExecTime: 938ms (938991930) - ExecTreeExecTime: 63ms (63999450) - InactiveTotalTime: 0ns (0) - OpenTime: 100ms (100999132) - ExecTreeOpenTime: 0ns (0) - PrepareTime: 46ms (46999596) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=29) - BytesSent: 156 B (156) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 16.9 KiB/s (17333) - PeakMemoryUsage: 36.6 KiB (37440) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 8ms (8999923) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 174 B (174) CodeGen - CodegenTime: 3ms (3999965) - CompileTime: 21ms (21999811) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 73 (73) - NumInstructions: 874 (874) - OptimizationTime: 75ms (75999347) - PeakMemoryUsage: 437.0 KiB (447488) - PrepareTime: 38ms (38999664) - TotalTime: 136ms (136998822) HDFS_SCAN_NODE (id=14) - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 212.5 KiB (217644) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 212.5 KiB (217644) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 212.5 KiB (217644) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 3 (3) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 1 (1) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 257.2 KiB (263381) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 62ms (62999458) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 10,000 (10000) - RowsReturned: 1 (1) - RowsReturnedRate: 15 per second (15) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 62ms (62999458) - MaterializeTupleTime(*): 1ms (1999983) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 2ms (2986000) - ScannerThreadsVoluntaryContextSwitches: 15 (15) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 63ms (63999450) Fragment F08 Instance 68440bc099e1b843:8baf45150000001b (host=ip-10-1-28-130.ec2.internal:22000) Hdfs split stats (:<# splits>/): 5:1/521.09 KB - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 481.6 KiB (493120) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 269.7 MiB (282791402) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 843ms (843992746) - TotalStorageWaitTime: 59ms (59999484) - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1.10s (1102990520) - TotalThreadsSysTime: 65.00us (65000) - TotalThreadsUserTime: 100ms (100192000) - TotalThreadsVoluntaryContextSwitches: 35 (35) - TotalTime: 1.09s (1086990658) Fragment Instance Lifecycle Timings - ExecTime: 938ms (938991930) - ExecTreeExecTime: 63ms (63999450) - InactiveTotalTime: 0ns (0) - OpenTime: 100ms (100999132) - ExecTreeOpenTime: 0ns (0) - PrepareTime: 46ms (46999596) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=29) - BytesSent: 156 B (156) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 16.9 KiB/s (17333) - PeakMemoryUsage: 36.6 KiB (37440) - RowsReturned: 1 (1) - SerializeBatchTime: 0ns (0) - TotalTime: 8ms (8999923) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 174 B (174) CodeGen - CodegenTime: 3ms (3999965) - CompileTime: 21ms (21999811) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 73 (73) - NumInstructions: 874 (874) - OptimizationTime: 75ms (75999347) - PeakMemoryUsage: 437.0 KiB (447488) - PrepareTime: 38ms (38999664) - TotalTime: 136ms (136998822) HDFS_SCAN_NODE (id=14) Hdfs split stats (:<# splits>/): 5:1/521.09 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/SNAPPY:1 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 212.5 KiB (217644) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 212.5 KiB (217644) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 212.5 KiB (217644) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 3 (3) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 1 (1) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 257.2 KiB (263381) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 62ms (62999458) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 10,000 (10000) - RowsReturned: 1 (1) - RowsReturnedRate: 15 per second (15) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 62ms (62999458) - MaterializeTupleTime(*): 1ms (1999983) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 2ms (2986000) - ScannerThreadsVoluntaryContextSwitches: 15 (15) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 63ms (63999450) Averaged Fragment F10 split sizes: min: 484.83 MB, max: 564.27 MB, avg: 508.95 MB, stddev: 27.24 MB completion times: min:1s260ms max:2s045ms mean: 1s471ms stddev:273.006ms execution rates: min:245.33 MB/sec max:427.16 MB/sec mean:355.94 MB/sec stddev:58.72 MB/sec num instances: 6 - AverageThreadTokens: 1.83 (1.8333333333333333) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 64.0 MiB (67142294) - PeakReservation: 1.9 MiB (2031616) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 303.8 MiB (318598954) - RowsProduced: 1,901 (1901) - TotalNetworkReceiveTime: 542ms (542168835) - TotalNetworkSendTime: 17ms (17333128) - TotalStorageWaitTime: 241ms (241500589) - TotalThreadsInvoluntaryContextSwitches: 41 (41) - TotalThreadsTotalWallClockTime: 2.41s (2406165809) - TotalThreadsSysTime: 93ms (93996166) - TotalThreadsUserTime: 517ms (517404000) - TotalThreadsVoluntaryContextSwitches: 197 (197) - TotalTime: 1.39s (1390662761) Fragment Instance Lifecycle Timings - ExecTime: 243ms (243331722) - ExecTreeExecTime: 199ms (199998601) - InactiveTotalTime: 0ns (0) - OpenTime: 1.07s (1067832413) - ExecTreeOpenTime: 704ms (704168166) - PrepareTime: 79ms (79498625) - ExecTreePrepareTime: 166.67us (166667) - TotalTime: 0ns (0) DataStreamSender (dst_id=34) - BytesSent: 834.8 KiB (854827) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 479.3 MiB/s (502537258) - OverallThroughput: 82.5 MiB/s (86535143) - PeakMemoryUsage: 23.2 KiB (23808) - RowsReturned: 1,901 (1901) - SerializeBatchTime: 833.33us (833328) - TotalTime: 37ms (37165705) - TransmitDataRPCTime: 666.66us (666659) - UncompressedRowBatchSize: 1.4 MiB (1442984) CodeGen - CodegenTime: 2ms (2166658) - CompileTime: 111ms (111666288) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 93 (93) - NumInstructions: 3,001 (3001) - OptimizationTime: 249ms (249164657) - PeakMemoryUsage: 1.5 MiB (1536512) - PrepareTime: 33ms (33666353) - TotalTime: 395ms (395163961) HASH_JOIN_NODE (id=2) - BuildRows: 4 (4) - BuildTime: 1ms (1499995) - InactiveTotalTime: 0ns (0) - LocalTime: 48ms (48666391) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 10.1 MiB (10608020) - ProbeRows: 1,901 (1901) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 1,901 (1901) - RowsReturnedRate: 2438 per second (2438) - TotalTime: 904ms (904333430) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 256.0 KiB (262144) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 256.0 KiB (262144) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=2) - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 4 (4) - HashBuckets: 6 (6) - HashCollisions: 0 (0) - HashTablesBuildTime: 333.33us (333332) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 50 (50) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 21.1 KiB (21632) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=33) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 4 (4) - RowsReturnedRate: 4 per second (4) - TotalTime: 542ms (542168835) DataStreamReceiver - BytesReceived: 57 B (57) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 524ms (524168828) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 4.0 KiB (4128) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=0) - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.58 (0.5833333333333334) - BytesRead: 114.4 MiB (119909544) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 114.4 MiB (119909544) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 114.4 MiB (119909544) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 89 (89) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 9 (9) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 57.1 MiB (59913978) - PerReadThreadRawHdfsThroughput: 1.9 GiB/s (2037640400) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 312ms (312164875) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,302,960 (2302960) - RowsReturned: 1,901 (1901) - RowsReturnedRate: 8081 per second (8081) - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 25 (25) - ScannerThreadsTotalWallClockTime: 551ms (551166181) - MaterializeTupleTime(*): 169ms (169166097) - ScannerThreadsSysTime: 72ms (72673500) - ScannerThreadsUserTime: 210ms (210175833) - ScannerThreadsVoluntaryContextSwitches: 131 (131) - TotalRawHdfsReadTime(*): 59ms (59666494) - TotalReadThroughput: 32.7 MiB/s (34256481) - TotalTime: 313ms (313664870) Filter 14 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,302,960 (2302960) - Rows rejected: 2,300,160 (2300160) - Rows total: 2,302,960 (2302960) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Fragment F10 Instance 68440bc099e1b843:8baf45150000002a (host=ip-10-1-27-82.ec2.internal:22000) Hdfs split stats (:<# splits>/): 2:1/20.95 MB 1:1/10.72 MB 8:2/30.20 MB 9:4/50.55 MB 5:2/40.20 MB 0:2/39.94 MB 6:3/42.46 MB 7:1/25.99 MB 3:3/162.40 MB 10:3/78.53 MB Filter 14 arrival: 877ms - AverageThreadTokens: 1.33 (1.3333333333333333) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 63.9 MiB (67047604) - PeakReservation: 1.9 MiB (2031616) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 303.4 MiB (318098845) - RowsProduced: 5,596 (5596) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 50ms (50998506) - TotalStorageWaitTime: 351ms (351989688) - TotalThreadsInvoluntaryContextSwitches: 60 (60) - TotalThreadsTotalWallClockTime: 2.10s (2100938442) - TotalThreadsSysTime: 98ms (98293000) - TotalThreadsUserTime: 564ms (564055000) - TotalThreadsVoluntaryContextSwitches: 204 (204) - TotalTime: 1.63s (1633952124) Fragment Instance Lifecycle Timings - ExecTime: 611ms (611982069) - ExecTreeExecTime: 539ms (539984179) - InactiveTotalTime: 0ns (0) - OpenTime: 743ms (743978201) - ExecTreeOpenTime: 151ms (151995546) - PrepareTime: 277ms (277991854) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=34) - BytesSent: 2.4 MiB (2511222) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 2.3 GiB/s (2511297338) - OverallThroughput: 12.4 MiB/s (13011894) - PeakMemoryUsage: 23.2 KiB (23808) - RowsReturned: 5,596 (5596) - SerializeBatchTime: 999.97us (999971) - TotalTime: 192ms (192994344) - TransmitDataRPCTime: 999.97us (999970) - UncompressedRowBatchSize: 4.0 MiB (4246104) CodeGen - CodegenTime: 1ms (1999941) - CompileTime: 118ms (118996513) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 93 (93) - NumInstructions: 3,001 (3001) - OptimizationTime: 465ms (465986347) - PeakMemoryUsage: 1.5 MiB (1536512) - PrepareTime: 63ms (63998123) - TotalTime: 649ms (649980954) HASH_JOIN_NODE (id=2) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 4 (4) - BuildTime: 1ms (1999942) - InactiveTotalTime: 0ns (0) - LocalTime: 10ms (10999678) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 10.8 MiB (11328208) - ProbeRows: 5,596 (5596) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 5,596 (5596) - RowsReturnedRate: 8086 per second (8086) - TotalTime: 691ms (691979725) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 256.0 KiB (262144) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 256.0 KiB (262144) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=2) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 4 (4) - HashBuckets: 6 (6) - HashCollisions: 0 (0) - HashTablesBuildTime: 999.97us (999971) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 50 (50) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 21.1 KiB (21632) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=33) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 4 (4) - RowsReturnedRate: 0 per second (0) - TotalTime: 0ns (0) DataStreamReceiver - BytesReceived: 57 B (57) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 4.0 KiB (4128) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 2:1/20.95 MB 1:1/10.72 MB 8:2/30.20 MB 9:4/50.55 MB 5:2/40.20 MB 0:2/39.94 MB 6:3/42.46 MB 7:1/25.99 MB 3:3/162.40 MB 10:3/78.53 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 22 out of 22 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:22 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 1.00 (1.0) - BytesRead: 115.3 MiB (120881252) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 115.3 MiB (120881252) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 115.3 MiB (120881252) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 110 (110) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 10 (10) - NumRowGroups: 22 (22) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 50.7 MiB (53125141) - PerReadThreadRawHdfsThroughput: 1.7 GiB/s (1831587820) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 678ms (678980106) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,410,638 (2410638) - RowsReturned: 5,596 (5596) - RowsReturnedRate: 8217 per second (8217) - ScanRangesComplete: 22 (22) - ScannerThreadsInvoluntaryContextSwitches: 42 (42) - ScannerThreadsTotalWallClockTime: 741ms (741978260) - MaterializeTupleTime(*): 219ms (219993548) - ScannerThreadsSysTime: 83ms (83873000) - ScannerThreadsUserTime: 233ms (233860000) - ScannerThreadsVoluntaryContextSwitches: 108 (108) - TotalRawHdfsReadTime(*): 65ms (65998065) - TotalReadThroughput: 25.8 MiB/s (27010047) - TotalTime: 680ms (680980047) Filter 14 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,410,638 (2410638) - Rows rejected: 2,403,088 (2403088) - Rows total: 2,410,638 (2410638) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Instance 68440bc099e1b843:8baf451500000029 (host=ip-10-1-18-116.ec2.internal:22000) Hdfs split stats (:<# splits>/): 5:1/45.48 MB 7:1/22.04 MB 2:1/28.41 MB 6:1/20.19 MB 10:2/18.58 MB 8:1/8.00 MB 4:2/43.54 MB 0:1/66.32 MB 11:2/41.67 MB 3:2/40.87 MB 1:2/72.29 MB 9:1/91.71 MB Filter 14 arrival: 1s020ms - AverageThreadTokens: 2.00 (2.0) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 65.2 MiB (68359906) - PeakReservation: 1.9 MiB (2031616) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 305.3 MiB (320138763) - RowsProduced: 290 (290) - TotalNetworkReceiveTime: 702ms (702017250) - TotalNetworkSendTime: 9ms (9000221) - TotalStorageWaitTime: 635ms (635015600) - TotalThreadsInvoluntaryContextSwitches: 35 (35) - TotalThreadsTotalWallClockTime: 3.18s (3179078114) - TotalThreadsSysTime: 87ms (87994000) - TotalThreadsUserTime: 521ms (521264000) - TotalThreadsVoluntaryContextSwitches: 169 (169) - TotalTime: 1.54s (1541037865) Fragment Instance Lifecycle Timings - ExecTime: 380ms (380009337) - ExecTreeExecTime: 338ms (338008305) - InactiveTotalTime: 0ns (0) - OpenTime: 1.14s (1135027889) - ExecTreeOpenTime: 844ms (844020739) - PrepareTime: 26ms (26000639) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=34) - BytesSent: 126.3 KiB (129348) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 123.4 MiB/s (129344895) - PeakMemoryUsage: 23.2 KiB (23808) - RowsReturned: 290 (290) - SerializeBatchTime: 0ns (0) - TotalTime: 1ms (1000024) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 214.9 KiB (220020) CodeGen - CodegenTime: 2ms (2000049) - CompileTime: 104ms (104002555) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 93 (93) - NumInstructions: 3,001 (3001) - OptimizationTime: 185ms (185004546) - PeakMemoryUsage: 1.5 MiB (1536512) - PrepareTime: 20ms (20000491) - TotalTime: 309ms (309007592) HASH_JOIN_NODE (id=2) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 4 (4) - BuildTime: 2ms (2000049) - InactiveTotalTime: 0ns (0) - LocalTime: 14ms (14000343) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 10.0 MiB (10463946) - ProbeRows: 290 (290) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 290 (290) - RowsReturnedRate: 245 per second (245) - TotalTime: 1.18s (1181029019) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 256.0 KiB (262144) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 256.0 KiB (262144) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=2) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 4 (4) - HashBuckets: 6 (6) - HashCollisions: 0 (0) - HashTablesBuildTime: 1ms (1000025) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 50 (50) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 21.1 KiB (21632) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=33) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 4 (4) - RowsReturnedRate: 5 per second (5) - TotalTime: 702ms (702017250) DataStreamReceiver - BytesReceived: 57 B (57) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 693ms (693017029) - InactiveTotalTime: 702ms (702017250) - PeakMemoryUsage: 4.0 KiB (4128) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 5:1/45.48 MB 7:1/22.04 MB 2:1/28.41 MB 6:1/20.19 MB 10:2/18.58 MB 8:1/8.00 MB 4:2/43.54 MB 0:1/66.32 MB 11:2/41.67 MB 3:2/40.87 MB 1:2/72.29 MB 9:1/91.71 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 17 out of 17 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:17 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 1.00 (1.0) - BytesRead: 116.5 MiB (122107946) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 116.5 MiB (122107946) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 116.5 MiB (122107946) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 85 (85) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 62.2 MiB (65188067) - PerReadThreadRawHdfsThroughput: 1.8 GiB/s (1969434653) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 464ms (464011402) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,379,429 (2379429) - RowsReturned: 290 (290) - RowsReturnedRate: 623 per second (623) - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 32 (32) - ScannerThreadsTotalWallClockTime: 960ms (960023589) - MaterializeTupleTime(*): 193ms (193004739) - ScannerThreadsSysTime: 69ms (69066000) - ScannerThreadsUserTime: 232ms (232213000) - ScannerThreadsVoluntaryContextSwitches: 109 (109) - TotalRawHdfsReadTime(*): 62ms (62001522) - TotalReadThroughput: 67.8 MiB/s (71054462) - TotalTime: 465ms (465011426) Filter 14 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,379,429 (2379429) - Rows rejected: 2,377,419 (2377419) - Rows total: 2,379,429 (2379429) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Instance 68440bc099e1b843:8baf45150000002b (host=ip-10-1-20-86.ec2.internal:22000) Hdfs split stats (:<# splits>/): 6:1/538.83 KB 2:1/5.46 MB 3:2/52.71 MB 11:2/53.01 MB 1:2/33.44 MB 9:1/1020.42 KB 0:2/22.05 MB 8:3/196.55 MB 4:2/153.78 MB Filter 14 arrival: 1s075ms - AverageThreadTokens: 2.00 (2.0) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 73.3 MiB (76876466) - PeakReservation: 1.9 MiB (2031616) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 312.0 MiB (327162809) - RowsProduced: 61 (61) - TotalNetworkReceiveTime: 477ms (477996575) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 133ms (133999039) - TotalThreadsInvoluntaryContextSwitches: 50 (50) - TotalThreadsTotalWallClockTime: 2.19s (2187984322) - TotalThreadsSysTime: 101ms (101057000) - TotalThreadsUserTime: 521ms (521009000) - TotalThreadsVoluntaryContextSwitches: 180 (180) - TotalTime: 1.32s (1317990556) Fragment Instance Lifecycle Timings - ExecTime: 85ms (85999384) - ExecTreeExecTime: 52ms (52999621) - InactiveTotalTime: 0ns (0) - OpenTime: 1.19s (1191991459) - ExecTreeOpenTime: 846ms (846993931) - PrepareTime: 39ms (39999713) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=34) - BytesSent: 27.8 KiB (28422) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 27.1 MiB/s (28422198) - PeakMemoryUsage: 23.2 KiB (23808) - RowsReturned: 61 (61) - SerializeBatchTime: 0ns (0) - TotalTime: 999.99us (999993) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 45.2 KiB (46308) CodeGen - CodegenTime: 1ms (1999985) - CompileTime: 129ms (129999069) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 93 (93) - NumInstructions: 3,001 (3001) - OptimizationTime: 212ms (212998473) - PeakMemoryUsage: 1.5 MiB (1536512) - PrepareTime: 25ms (25999813) - TotalTime: 369ms (369997348) HASH_JOIN_NODE (id=2) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 4 (4) - BuildTime: 999.99us (999993) - InactiveTotalTime: 0ns (0) - LocalTime: 227ms (227998366) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 10.0 MiB (10464064) - ProbeRows: 61 (61) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 61 (61) - RowsReturnedRate: 67 per second (67) - TotalTime: 899ms (899993552) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 256.0 KiB (262144) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 256.0 KiB (262144) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=2) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 4 (4) - HashBuckets: 6 (6) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 50 (50) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 21.1 KiB (21632) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=33) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 4 (4) - RowsReturnedRate: 8 per second (8) - TotalTime: 477ms (477996575) DataStreamReceiver - BytesReceived: 57 B (57) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 450ms (450996769) - InactiveTotalTime: 477ms (477996575) - PeakMemoryUsage: 4.0 KiB (4128) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 6:1/538.83 KB 2:1/5.46 MB 3:2/52.71 MB 11:2/53.01 MB 1:2/33.44 MB 9:1/1020.42 KB 0:2/22.05 MB 8:3/196.55 MB 4:2/153.78 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 16 out of 16 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:16 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 1.00 (1.0) - BytesRead: 113.9 MiB (119480622) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 113.9 MiB (119480622) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 113.9 MiB (119480622) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 84 (84) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 9 (9) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 69.3 MiB (72664187) - PerReadThreadRawHdfsThroughput: 2.1 GiB/s (2254367564) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 193ms (193998611) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,215,077 (2215077) - RowsReturned: 61 (61) - RowsReturnedRate: 312 per second (312) - ScanRangesComplete: 16 (16) - ScannerThreadsInvoluntaryContextSwitches: 21 (21) - ScannerThreadsTotalWallClockTime: 430ms (430996911) - MaterializeTupleTime(*): 160ms (160998853) - ScannerThreadsSysTime: 75ms (75553000) - ScannerThreadsUserTime: 210ms (210121000) - ScannerThreadsVoluntaryContextSwitches: 133 (133) - TotalRawHdfsReadTime(*): 52ms (52999619) - TotalReadThroughput: 31.3 MiB/s (32796659) - TotalTime: 194ms (194998604) Filter 14 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,215,077 (2215077) - Rows rejected: 2,214,705 (2214705) - Rows total: 2,215,077 (2215077) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Instance 68440bc099e1b843:8baf45150000002e (host=ip-10-1-28-130.ec2.internal:22000) Hdfs split stats (:<# splits>/): 6:1/6.50 MB 8:3/96.75 MB 9:2/72.33 MB 7:3/82.19 MB 2:1/19.76 MB 11:1/6.05 MB 4:1/5.13 MB 3:2/21.53 MB 5:2/11.51 MB 1:1/8.41 MB 10:4/58.84 MB 0:4/175.26 MB Filter 14 arrival: 1s060ms - AverageThreadTokens: 2.00 (2.0) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 49.0 MiB (51392558) - PeakReservation: 1.9 MiB (2031616) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 290.7 MiB (304799637) - RowsProduced: 2,203 (2203) - TotalNetworkReceiveTime: 592ms (592994903) - TotalNetworkSendTime: 8ms (8999922) - TotalStorageWaitTime: 146ms (146998734) - TotalThreadsInvoluntaryContextSwitches: 51 (51) - TotalThreadsTotalWallClockTime: 2.25s (2254980619) - TotalThreadsSysTime: 94ms (94196000) - TotalThreadsUserTime: 561ms (561735000) - TotalThreadsVoluntaryContextSwitches: 253 (253) - TotalTime: 1.30s (1300988819) Fragment Instance Lifecycle Timings - ExecTime: 138ms (138998805) - ExecTreeExecTime: 112ms (112999030) - InactiveTotalTime: 0ns (0) - OpenTime: 1.09s (1089990632) - ExecTreeOpenTime: 695ms (695994019) - PrepareTime: 71ms (71999382) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=34) - BytesSent: 970.6 KiB (993888) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 473.9 MiB/s (496948224) - OverallThroughput: 52.7 MiB/s (55216472) - PeakMemoryUsage: 23.2 KiB (23808) - RowsReturned: 2,203 (2203) - SerializeBatchTime: 999.99us (999992) - TotalTime: 17ms (17999846) - TransmitDataRPCTime: 1ms (1999983) - UncompressedRowBatchSize: 1.6 MiB (1671996) CodeGen - CodegenTime: 3ms (3999966) - CompileTime: 112ms (112999028) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 93 (93) - NumInstructions: 3,001 (3001) - OptimizationTime: 277ms (277997611) - PeakMemoryUsage: 1.5 MiB (1536512) - PrepareTime: 48ms (48999579) - TotalTime: 439ms (439996218) HASH_JOIN_NODE (id=2) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 4 (4) - BuildTime: 1ms (1999983) - InactiveTotalTime: 0ns (0) - LocalTime: 11ms (11999898) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 10.0 MiB (10463979) - ProbeRows: 2,203 (2203) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 2,203 (2203) - RowsReturnedRate: 2723 per second (2723) - TotalTime: 808ms (808993049) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 256.0 KiB (262144) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 256.0 KiB (262144) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=2) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 4 (4) - HashBuckets: 6 (6) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 50 (50) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 21.1 KiB (21632) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=33) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 4 (4) - RowsReturnedRate: 6 per second (6) - TotalTime: 592ms (592994903) DataStreamReceiver - BytesReceived: 57 B (57) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 575ms (575995049) - InactiveTotalTime: 592ms (592994903) - PeakMemoryUsage: 4.0 KiB (4128) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 6:1/6.50 MB 8:3/96.75 MB 9:2/72.33 MB 7:3/82.19 MB 2:1/19.76 MB 11:1/6.05 MB 4:1/5.13 MB 3:2/21.53 MB 5:2/11.51 MB 1:1/8.41 MB 10:4/58.84 MB 0:4/175.26 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 25 out of 25 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:25 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 107.7 MiB (112953586) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 107.7 MiB (112953586) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 107.7 MiB (112953586) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 125 (125) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 12 (12) - NumRowGroups: 25 (25) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 43.4 MiB (45478242) - PerReadThreadRawHdfsThroughput: 1.6 GiB/s (1764914976) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 203ms (203998248) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,324,748 (2324748) - RowsReturned: 2,203 (2203) - RowsReturnedRate: 10799 per second (10799) - ScanRangesComplete: 25 (25) - ScannerThreadsInvoluntaryContextSwitches: 22 (22) - ScannerThreadsTotalWallClockTime: 430ms (430996296) - MaterializeTupleTime(*): 156ms (156998650) - ScannerThreadsSysTime: 71ms (71282000) - ScannerThreadsUserTime: 194ms (194108000) - ScannerThreadsVoluntaryContextSwitches: 170 (170) - TotalRawHdfsReadTime(*): 63ms (63999449) - TotalReadThroughput: 0 B/s (0) - TotalTime: 203ms (203998248) Filter 14 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,324,748 (2324748) - Rows rejected: 2,322,392 (2322392) - Rows total: 2,324,748 (2324748) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Instance 68440bc099e1b843:8baf45150000002c (host=ip-10-1-21-97.ec2.internal:22000) Hdfs split stats (:<# splits>/): 3:2/40.27 MB 11:2/62.14 MB 10:1/26.26 MB 1:1/26.16 MB 8:1/30.70 MB 4:3/186.89 MB 9:2/112.62 MB Filter 14 arrival: 1s028ms - AverageThreadTokens: 2.00 (2.0) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 62.3 MiB (65329446) - PeakReservation: 1.9 MiB (2031616) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 301.4 MiB (316058244) - RowsProduced: 14 (14) - TotalNetworkReceiveTime: 669ms (669001510) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 123ms (123000276) - TotalThreadsInvoluntaryContextSwitches: 45 (45) - TotalThreadsTotalWallClockTime: 2.37s (2370005346) - TotalThreadsSysTime: 109ms (109699000) - TotalThreadsUserTime: 516ms (516830000) - TotalThreadsVoluntaryContextSwitches: 193 (193) - TotalTime: 1.30s (1299002931) Fragment Instance Lifecycle Timings - ExecTime: 80ms (80000180) - ExecTreeExecTime: 52ms (52000116) - InactiveTotalTime: 0ns (0) - OpenTime: 1.18s (1182002667) - ExecTreeOpenTime: 861ms (861001943) - PrepareTime: 37ms (37000084) - ExecTreePrepareTime: 1ms (1000003) - TotalTime: 0ns (0) DataStreamSender (dst_id=34) - BytesSent: 6.8 KiB (6978) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 6.7 MiB/s (6977986) - OverallThroughput: 1.3 MiB/s (1395596) - PeakMemoryUsage: 23.2 KiB (23808) - RowsReturned: 14 (14) - SerializeBatchTime: 0ns (0) - TotalTime: 5ms (5000011) - TransmitDataRPCTime: 1ms (1000002) - UncompressedRowBatchSize: 10.5 KiB (10740) CodeGen - CodegenTime: 2ms (2000004) - CompileTime: 112ms (112000253) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 93 (93) - NumInstructions: 3,001 (3001) - OptimizationTime: 207ms (207000467) - PeakMemoryUsage: 1.5 MiB (1536512) - PrepareTime: 26ms (26000058) - TotalTime: 346ms (346000780) HASH_JOIN_NODE (id=2) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 4 (4) - BuildTime: 1ms (1000002) - InactiveTotalTime: 0ns (0) - LocalTime: 23ms (23000050) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 10.0 MiB (10463953) - ProbeRows: 14 (14) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 14 (14) - RowsReturnedRate: 15 per second (15) - TotalTime: 915ms (915002065) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 256.0 KiB (262144) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 256.0 KiB (262144) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=2) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 4 (4) - HashBuckets: 6 (6) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 50 (50) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 21.1 KiB (21632) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=33) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 4 (4) - RowsReturnedRate: 5 per second (5) - TotalTime: 669ms (669001510) DataStreamReceiver - BytesReceived: 57 B (57) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 644ms (644001453) - InactiveTotalTime: 669ms (669001510) - PeakMemoryUsage: 4.0 KiB (4128) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 3:2/40.27 MB 11:2/62.14 MB 10:1/26.26 MB 1:1/26.16 MB 8:1/30.70 MB 4:3/186.89 MB 9:2/112.62 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 12 out of 12 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% File Formats: PARQUET/NONE:12 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 125.9 MiB (132017281) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 125.9 MiB (132017281) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 125.9 MiB (132017281) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 60 (60) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 7 (7) - NumRowGroups: 12 (12) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 58.3 MiB (61117212) - PerReadThreadRawHdfsThroughput: 1.8 GiB/s (1970402679) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 218ms (218000493) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,289,654 (2289654) - RowsReturned: 14 (14) - RowsReturnedRate: 62 per second (62) - ScanRangesComplete: 12 (12) - ScannerThreadsInvoluntaryContextSwitches: 27 (27) - ScannerThreadsTotalWallClockTime: 438ms (438000987) - MaterializeTupleTime(*): 154ms (154000345) - ScannerThreadsSysTime: 80ms (80810000) - ScannerThreadsUserTime: 215ms (215433000) - ScannerThreadsVoluntaryContextSwitches: 138 (138) - TotalRawHdfsReadTime(*): 67ms (67000153) - TotalReadThroughput: 0 B/s (0) - TotalTime: 223ms (223000505) Filter 14 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,289,654 (2289654) - Rows rejected: 2,289,359 (2289359) - Rows total: 2,289,654 (2289654) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Instance 68440bc099e1b843:8baf45150000002d (host=ip-10-1-16-98.ec2.internal:22000) Hdfs split stats (:<# splits>/): 4:1/15.87 MB 7:1/26.19 MB 5:1/52.06 MB 3:2/68.83 MB 9:1/60.53 MB 11:1/21.82 MB 8:2/45.19 MB 6:3/60.65 MB 0:3/133.69 MB Filter 14 arrival: 1s075ms - AverageThreadTokens: 1.67 (1.6666666666666667) - BloomFilterBytes: 1.0 MiB (1048576) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 70.4 MiB (73847787) - PeakReservation: 1.9 MiB (2031616) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 310.3 MiB (325335426) - RowsProduced: 3,246 (3246) - TotalNetworkReceiveTime: 811ms (811002772) - TotalNetworkSendTime: 35ms (35000121) - TotalStorageWaitTime: 58ms (58000198) - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 2.34s (2344008011) - TotalThreadsSysTime: 72ms (72738000) - TotalThreadsUserTime: 419ms (419531000) - TotalThreadsVoluntaryContextSwitches: 187 (187) - TotalTime: 1.25s (1251004275) Fragment Instance Lifecycle Timings - ExecTime: 163ms (163000557) - ExecTreeExecTime: 104ms (104000356) - InactiveTotalTime: 0ns (0) - OpenTime: 1.06s (1064003635) - ExecTreeOpenTime: 825ms (825002819) - PrepareTime: 24ms (24000083) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=34) - BytesSent: 1.4 MiB (1459104) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 278.3 MiB/s (291819807) - PeakMemoryUsage: 23.2 KiB (23808) - RowsReturned: 3,246 (3246) - SerializeBatchTime: 3ms (3000010) - TotalTime: 5ms (5000017) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 2.3 MiB (2462736) CodeGen - CodegenTime: 1ms (1000003) - CompileTime: 92ms (92000314) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 93 (93) - NumInstructions: 3,001 (3001) - OptimizationTime: 146ms (146000499) - PeakMemoryUsage: 1.5 MiB (1536512) - PrepareTime: 17ms (17000058) - TotalTime: 256ms (256000875) HASH_JOIN_NODE (id=2) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 4 (4) - BuildTime: 1ms (1000003) - InactiveTotalTime: 0ns (0) - LocalTime: 4ms (4000013) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 10.0 MiB (10463972) - ProbeRows: 3,246 (3246) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 3,246 (3246) - RowsReturnedRate: 3494 per second (3494) - TotalTime: 929ms (929003175) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 256.0 KiB (262144) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 256.0 KiB (262144) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=2) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 4 (4) - HashBuckets: 6 (6) - HashCollisions: 0 (0) - HashTablesBuildTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 50 (50) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 21.1 KiB (21632) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=33) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 4 (4) - RowsReturnedRate: 4 per second (4) - TotalTime: 811ms (811002772) DataStreamReceiver - BytesReceived: 57 B (57) - DeserializeRowBatchTimer: 0ns (0) - FirstBatchArrivalWaitTime: 781ms (781002669) - InactiveTotalTime: 811ms (811002772) - PeakMemoryUsage: 4.0 KiB (4128) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 4:1/15.87 MB 7:1/26.19 MB 5:1/52.06 MB 3:2/68.83 MB 9:1/60.53 MB 11:1/21.82 MB 8:2/45.19 MB 6:3/60.65 MB 0:3/133.69 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 15 out of 15 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:15 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.50 (0.5) - BytesRead: 106.8 MiB (112016579) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 106.8 MiB (112016579) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 106.8 MiB (112016579) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 75 (75) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 9 (9) - NumRowGroups: 15 (15) - NumScannerThreadsStarted: 2 (2) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 59.0 MiB (61911019) - PerReadThreadRawHdfsThroughput: 2.3 GiB/s (2435134710) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 114ms (114000390) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 2,198,214 (2198214) - RowsReturned: 3,246 (3246) - RowsReturnedRate: 28473 per second (28473) - ScanRangesComplete: 15 (15) - ScannerThreadsInvoluntaryContextSwitches: 6 (6) - ScannerThreadsTotalWallClockTime: 305ms (305001044) - MaterializeTupleTime(*): 130ms (130000449) - ScannerThreadsSysTime: 55ms (55457000) - ScannerThreadsUserTime: 175ms (175320000) - ScannerThreadsVoluntaryContextSwitches: 131 (131) - TotalRawHdfsReadTime(*): 46ms (46000157) - TotalReadThroughput: 71.2 MiB/s (74677719) - TotalTime: 114ms (114000390) Filter 14 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 2,198,214 (2198214) - Rows rejected: 2,194,002 (2194002) - Rows total: 2,198,214 (2198214) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) Averaged Fragment F11 split sizes: min: 4.58 KB, max: 4.58 KB, avg: 4.58 KB, stddev: 0 completion times: min:924.992ms max:924.992ms mean: 924.992ms stddev:0.000ns execution rates: min:4.95 KB/sec max:4.95 KB/sec mean:4.95 KB/sec stddev:0.56 B/sec num instances: 1 - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 232.6 KiB (238141) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 245.7 MiB (257622744) - RowsProduced: 4 (4) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 17ms (17999472) - TotalStorageWaitTime: 28ms (28999151) - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 293ms (293991386) - TotalThreadsSysTime: 1ms (1732000) - TotalThreadsUserTime: 38ms (38643000) - TotalThreadsVoluntaryContextSwitches: 30 (30) - TotalTime: 494ms (494985496) Fragment Instance Lifecycle Timings - ExecTime: 110ms (110996747) - ExecTreeExecTime: 75ms (75997773) - InactiveTotalTime: 0ns (0) - OpenTime: 124ms (124996338) - ExecTreeOpenTime: 0ns (0) - PrepareTime: 258ms (258992411) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=33) - BytesSent: 342 B (342) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 2.2 KiB/s (2280) - PeakMemoryUsage: 59.1 KiB (60480) - RowsReturned: 4 (4) - SerializeBatchTime: 0ns (0) - TotalTime: 149ms (149995605) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 504 B (504) CodeGen - CodegenTime: 9ms (9999707) - CompileTime: 34ms (34998975) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 20 (20) - NumInstructions: 319 (319) - OptimizationTime: 55ms (55998359) - PeakMemoryUsage: 159.5 KiB (163328) - PrepareTime: 111ms (111996718) - TotalTime: 203ms (203994023) HDFS_SCAN_NODE (id=1) - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 4.8 KiB (4874) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 4.8 KiB (4874) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 4.8 KiB (4874) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 3 (3) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 1 (1) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 165.0 KiB (168982) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 57ms (57998301) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 8 (8) - RowsReturned: 4 (4) - RowsReturnedRate: 52 per second (52) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 57ms (57998301) - MaterializeTupleTime(*): 0ns (0) - ScannerThreadsSysTime: 629.00us (629000) - ScannerThreadsUserTime: 0ns (0) - ScannerThreadsVoluntaryContextSwitches: 9 (9) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 75ms (75997773) Fragment F11 Instance 68440bc099e1b843:8baf451500000028 (host=ip-10-1-27-82.ec2.internal:22000) Hdfs split stats (:<# splits>/): 7:1/4.58 KB - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 232.6 KiB (238141) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 245.7 MiB (257622744) - RowsProduced: 4 (4) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 17ms (17999472) - TotalStorageWaitTime: 28ms (28999151) - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 293ms (293991386) - TotalThreadsSysTime: 1ms (1732000) - TotalThreadsUserTime: 38ms (38643000) - TotalThreadsVoluntaryContextSwitches: 30 (30) - TotalTime: 494ms (494985496) Fragment Instance Lifecycle Timings - ExecTime: 110ms (110996747) - ExecTreeExecTime: 75ms (75997773) - InactiveTotalTime: 0ns (0) - OpenTime: 124ms (124996338) - ExecTreeOpenTime: 0ns (0) - PrepareTime: 258ms (258992411) - ExecTreePrepareTime: 0ns (0) - TotalTime: 0ns (0) DataStreamSender (dst_id=33) - BytesSent: 342 B (342) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 0 B/s (0) - OverallThroughput: 2.2 KiB/s (2280) - PeakMemoryUsage: 59.1 KiB (60480) - RowsReturned: 4 (4) - SerializeBatchTime: 0ns (0) - TotalTime: 149ms (149995605) - TransmitDataRPCTime: 0ns (0) - UncompressedRowBatchSize: 504 B (504) CodeGen - CodegenTime: 9ms (9999707) - CompileTime: 34ms (34998975) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1950884) - NumFunctions: 20 (20) - NumInstructions: 319 (319) - OptimizationTime: 55ms (55998359) - PeakMemoryUsage: 159.5 KiB (163328) - PrepareTime: 111ms (111996718) - TotalTime: 203ms (203994023) HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 7:1/4.58 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% File Formats: PARQUET/NONE:1 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 4.8 KiB (4874) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 4.8 KiB (4874) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 4.8 KiB (4874) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 3 (3) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumRowGroups: 1 (1) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 165.0 KiB (168982) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 57ms (57998301) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 8 (8) - RowsReturned: 4 (4) - RowsReturnedRate: 52 per second (52) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 57ms (57998301) - MaterializeTupleTime(*): 0ns (0) - ScannerThreadsSysTime: 629.00us (629000) - ScannerThreadsUserTime: 0ns (0) - ScannerThreadsVoluntaryContextSwitches: 9 (9) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 75ms (75997773)