Query (id=b149ee4c1d799434:62edf82a00000000) DEBUG MODE WARNING: Query profile created while running a DEBUG build of Impala. Use RELEASE builds to measure query performance. Summary Session ID: d440ca6c1062788:86d3b2848ede4899 Session Type: HIVESERVER2 HiveServer2 Protocol Version: V6 Start Time: 2017-05-31 23:26:28.640979000 End Time: 2017-05-31 23:31:46.438968000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.8.0-cdh5.11.0 DEBUG (build e09660de6b503a15f07e84b99b63e8e745854c34) User: redacted Connected User: redacted Delegated User: Network Address: redacted Default Db: tpcds_300_decimal_parquet Sql Statement: /* Mem: 133 MB. Coordinator: redacted-host8. */ select c_last_name, c_first_name, c_salutation, c_preferred_cust_flag, ss_ticket_number, cnt from (select ss_ticket_number, ss_customer_sk, count(*) cnt from store_sales, date_dim, store, household_demographics where store_sales.ss_sold_date_sk = date_dim.d_date_sk and store_sales.ss_store_sk = store.s_store_sk and store_sales.ss_hdemo_sk = household_demographics.hd_demo_sk and (date_dim.d_dom between 1 and 3 or date_dim.d_dom between 25 and 28) and (household_demographics.hd_buy_potential = '>10000' or household_demographics.hd_buy_potential = 'unknown') and household_demographics.hd_vehicle_count > 0 and (case when household_demographics.hd_vehicle_count > 0 then household_demographics.hd_dep_count / household_demographics.hd_vehicle_count else null end) > 1.2 and date_dim.d_year in (1998, 1998 + 1, 1998 + 2) and store.s_county in ('Saginaw County', 'Sumner County', 'Appanoose County', 'Daviess County', 'Fairfield County', 'Raleigh County', 'Ziebach County', 'Williamson County') and ss_sold_date_sk between 2450816 and 2451910 -- partition key filter group by ss_ticket_number, ss_customer_sk ) dn, customer where ss_customer_sk = c_customer_sk and cnt between 15 and 20 order by c_last_name, c_first_name, c_salutation, c_preferred_cust_flag desc limit 100000; Coordinator: redacted-host8:22000 Query Options (non default): ABORT_ON_ERROR=1,MEM_LIMIT=139460608,RUNTIME_FILTER_MODE=0 Plan: ---------------- Estimated Per-Host Requirements: Memory=388.54MB VCores=4 PLAN-ROOT SINK | 18:MERGING-EXCHANGE [UNPARTITIONED] | order by: c_last_name ASC, c_first_name ASC, c_salutation ASC, c_preferred_cust_flag DESC | limit: 100000 | hosts=8 per-host-mem=unavailable | tuple-ids=7 row-size=96B cardinality=29939 | 10:TOP-N [LIMIT=100000] | order by: c_last_name ASC, c_first_name ASC, c_salutation ASC, c_preferred_cust_flag DESC | hosts=8 per-host-mem=2.73MB | tuple-ids=7 row-size=96B cardinality=29939 | 09:HASH JOIN [INNER JOIN, PARTITIONED] | hash predicates: c_customer_sk = ss_customer_sk | hosts=8 per-host-mem=16.28MB | tuple-ids=6,4 row-size=112B cardinality=29939 | |--17:EXCHANGE [HASH(ss_customer_sk)] | | hosts=8 per-host-mem=0B | | tuple-ids=4 row-size=24B cardinality=5173436 | | | 15:AGGREGATE [FINALIZE] | | output: count:merge(*) | | group by: ss_ticket_number, ss_customer_sk | | having: count(*) <= 20, count(*) >= 15 | | hosts=8 per-host-mem=130.25MB | | tuple-ids=4 row-size=24B cardinality=5173436 | | | 14:EXCHANGE [HASH(ss_ticket_number,ss_customer_sk)] | | hosts=8 per-host-mem=0B | | tuple-ids=4 row-size=24B cardinality=5173436 | | | 07:AGGREGATE [STREAMING] | | output: count(*) | | group by: ss_ticket_number, ss_customer_sk | | hosts=8 per-host-mem=130.25MB | | tuple-ids=4 row-size=24B cardinality=5173436 | | | 06:HASH JOIN [INNER JOIN, BROADCAST] | | hash predicates: store_sales.ss_store_sk = store.s_store_sk | | hosts=8 per-host-mem=16.63KB | | tuple-ids=0,3,1,2 row-size=134B cardinality=5173436 | | | |--13:EXCHANGE [BROADCAST] | | | hosts=1 per-host-mem=0B | | | tuple-ids=2 row-size=39B cardinality=402 | | | | | 02:SCAN HDFS [tpcds_300_decimal_parquet.store, RANDOM] | | partitions=1/1 files=1 size=71.36KB | | predicates: store.s_county IN ('Saginaw County', 'Sumner County', 'Appanoose County', 'Daviess County', 'Fairfield County', 'Raleigh County', 'Ziebach County', 'Williamson County') | | table stats: 804 rows total | | column stats: all | | hosts=1 per-host-mem=32.00MB | | tuple-ids=2 row-size=39B cardinality=402 | | | 05:HASH JOIN [INNER JOIN, BROADCAST] | | hash predicates: store_sales.ss_sold_date_sk = date_dim.d_date_sk | | hosts=8 per-host-mem=6.08KB | | tuple-ids=0,3,1 row-size=96B cardinality=5173436 | | | |--12:EXCHANGE [BROADCAST] | | | hosts=1 per-host-mem=0B | | | tuple-ids=1 row-size=16B cardinality=354 | | | | | 01:SCAN HDFS [tpcds_300_decimal_parquet.date_dim, RANDOM] | | partitions=1/1 files=1 size=2.15MB | | predicates: date_dim.d_year IN (1998, 1999, 2000), tpcds_300_decimal_parquet.date_dim.d_date_sk <= 2451910, tpcds_300_decimal_parquet.date_dim.d_date_sk >= 2450816, (date_dim.d_dom >= 1 AND date_dim.d_dom <= 3 OR date_dim.d_dom >= 25 AND date_dim.d_dom <= 28) | | table stats: 73049 rows total | | column stats: all | | hosts=1 per-host-mem=48.00MB | | tuple-ids=1 row-size=16B cardinality=354 | | | 04:HASH JOIN [INNER JOIN, BROADCAST] | | hash predicates: store_sales.ss_hdemo_sk = household_demographics.hd_demo_sk | | hosts=8 per-host-mem=16.89KB | | tuple-ids=0,3 row-size=80B cardinality=27319040 | | | |--11:EXCHANGE [BROADCAST] | | | hosts=1 per-host-mem=0B | | | tuple-ids=3 row-size=40B cardinality=398 | | | | | 03:SCAN HDFS [tpcds_300_decimal_parquet.household_demographics, RANDOM] | | partitions=1/1 files=1 size=41.12KB | | predicates: household_demographics.hd_vehicle_count > 0, (CASE WHEN household_demographics.hd_vehicle_count > 0 THEN household_demographics.hd_dep_count / household_demographics.hd_vehicle_count ELSE NULL END) > 1.2, (household_demographics.hd_buy_potential = '>10000' OR household_demographics.hd_buy_potential = 'unknown') | | table stats: 7200 rows total | | column stats: all | | hosts=1 per-host-mem=64.00MB | | tuple-ids=3 row-size=40B cardinality=398 | | | 00:SCAN HDFS [tpcds_300_decimal_parquet.store_sales, RANDOM] | partitions=1093/1824 files=1093 size=22.33GB | table stats: 864001869 rows total | column stats: all | hosts=8 per-host-mem=128.00MB | tuple-ids=0 row-size=40B cardinality=494213786 | 16:EXCHANGE [HASH(c_customer_sk)] | hosts=8 per-host-mem=0B | tuple-ids=6 row-size=88B cardinality=5000000 | 08:SCAN HDFS [tpcds_300_decimal_parquet.customer, RANDOM] partitions=1/1 files=8 size=266.45MB table stats: 5000000 rows total column stats: all hosts=8 per-host-mem=240.00MB tuple-ids=6 row-size=88B cardinality=5000000 ---------------- Estimated Per-Host Mem: 407415701 Estimated Per-Host VCores: 4 Request Pool: redacted Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ---------------------------------------------------------------------------------------------------------------------------------- 18:MERGING-EXCHANGE 1 33.227ms 33.227ms 18.96K 29.94K 0 -1.00 B UNPARTITIONED 10:TOP-N 7 29.175ms 55.339ms 18.96K 29.94K 8.58 MB 2.73 MB 09:HASH JOIN 7 225.597ms 534.893ms 18.96K 29.94K 1.66 MB 16.28 MB INNER JOIN, PARTITIONED |--17:EXCHANGE 7 1.020ms 1.603ms 18.96K 5.17M 0 0 HASH(ss_customer_sk) | 15:AGGREGATE 8 431.143ms 1s095ms 18.96K 5.17M 3.11 MB 130.25 MB FINALIZE | 14:EXCHANGE 8 10.438ms 30.966ms 209.92K 5.17M 0 0 HASH(ss_ticket_number,ss_cu... | 07:AGGREGATE 8 10s430ms 12s722ms 209.92K 5.17M 14.07 MB 130.25 MB STREAMING | 06:HASH JOIN 8 1s420ms 4s196ms 2.18M 5.17M 10.94 MB 16.63 KB INNER JOIN, BROADCAST | |--13:EXCHANGE 8 94.583us 147.056us 229 402 0 0 BROADCAST | | 02:SCAN HDFS 1 15.655ms 15.655ms 229 402 140.00 KB 32.00 MB tpcds_300_decimal_parquet.s... | 05:HASH JOIN 8 646.297ms 806.900ms 7.94M 5.17M 3.13 MB 6.08 KB INNER JOIN, BROADCAST | |--12:EXCHANGE 8 92.823us 149.824us 251 354 0 0 BROADCAST | | 01:SCAN HDFS 1 248.802ms 248.802ms 251 354 3.05 MB 48.00 MB tpcds_300_decimal_parquet.d... | 04:HASH JOIN 8 6s316ms 6s538ms 34.84M 27.32M 1.97 MB 16.89 KB INNER JOIN, BROADCAST | |--11:EXCHANGE 8 162.918us 314.208us 520 398 0 0 BROADCAST | | 03:SCAN HDFS 1 148.670ms 148.670ms 520 398 575.20 KB 64.00 MB tpcds_300_decimal_parquet.h... | 00:SCAN HDFS 8 4m29s 4m35s 494.21M 494.21M 13.89 MB 128.00 MB tpcds_300_decimal_parquet.s... 16:EXCHANGE 7 367.891ms 437.716ms 5.00M 5.00M 0 0 HASH(c_customer_sk) 08:SCAN HDFS 7 320.462ms 444.712ms 5.00M 5.00M 23.12 MB 240.00 MB tpcds_300_decimal_parquet.c... - ComputeScanRangeAssignmentTimer: 1156560672 - InactiveTotalTime: 0 - TotalTime: 0 Planner Timeline Analysis finished: 905264413 Equivalence classes computed: 908275321 Single node plan created: 939478688 Distributed plan created: 940572206 Lineage info computed: 940978872 Planning finished: 952103125 Query Timeline Query submitted: 107123616 Planning finished: 4254123392 Submit for admission: 8817820784 Completed admission: 10115250720 Ready to start 34 fragment instances: 11770943560 All 34 fragment instances started: 16254052016 Rows available: 315829164040 First row fetched: 315846195664 Unregister query: 317772504640 ImpalaServer - ClientFetchWaitTimer: 1844302784 - InactiveTotalTime: 0 - RowMaterializationTimer: 98867504 - TotalTime: 0 Execution Profile b149ee4c1d799434:62edf82a00000000 Fragment instance start latencies: Count: 34, 25th %-ile: 60ms, 50th %-ile: 93ms, 75th %-ile: 146ms, 90th %-ile: 194ms, 95th %-ile: 198ms, 99.9th %-ile: 316ms Per Node Peak Memory Usage: redacted-host1:22000(71.05 MB) redacted-host3:22000(69.16 MB) redacted-host7:22000(66.87 MB) redacted-host2:22000(69.71 MB) redacted-host5:22000(70.79 MB) redacted-host6:22000(67.01 MB) redacted-host4:22000(71.12 MB) redacted-host8:22000(28.15 MB) - FiltersReceived: 0 - FinalizationTimer: 0 - InactiveTotalTime: 0 - TotalTime: 304905609248 Coordinator Fragment F07 Instance b149ee4c1d799434:62edf82a00000000 (host=redacted-host8:22000) - AverageThreadTokens: 0.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 1996420 - PerHostPeakMemUsage: 29514737 - RowsProduced: 18956 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 0 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 148 - TotalThreadsTotalWallClockTime: 304436471560 - TotalThreadsSysTime: 9998000 - TotalThreadsUserTime: 297955000 - TotalThreadsVoluntaryContextSwitches: 126 - TotalTime: 305058748448 Fragment Instance Lifecycle Timings - ExecTime: 1828982512 - ExecTreeExecTime: 32440864 - InactiveTotalTime: 0 - OpenTime: 302627270528 - ExecTreeOpenTime: 301973288048 - PrepareTime: 602367440 - ExecTreePrepareTime: 95287584 - TotalTime: 0 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 96 - BlocksRecycled: 0 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 34603008 - PeakMemoryUsage: 1934592 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 PLAN_ROOT_SINK - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - TotalTime: 0 CodeGen - CodegenTime: 32319872 - CompileTime: 50173976 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 31 - NumInstructions: 755 - OptimizationTime: 554571344 - PeakMemoryUsage: 386560 - PrepareTime: 439746160 - TotalTime: 1045653112 EXCHANGE_NODE (id=18) ExecOption: Codegen Enabled - BytesReceived: 984602 - ConvertRowBatchTime: 0 - DeserializeRowBatchTimer: 20502664 - FirstBatchArrivalWaitTime: 301972383824 - InactiveTotalTime: 301972388176 - MergeGetNext: 32202656 - MergeGetNextBatch: 288896 - PeakMemoryUsage: 0 - RowsReturned: 18956 - RowsReturnedRate: 62 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 302005615792 Averaged Fragment F06 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:5m2s max:5m3s mean: 5m2s stddev:123.318ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 7 - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 22766104 - PerHostPeakMemUsage: 72758882 - RowsProduced: 2708 - TotalNetworkReceiveTime: 299208455598 - TotalNetworkSendTime: 3739248 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 781 - TotalThreadsTotalWallClockTime: 302080566037 - TotalThreadsSysTime: 42136571 - TotalThreadsUserTime: 1249952714 - TotalThreadsVoluntaryContextSwitches: 2533 - TotalTime: 302909455045 Fragment Instance Lifecycle Timings - ExecTime: 11236604 - ExecTreeExecTime: 764757 - InactiveTotalTime: 0 - OpenTime: 302088144890 - ExecTreeOpenTime: 299584486370 - PrepareTime: 809952249 - ExecTreePrepareTime: 60451361 - TotalTime: 0 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 96 - BlocksRecycled: 20 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 34603008 - PeakMemoryUsage: 2005589 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 DataStreamSender (dst_id=18) - BytesSent: 140657 - InactiveTotalTime: 0 - NetworkThroughput(*): 144708203 - OverallThroughput: 17005021 - PeakMemoryUsage: 1616 - RowsReturned: 2708 - SerializeBatchTime: 5940609 - TotalTime: 9624154 - TransmitDataRPCTime: 992410 - UncompressedRowBatchSize: 272645 CodeGen - CodegenTime: 39718694 - CompileTime: 486972986 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 134 - NumInstructions: 4382 - OptimizationTime: 1916760508 - PeakMemoryUsage: 2243584 - PrepareTime: 432933278 - TotalTime: 2854384520 SORT_NODE (id=10) - InactiveTotalTime: 0 - InsertBatchTime: 3201378 - PeakMemoryUsage: 8988127 - RowsReturned: 2708 - RowsReturnedRate: 8 - TotalTime: 299645687918 HASH_JOIN_NODE (id=9) - BuildRows: 2708 - BuildTime: 3276402 - InactiveTotalTime: 0 - LocalTime: 225597834 - PeakMemoryUsage: 1588647 - ProbeRows: 714285 - ProbeRowsPartitioned: 0 - ProbeTime: 101023781 - RowsReturned: 2708 - RowsReturnedRate: 8 - TotalTime: 299644107392 Hash Join Builder (join_node_id=9) - BuildRowsPartitionTime: 854126 - BuildRowsPartitioned: 2708 - GetNewBlockTime: 93014 - HashBuckets: 5924 - HashCollisions: 0 - HashTablesBuildTime: 1958098 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1464594 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=17) - BytesReceived: 45578 - ConvertRowBatchTime: 806658 - DeserializeRowBatchTimer: 1678666 - FirstBatchArrivalWaitTime: 295992996349 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 2708 - RowsReturnedRate: 8 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 296019986274 EXCHANGE_NODE (id=16) - BytesReceived: 35936105 - ConvertRowBatchTime: 184433257 - DeserializeRowBatchTimer: 2994996509 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 714285 - RowsReturnedRate: 210175 - SendersBlockedTimer: 274442308144 - SendersBlockedTotalTimer(*): 1411158255043 - TotalTime: 3398523283 Fragment F06 Instance b149ee4c1d799434:1b (host=redacted-host6:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 22889537 - PerHostPeakMemUsage: 70264786 - RowsProduced: 2684 - TotalNetworkReceiveTime: 299127196336 - TotalNetworkSendTime: 2444592 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 792 - TotalThreadsTotalWallClockTime: 302190934296 - TotalThreadsSysTime: 42994000 - TotalThreadsUserTime: 1215815000 - TotalThreadsVoluntaryContextSwitches: 2579 - TotalTime: 303142832816 Fragment Instance Lifecycle Timings - ExecTime: 10942844 - ExecTreeExecTime: 578772 - InactiveTotalTime: 0 - OpenTime: 302198177128 - ExecTreeOpenTime: 299494146756 - PrepareTime: 933621364 - ExecTreePrepareTime: 66538944 - TotalTime: 0 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 96 - BlocksRecycled: 23 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 34603008 - PeakMemoryUsage: 2163968 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 DataStreamSender (dst_id=18) - BytesSent: 139510 - InactiveTotalTime: 0 - NetworkThroughput(*): 138428046 - OverallThroughput: 26529390 - PeakMemoryUsage: 1616 - RowsReturned: 2684 - SerializeBatchTime: 4211296 - TotalTime: 5258696 - TransmitDataRPCTime: 1007816 - UncompressedRowBatchSize: 270345 CodeGen - CodegenTime: 16613192 - CompileTime: 570994288 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 134 - NumInstructions: 4382 - OptimizationTime: 1939547588 - PeakMemoryUsage: 2243584 - PrepareTime: 660446540 - TotalTime: 3188177544 SORT_NODE (id=10) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - InsertBatchTime: 2784492 - PeakMemoryUsage: 8981746 - RowsReturned: 2684 - RowsReturnedRate: 8 - TotalTime: 299561254220 HASH_JOIN_NODE (id=9) ExecOption: Probe Side Codegen Enabled - BuildRows: 2684 - BuildTime: 5700680 - InactiveTotalTime: 0 - LocalTime: 250053356 - PeakMemoryUsage: 1744403 - ProbeRows: 713910 - ProbeRowsPartitioned: 0 - ProbeTime: 94170492 - RowsReturned: 2684 - RowsReturnedRate: 8 - TotalTime: 299583318432 Hash Join Builder (join_node_id=9) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 1033244 - BuildRowsPartitioned: 2684 - GetNewBlockTime: 54996 - HashBuckets: 5888 - HashCollisions: 0 - HashTablesBuildTime: 4327384 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1623168 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=17) - BytesReceived: 45112 - ConvertRowBatchTime: 802088 - DeserializeRowBatchTimer: 607744 - FirstBatchArrivalWaitTime: 295901259740 - InactiveTotalTime: 295931219392 - PeakMemoryUsage: 0 - RowsReturned: 2684 - RowsReturnedRate: 9 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 295932298700 EXCHANGE_NODE (id=16) - BytesReceived: 35917641 - ConvertRowBatchTime: 179246192 - DeserializeRowBatchTimer: 2785950808 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 3076289224 - PeakMemoryUsage: 0 - RowsReturned: 713910 - RowsReturnedRate: 209913 - SendersBlockedTimer: 274541076352 - SendersBlockedTotalTimer(*): 1372390990444 - TotalTime: 3400966376 Instance b149ee4c1d799434:1c (host=redacted-host5:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 22056281 - PerHostPeakMemUsage: 74232755 - RowsProduced: 2732 - TotalNetworkReceiveTime: 300546639432 - TotalNetworkSendTime: 3804752 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 762 - TotalThreadsTotalWallClockTime: 302531354336 - TotalThreadsSysTime: 40994000 - TotalThreadsUserTime: 1251810000 - TotalThreadsVoluntaryContextSwitches: 2321 - TotalTime: 303067684304 Fragment Instance Lifecycle Timings - ExecTime: 11451936 - ExecTreeExecTime: 923040 - InactiveTotalTime: 0 - OpenTime: 302527232008 - ExecTreeOpenTime: 300934852744 - PrepareTime: 528874080 - ExecTreePrepareTime: 125901392 - TotalTime: 0 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 96 - BlocksRecycled: 18 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 34603008 - PeakMemoryUsage: 1377536 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 DataStreamSender (dst_id=18) - BytesSent: 142440 - InactiveTotalTime: 0 - NetworkThroughput(*): 130820040 - OverallThroughput: 11970150 - PeakMemoryUsage: 1616 - RowsReturned: 2732 - SerializeBatchTime: 6400640 - TotalTime: 11899600 - TransmitDataRPCTime: 1088824 - UncompressedRowBatchSize: 275245 CodeGen - CodegenTime: 54149144 - CompileTime: 233329936 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 134 - NumInstructions: 4382 - OptimizationTime: 1329530024 - PeakMemoryUsage: 2243584 - PrepareTime: 207829688 - TotalTime: 1807675792 SORT_NODE (id=10) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - InsertBatchTime: 3147640 - PeakMemoryUsage: 8984765 - RowsReturned: 2732 - RowsReturnedRate: 9 - TotalTime: 301061661448 HASH_JOIN_NODE (id=9) ExecOption: Probe Side Codegen Enabled - BuildRows: 2732 - BuildTime: 3644200 - InactiveTotalTime: 0 - LocalTime: 260024000 - PeakMemoryUsage: 1412636 - ProbeRows: 713170 - ProbeRowsPartitioned: 0 - ProbeTime: 110404512 - RowsReturned: 2732 - RowsReturnedRate: 9 - TotalTime: 301025569472 Hash Join Builder (join_node_id=9) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 929272 - BuildRowsPartitioned: 2732 - GetNewBlockTime: 65840 - HashBuckets: 5376 - HashCollisions: 0 - HashTablesBuildTime: 2002032 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1287296 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=17) - BytesReceived: 46052 - ConvertRowBatchTime: 877280 - DeserializeRowBatchTimer: 556336 - FirstBatchArrivalWaitTime: 297333340056 - InactiveTotalTime: 297363706656 - PeakMemoryUsage: 0 - RowsReturned: 2732 - RowsReturnedRate: 9 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 297364806856 EXCHANGE_NODE (id=16) - BytesReceived: 35880490 - ConvertRowBatchTime: 192500272 - DeserializeRowBatchTimer: 2205893320 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 3046442672 - PeakMemoryUsage: 0 - RowsReturned: 713170 - RowsReturnedRate: 209710 - SendersBlockedTimer: 274411737576 - SendersBlockedTotalTimer(*): 1097461140984 - TotalTime: 3400738616 Instance b149ee4c1d799434:1e (host=redacted-host7:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 22793580 - PerHostPeakMemUsage: 70120109 - RowsProduced: 2765 - TotalNetworkReceiveTime: 299835004456 - TotalNetworkSendTime: 3909256 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 846 - TotalThreadsTotalWallClockTime: 302479910264 - TotalThreadsSysTime: 39994000 - TotalThreadsUserTime: 1234812000 - TotalThreadsVoluntaryContextSwitches: 2677 - TotalTime: 302950013384 Fragment Instance Lifecycle Timings - ExecTime: 12245672 - ExecTreeExecTime: 777584 - InactiveTotalTime: 0 - OpenTime: 302469244384 - ExecTreeOpenTime: 300208459296 - PrepareTime: 468390456 - ExecTreePrepareTime: 644480 - TotalTime: 0 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 96 - BlocksRecycled: 20 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 34603008 - PeakMemoryUsage: 2163968 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 DataStreamSender (dst_id=18) - BytesSent: 143254 - InactiveTotalTime: 0 - NetworkThroughput(*): 121229918 - OverallThroughput: 16800677 - PeakMemoryUsage: 1616 - RowsReturned: 2765 - SerializeBatchTime: 7291032 - TotalTime: 8526680 - TransmitDataRPCTime: 1181672 - UncompressedRowBatchSize: 278319 CodeGen - CodegenTime: 53525112 - CompileTime: 345712856 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 134 - NumInstructions: 4382 - OptimizationTime: 1810717688 - PeakMemoryUsage: 2243584 - PrepareTime: 363363128 - TotalTime: 2521521488 SORT_NODE (id=10) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - InsertBatchTime: 3170848 - PeakMemoryUsage: 8986126 - RowsReturned: 2765 - RowsReturnedRate: 9 - TotalTime: 300209866680 HASH_JOIN_NODE (id=9) ExecOption: Probe Side Codegen Enabled - BuildRows: 2765 - BuildTime: 4966936 - InactiveTotalTime: 0 - LocalTime: 117788984 - PeakMemoryUsage: 1560157 - ProbeRows: 715524 - ProbeRowsPartitioned: 0 - ProbeTime: 93711064 - RowsReturned: 2765 - RowsReturnedRate: 9 - TotalTime: 300155050056 Hash Join Builder (join_node_id=9) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 1428920 - BuildRowsPartitioned: 2765 - GetNewBlockTime: 198600 - HashBuckets: 6656 - HashCollisions: 0 - HashTablesBuildTime: 2873288 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1438848 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=17) - BytesReceived: 46463 - ConvertRowBatchTime: 1381248 - DeserializeRowBatchTimer: 7707120 - FirstBatchArrivalWaitTime: 296621754888 - InactiveTotalTime: 296645195072 - PeakMemoryUsage: 0 - RowsReturned: 2765 - RowsReturnedRate: 9 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 296646798968 EXCHANGE_NODE (id=16) - BytesReceived: 36000950 - ConvertRowBatchTime: 177385720 - DeserializeRowBatchTimer: 2565178208 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 3063909320 - PeakMemoryUsage: 0 - RowsReturned: 715524 - RowsReturnedRate: 211040 - SendersBlockedTimer: 274375370608 - SendersBlockedTotalTimer(*): 1646032462560 - TotalTime: 3390462104 Instance b149ee4c1d799434:1d (host=redacted-host2:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 22934222 - PerHostPeakMemUsage: 73096566 - RowsProduced: 2727 - TotalNetworkReceiveTime: 296973955452 - TotalNetworkSendTime: 2309460 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 864 - TotalThreadsTotalWallClockTime: 300585565700 - TotalThreadsSysTime: 35995000 - TotalThreadsUserTime: 1312800000 - TotalThreadsVoluntaryContextSwitches: 2483 - TotalTime: 302899791544 Fragment Instance Lifecycle Timings - ExecTime: 10679376 - ExecTreeExecTime: 745028 - InactiveTotalTime: 0 - OpenTime: 300628640880 - ExecTreeOpenTime: 297360240132 - PrepareTime: 2260325308 - ExecTreePrepareTime: 202668272 - TotalTime: 0 DataStreamSender (dst_id=18) - BytesSent: 141566 - InactiveTotalTime: 0 - NetworkThroughput(*): 144436236 - OverallThroughput: 16960593 - PeakMemoryUsage: 1616 - RowsReturned: 2727 - SerializeBatchTime: 7301764 - TotalTime: 8346760 - TransmitDataRPCTime: 980128 - UncompressedRowBatchSize: 274397 CodeGen - CodegenTime: 17622920 - CompileTime: 418185056 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 134 - NumInstructions: 4382 - OptimizationTime: 2753005084 - PeakMemoryUsage: 2243584 - PrepareTime: 706685528 - TotalTime: 3914529968 SORT_NODE (id=10) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - InsertBatchTime: 3078184 - PeakMemoryUsage: 9000805 - RowsReturned: 2727 - RowsReturnedRate: 9 - TotalTime: 297563638612 HASH_JOIN_NODE (id=9) ExecOption: Probe Side Codegen Enabled - BuildRows: 2727 - BuildTime: 1805480 - InactiveTotalTime: 0 - LocalTime: 534893812 - PeakMemoryUsage: 1622436 - ProbeRows: 714139 - ProbeRowsPartitioned: 0 - ProbeTime: 108558764 - RowsReturned: 2727 - RowsReturnedRate: 9 - TotalTime: 297734738652 Hash Join Builder (join_node_id=9) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 543592 - BuildRowsPartitioned: 2727 - GetNewBlockTime: 91696 - HashBuckets: 6144 - HashCollisions: 0 - HashTablesBuildTime: 803428 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1496192 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=17) - BytesReceived: 45849 - ConvertRowBatchTime: 556880 - DeserializeRowBatchTimer: 982552 - FirstBatchArrivalWaitTime: 293761757128 - InactiveTotalTime: 293794876356 - PeakMemoryUsage: 0 - RowsReturned: 2727 - RowsReturnedRate: 9 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 293795661888 EXCHANGE_NODE (id=16) - BytesReceived: 35925759 - ConvertRowBatchTime: 199438264 - DeserializeRowBatchTimer: 3682244224 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 3036605792 - PeakMemoryUsage: 0 - RowsReturned: 714139 - RowsReturnedRate: 209782 - SendersBlockedTimer: 274461812888 - SendersBlockedTotalTimer(*): 1372175890444 - TotalTime: 3404182952 Instance b149ee4c1d799434:1f (host=redacted-host3:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 22912823 - PerHostPeakMemUsage: 72516979 - RowsProduced: 2710 - TotalNetworkReceiveTime: 300369348448 - TotalNetworkSendTime: 3788024 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 761 - TotalThreadsTotalWallClockTime: 302249983840 - TotalThreadsSysTime: 45993000 - TotalThreadsUserTime: 1230813000 - TotalThreadsVoluntaryContextSwitches: 2454 - TotalTime: 302793771960 Fragment Instance Lifecycle Timings - ExecTime: 8771256 - ExecTreeExecTime: 542096 - InactiveTotalTime: 0 - OpenTime: 302246536536 - ExecTreeOpenTime: 300739444064 - PrepareTime: 538361944 - ExecTreePrepareTime: 774552 - TotalTime: 0 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 96 - BlocksRecycled: 22 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 34603008 - PeakMemoryUsage: 2000128 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 DataStreamSender (dst_id=18) - BytesSent: 140061 - InactiveTotalTime: 0 - NetworkThroughput(*): 162513923 - OverallThroughput: 7280940 - PeakMemoryUsage: 1616 - RowsReturned: 2710 - SerializeBatchTime: 4189048 - TotalTime: 19236664 - TransmitDataRPCTime: 861840 - UncompressedRowBatchSize: 272638 CodeGen - CodegenTime: 53721328 - CompileTime: 229381296 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 134 - NumInstructions: 4382 - OptimizationTime: 1131054912 - PeakMemoryUsage: 2243584 - PrepareTime: 496582704 - TotalTime: 1870802584 SORT_NODE (id=10) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - InsertBatchTime: 3648352 - PeakMemoryUsage: 8984437 - RowsReturned: 2710 - RowsReturnedRate: 9 - TotalTime: 300740742368 HASH_JOIN_NODE (id=9) ExecOption: Probe Side Codegen Enabled - BuildRows: 2710 - BuildTime: 2266248 - InactiveTotalTime: 0 - LocalTime: 123595544 - PeakMemoryUsage: 1685111 - ProbeRows: 714050 - ProbeRowsPartitioned: 0 - ProbeTime: 101464440 - RowsReturned: 2710 - RowsReturnedRate: 9 - TotalTime: 300699531904 Hash Join Builder (join_node_id=9) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 613464 - BuildRowsPartitioned: 2710 - GetNewBlockTime: 94264 - HashBuckets: 5888 - HashCollisions: 0 - HashTablesBuildTime: 1221976 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1557632 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=17) - BytesReceived: 45619 - ConvertRowBatchTime: 722632 - DeserializeRowBatchTimer: 413040 - FirstBatchArrivalWaitTime: 297154395168 - InactiveTotalTime: 297176622992 - PeakMemoryUsage: 0 - RowsReturned: 2710 - RowsReturnedRate: 9 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 297177496048 EXCHANGE_NODE (id=16) - BytesReceived: 35918995 - ConvertRowBatchTime: 180971432 - DeserializeRowBatchTimer: 2149900416 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 2960723336 - PeakMemoryUsage: 0 - RowsReturned: 714050 - RowsReturnedRate: 210111 - SendersBlockedTimer: 274550510248 - SendersBlockedTotalTimer(*): 1646785244448 - TotalTime: 3398440312 Instance b149ee4c1d799434:20 (host=redacted-host1:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 22928797 - PerHostPeakMemUsage: 74501349 - RowsProduced: 2635 - TotalNetworkReceiveTime: 298456664888 - TotalNetworkSendTime: 5567936 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 733 - TotalThreadsTotalWallClockTime: 302216865104 - TotalThreadsSysTime: 44993000 - TotalThreadsUserTime: 1268807000 - TotalThreadsVoluntaryContextSwitches: 2690 - TotalTime: 302764533456 Fragment Instance Lifecycle Timings - ExecTime: 13614816 - ExecTreeExecTime: 1090648 - InactiveTotalTime: 0 - OpenTime: 302209084344 - ExecTreeOpenTime: 298826716776 - PrepareTime: 541716728 - ExecTreePrepareTime: 26210112 - TotalTime: 0 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 96 - BlocksRecycled: 21 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 34603008 - PeakMemoryUsage: 2163968 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 DataStreamSender (dst_id=18) - BytesSent: 136932 - InactiveTotalTime: 0 - NetworkThroughput(*): 186686080 - OverallThroughput: 18629007 - PeakMemoryUsage: 1616 - RowsReturned: 2635 - SerializeBatchTime: 6575072 - TotalTime: 7350472 - TransmitDataRPCTime: 733488 - UncompressedRowBatchSize: 265290 CodeGen - CodegenTime: 30853488 - CompileTime: 722315856 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 134 - NumInstructions: 4382 - OptimizationTime: 2611178240 - PeakMemoryUsage: 2243584 - PrepareTime: 334987104 - TotalTime: 3670491976 SORT_NODE (id=10) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - InsertBatchTime: 2846304 - PeakMemoryUsage: 8990865 - RowsReturned: 2635 - RowsReturnedRate: 8 - TotalTime: 298854000832 HASH_JOIN_NODE (id=9) ExecOption: Probe Side Codegen Enabled - BuildRows: 2635 - BuildTime: 2139248 - InactiveTotalTime: 0 - LocalTime: 173094408 - PeakMemoryUsage: 1613545 - ProbeRows: 713823 - ProbeRowsPartitioned: 0 - ProbeTime: 100363176 - RowsReturned: 2635 - RowsReturnedRate: 8 - TotalTime: 298837231952 Hash Join Builder (join_node_id=9) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 588544 - BuildRowsPartitioned: 2635 - GetNewBlockTime: 77928 - HashBuckets: 5632 - HashCollisions: 0 - HashTablesBuildTime: 1204184 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1488000 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=17) - BytesReceived: 44422 - ConvertRowBatchTime: 604752 - DeserializeRowBatchTimer: 394280 - FirstBatchArrivalWaitTime: 295240764912 - InactiveTotalTime: 295261948072 - PeakMemoryUsage: 0 - RowsReturned: 2635 - RowsReturnedRate: 8 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 295262735392 EXCHANGE_NODE (id=16) - BytesReceived: 35914407 - ConvertRowBatchTime: 183071800 - DeserializeRowBatchTimer: 4016543384 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 3052604688 - PeakMemoryUsage: 0 - RowsReturned: 713823 - RowsReturnedRate: 209861 - SendersBlockedTimer: 274481213656 - SendersBlockedTotalTimer(*): 1097662338904 - TotalTime: 3401402152 Instance b149ee4c1d799434:21 (host=redacted-host4:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 22847490 - PerHostPeakMemUsage: 74579634 - RowsProduced: 2703 - TotalNetworkReceiveTime: 299150380176 - TotalNetworkSendTime: 4350720 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 713 - TotalThreadsTotalWallClockTime: 302309348720 - TotalThreadsSysTime: 43993000 - TotalThreadsUserTime: 1234812000 - TotalThreadsVoluntaryContextSwitches: 2532 - TotalTime: 302747557856 Fragment Instance Lifecycle Timings - ExecTime: 10950328 - ExecTreeExecTime: 696136 - InactiveTotalTime: 0 - OpenTime: 302338098952 - ExecTreeOpenTime: 299527544824 - PrepareTime: 398375864 - ExecTreePrepareTime: 421776 - TotalTime: 0 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 96 - BlocksRecycled: 19 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 34603008 - PeakMemoryUsage: 2163968 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 DataStreamSender (dst_id=18) - BytesSent: 140839 - InactiveTotalTime: 0 - NetworkThroughput(*): 128843184 - OverallThroughput: 20864394 - PeakMemoryUsage: 1616 - RowsReturned: 2703 - SerializeBatchTime: 5615416 - TotalTime: 6750208 - TransmitDataRPCTime: 1093104 - UncompressedRowBatchSize: 272281 CodeGen - CodegenTime: 51545680 - CompileTime: 888891616 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 134 - NumInstructions: 4382 - OptimizationTime: 1842290024 - PeakMemoryUsage: 2243584 - PrepareTime: 260638256 - TotalTime: 3007492288 SORT_NODE (id=10) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - InsertBatchTime: 3733832 - PeakMemoryUsage: 8988151 - RowsReturned: 2703 - RowsReturnedRate: 9 - TotalTime: 299528651272 HASH_JOIN_NODE (id=9) ExecOption: Probe Side Codegen Enabled - BuildRows: 2703 - BuildTime: 2412024 - InactiveTotalTime: 0 - LocalTime: 119734736 - PeakMemoryUsage: 1482244 - ProbeRows: 715384 - ProbeRowsPartitioned: 0 - ProbeTime: 98494024 - RowsReturned: 2703 - RowsReturnedRate: 9 - TotalTime: 299473311280 Hash Join Builder (join_node_id=9) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 841848 - BuildRowsPartitioned: 2703 - GetNewBlockTime: 67776 - HashBuckets: 5888 - HashCollisions: 0 - HashTablesBuildTime: 1274400 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1361024 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=17) - BytesReceived: 45533 - ConvertRowBatchTime: 701728 - DeserializeRowBatchTimer: 1089592 - FirstBatchArrivalWaitTime: 295937702552 - InactiveTotalTime: 295959194912 - PeakMemoryUsage: 0 - RowsReturned: 2703 - RowsReturnedRate: 9 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 295960106072 EXCHANGE_NODE (id=16) - BytesReceived: 35994497 - ConvertRowBatchTime: 178419120 - DeserializeRowBatchTimer: 3559265208 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 2977845192 - PeakMemoryUsage: 0 - RowsReturned: 715384 - RowsReturnedRate: 210811 - SendersBlockedTimer: 274274435680 - SendersBlockedTotalTimer(*): 1645599717520 - TotalTime: 3393470472 Averaged Fragment F00 split sizes: min: 32.87 MB, max: 65.73 MB, avg: 38.06 MB, stddev: 11.30 MB completion times: min:5m max:5m2s mean: 5m stddev:729.387ms execution rates: min:112.02 KB/sec max:222.44 KB/sec mean:129.48 KB/sec stddev:37.97 KB/sec num instances: 7 - AverageThreadTokens: 1.996617092951709 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 23581117 - PerHostPeakMemUsage: 72758882 - RowsProduced: 714285 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 296599661688 - TotalStorageWaitTime: 49285862 - TotalThreadsInvoluntaryContextSwitches: 1403 - TotalThreadsTotalWallClockTime: 599708906798 - TotalThreadsSysTime: 102412714 - TotalThreadsUserTime: 3431335000 - TotalThreadsVoluntaryContextSwitches: 1394 - TotalTime: 300782431380 Fragment Instance Lifecycle Timings - ExecTime: 300015777448 - ExecTreeExecTime: 321940298 - InactiveTotalTime: 0 - OpenTime: 204586004 - ExecTreeOpenTime: 139753 - PrepareTime: 561933161 - ExecTreePrepareTime: 344005 - TotalTime: 0 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 96 - BlocksRecycled: 21 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 34603008 - PeakMemoryUsage: 1623296 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 DataStreamSender (dst_id=16) - BytesSent: 35936105 - InactiveTotalTime: 0 - NetworkThroughput(*): 135005283 - OverallThroughput: 26169568 - PeakMemoryUsage: 213248 - RowsReturned: 714285 - SerializeBatchTime: 1094795851 - TotalTime: 1365289980 - TransmitDataRPCTime: 269060948 - UncompressedRowBatchSize: 66187166 CodeGen - CodegenTime: 16038984 - CompileTime: 7777675 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 5 - NumInstructions: 98 - OptimizationTime: 101207465 - PeakMemoryUsage: 50176 - PrepareTime: 298607388 - TotalTime: 444300861 HDFS_SCAN_NODE (id=8) - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.9987881060204409 - BytesRead: 5914124 - BytesReadDataNodeCache: 0 - BytesReadLocal: 5914124 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 5914124 - DecompressionTime: 176035218 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 5 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 23357651 - PerReadThreadRawHdfsThroughput: 572889296 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 197450259 - RowBatchQueuePutWaitTime: 296253725064 - RowsRead: 714285 - RowsReturned: 714285 - RowsReturnedRate: 2364854 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 458 - ScannerThreadsTotalWallClockTime: 299510300194 - MaterializeTupleTime(*): 2791875924 - ScannerThreadsSysTime: 52277285 - ScannerThreadsUserTime: 1180962714 - ScannerThreadsVoluntaryContextSwitches: 919 - TotalRawHdfsReadTime(*): 11964746 - TotalReadThroughput: 19994 - TotalTime: 320462672 Fragment F00 Instance b149ee4c1d799434:3 (host=redacted-host2:22000) Hdfs split stats (:<# splits>/): 1:1/32.87 MB 11:1/32.86 MB - AverageThreadTokens: 1.994949494949495 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 22547461 - PerHostPeakMemUsage: 73096566 - RowsProduced: 1233332 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 296116252956 - TotalStorageWaitTime: 20710432 - TotalThreadsInvoluntaryContextSwitches: 1552 - TotalThreadsTotalWallClockTime: 601684140880 - TotalThreadsSysTime: 111983000 - TotalThreadsUserTime: 5536159000 - TotalThreadsVoluntaryContextSwitches: 1980 - TotalTime: 302563400772 Fragment Instance Lifecycle Timings - ExecTime: 301001841848 - ExecTreeExecTime: 420594212 - InactiveTotalTime: 0 - OpenTime: 437883476 - ExecTreeOpenTime: 104088 - PrepareTime: 1123544428 - ExecTreePrepareTime: 621892 - TotalTime: 0 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 96 - BlocksRecycled: 21 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 34603008 - PeakMemoryUsage: 1623296 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 DataStreamSender (dst_id=16) - BytesSent: 61424254 - InactiveTotalTime: 0 - NetworkThroughput(*): 129026073 - OverallThroughput: 28355069 - PeakMemoryUsage: 213248 - RowsReturned: 1233332 - SerializeBatchTime: 1688112308 - TotalTime: 2166252980 - TransmitDataRPCTime: 476060788 - UncompressedRowBatchSize: 114280306 CodeGen - CodegenTime: 3211136 - CompileTime: 10816416 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 5 - NumInstructions: 98 - OptimizationTime: 113042048 - PeakMemoryUsage: 50176 - PrepareTime: 417015820 - TotalTime: 593122504 HDFS_SCAN_NODE (id=8) Hdfs split stats (:<# splits>/): 1:1/32.87 MB 11:1/32.86 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 2 out of 2 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:10 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.9983108108108109 - BytesRead: 10218332 - BytesReadDataNodeCache: 0 - BytesReadLocal: 10218332 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 10218332 - DecompressionTime: 353181828 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 5 - NumDisksAccessed: 2 - NumRowGroups: 2 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 22265221 - PerReadThreadRawHdfsThroughput: 572732410 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 312145848 - RowBatchQueuePutWaitTime: 294581922372 - RowsRead: 1233332 - RowsReturned: 1233332 - RowsReturnedRate: 2954954 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 650 - ScannerThreadsTotalWallClockTime: 300269621980 - MaterializeTupleTime(*): 4851110040 - ScannerThreadsSysTime: 56991000 - ScannerThreadsUserTime: 2064686000 - ScannerThreadsVoluntaryContextSwitches: 1446 - TotalRawHdfsReadTime(*): 17841372 - TotalReadThroughput: 34453 - TotalTime: 417377692 Instance b149ee4c1d799434:7 (host=redacted-host4:22000) Hdfs split stats (:<# splits>/): 11:1/33.74 MB - AverageThreadTokens: 1.99660441426146 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 24452432 - PerHostPeakMemUsage: 74579634 - RowsProduced: 633338 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 296630033392 - TotalStorageWaitTime: 50680160 - TotalThreadsInvoluntaryContextSwitches: 1606 - TotalThreadsTotalWallClockTime: 599972862936 - TotalThreadsSysTime: 115982000 - TotalThreadsUserTime: 3339491000 - TotalThreadsVoluntaryContextSwitches: 1438 - TotalTime: 300729227992 Fragment Instance Lifecycle Timings - ExecTime: 300150634064 - ExecTreeExecTime: 392227536 - InactiveTotalTime: 0 - OpenTime: 209063512 - ExecTreeOpenTime: 123408 - PrepareTime: 369322944 - ExecTreePrepareTime: 277624 - TotalTime: 0 DataStreamSender (dst_id=16) - BytesSent: 32004530 - InactiveTotalTime: 0 - NetworkThroughput(*): 138965313 - OverallThroughput: 23898215 - PeakMemoryUsage: 213248 - RowsReturned: 633338 - SerializeBatchTime: 1107563120 - TotalTime: 1339201672 - TransmitDataRPCTime: 230305888 - UncompressedRowBatchSize: 58687066 CodeGen - CodegenTime: 31853896 - CompileTime: 9505928 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 5 - NumInstructions: 98 - OptimizationTime: 156051520 - PeakMemoryUsage: 50176 - PrepareTime: 177270256 - TotalTime: 343622920 HDFS_SCAN_NODE (id=8) Hdfs split stats (:<# splits>/): 11:1/33.74 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:5 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.9982993197278912 - BytesRead: 5240386 - BytesReadDataNodeCache: 0 - BytesReadLocal: 5240386 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 5240386 - DecompressionTime: 128258808 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 5 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 24238761 - PerReadThreadRawHdfsThroughput: 844096271 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 317325736 - RowBatchQueuePutWaitTime: 295916851048 - RowsRead: 633338 - RowsReturned: 633338 - RowsReturnedRate: 1619275 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 459 - ScannerThreadsTotalWallClockTime: 299642150120 - MaterializeTupleTime(*): 3206909944 - ScannerThreadsSysTime: 60990000 - ScannerThreadsUserTime: 1047840000 - ScannerThreadsVoluntaryContextSwitches: 1003 - TotalRawHdfsReadTime(*): 6208280 - TotalReadThroughput: 17789 - TotalTime: 391124376 Instance b149ee4c1d799434:1 (host=redacted-host6:22000) Hdfs split stats (:<# splits>/): 4:1/33.74 MB - AverageThreadTokens: 1.994915254237288 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 24452452 - PerHostPeakMemUsage: 70264786 - RowsProduced: 633333 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 296000419252 - TotalStorageWaitTime: 80172932 - TotalThreadsInvoluntaryContextSwitches: 1621 - TotalThreadsTotalWallClockTime: 598607338368 - TotalThreadsSysTime: 94986000 - TotalThreadsUserTime: 3143522000 - TotalThreadsVoluntaryContextSwitches: 1276 - TotalTime: 300544313420 Fragment Instance Lifecycle Timings - ExecTime: 299439943140 - ExecTreeExecTime: 446200696 - InactiveTotalTime: 0 - OpenTime: 198293128 - ExecTreeOpenTime: 143608 - PrepareTime: 905940208 - ExecTreePrepareTime: 294692 - TotalTime: 0 DataStreamSender (dst_id=16) - BytesSent: 31461348 - InactiveTotalTime: 0 - NetworkThroughput(*): 143978091 - OverallThroughput: 25362933 - PeakMemoryUsage: 213248 - RowsReturned: 633333 - SerializeBatchTime: 1020693828 - TotalTime: 1240445940 - TransmitDataRPCTime: 218514828 - UncompressedRowBatchSize: 58686752 CodeGen - CodegenTime: 19123404 - CompileTime: 8342584 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 5 - NumInstructions: 98 - OptimizationTime: 81867048 - PeakMemoryUsage: 50176 - PrepareTime: 500030784 - TotalTime: 748042776 HDFS_SCAN_NODE (id=8) Hdfs split stats (:<# splits>/): 4:1/33.74 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:5 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.9982993197278912 - BytesRead: 5240466 - BytesReadDataNodeCache: 0 - BytesReadLocal: 5240466 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 5240466 - DecompressionTime: 200139836 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 5 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 24238781 - PerReadThreadRawHdfsThroughput: 374973918 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 200044176 - RowBatchQueuePutWaitTime: 296030043444 - RowsRead: 633333 - RowsReturned: 633333 - RowsReturnedRate: 1424140 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 483 - ScannerThreadsTotalWallClockTime: 298981529976 - MaterializeTupleTime(*): 2470261196 - ScannerThreadsSysTime: 51992000 - ScannerThreadsUserTime: 1033842000 - ScannerThreadsVoluntaryContextSwitches: 813 - TotalRawHdfsReadTime(*): 13975548 - TotalReadThroughput: 17757 - TotalTime: 444712444 Instance b149ee4c1d799434:4 (host=redacted-host7:22000) Hdfs split stats (:<# splits>/): 10:1/32.88 MB - AverageThreadTokens: 1.996615905245347 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 22355291 - PerHostPeakMemUsage: 70120109 - RowsProduced: 616664 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 296970236448 - TotalStorageWaitTime: 81017752 - TotalThreadsInvoluntaryContextSwitches: 1243 - TotalThreadsTotalWallClockTime: 599632469800 - TotalThreadsSysTime: 86987000 - TotalThreadsUserTime: 3084530000 - TotalThreadsVoluntaryContextSwitches: 1337 - TotalTime: 300491236848 Fragment Instance Lifecycle Timings - ExecTime: 299963038128 - ExecTreeExecTime: 219082928 - InactiveTotalTime: 0 - OpenTime: 185056624 - ExecTreeOpenTime: 184344 - PrepareTime: 343039144 - ExecTreePrepareTime: 356632 - TotalTime: 0 DataStreamSender (dst_id=16) - BytesSent: 31227237 - InactiveTotalTime: 0 - NetworkThroughput(*): 144729352 - OverallThroughput: 25618288 - PeakMemoryUsage: 213248 - RowsReturned: 616664 - SerializeBatchTime: 1001848656 - TotalTime: 1218943136 - TransmitDataRPCTime: 215762984 - UncompressedRowBatchSize: 57140425 CodeGen - CodegenTime: 3785064 - CompileTime: 8824080 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 5 - NumInstructions: 98 - OptimizationTime: 136865376 - PeakMemoryUsage: 50176 - PrepareTime: 231300296 - TotalTime: 415569808 HDFS_SCAN_NODE (id=8) Hdfs split stats (:<# splits>/): 10:1/32.88 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:5 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.9983050847457627 - BytesRead: 5109467 - BytesReadDataNodeCache: 0 - BytesReadLocal: 5109467 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 5109467 - DecompressionTime: 148218848 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 5 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 22141621 - PerReadThreadRawHdfsThroughput: 656688459 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 140312896 - RowBatchQueuePutWaitTime: 296654332992 - RowsRead: 616664 - RowsReturned: 616664 - RowsReturnedRate: 2830013 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 403 - ScannerThreadsTotalWallClockTime: 299500559936 - MaterializeTupleTime(*): 2468702448 - ScannerThreadsSysTime: 45993000 - ScannerThreadsUserTime: 981850000 - ScannerThreadsVoluntaryContextSwitches: 908 - TotalRawHdfsReadTime(*): 7780656 - TotalReadThroughput: 17284 - TotalTime: 217901408 Instance b149ee4c1d799434:6 (host=redacted-host1:22000) Hdfs split stats (:<# splits>/): 3:1/32.87 MB - AverageThreadTokens: 1.9966101694915255 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 22355298 - PerHostPeakMemUsage: 74501349 - RowsProduced: 616666 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 296320290600 - TotalStorageWaitTime: 5377136 - TotalThreadsInvoluntaryContextSwitches: 1520 - TotalThreadsTotalWallClockTime: 599164346672 - TotalThreadsSysTime: 138978000 - TotalThreadsUserTime: 3122524000 - TotalThreadsVoluntaryContextSwitches: 1464 - TotalTime: 300434771208 Fragment Instance Lifecycle Timings - ExecTime: 299777954552 - ExecTreeExecTime: 324225888 - InactiveTotalTime: 0 - OpenTime: 195128576 - ExecTreeOpenTime: 110864 - PrepareTime: 461532392 - ExecTreePrepareTime: 477384 - TotalTime: 0 DataStreamSender (dst_id=16) - BytesSent: 31973504 - InactiveTotalTime: 0 - NetworkThroughput(*): 114935559 - OverallThroughput: 24663422 - PeakMemoryUsage: 213248 - RowsReturned: 616666 - SerializeBatchTime: 1016769696 - TotalTime: 1296393632 - TransmitDataRPCTime: 278186352 - UncompressedRowBatchSize: 57142019 CodeGen - CodegenTime: 23511864 - CompileTime: 4140256 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 5 - NumInstructions: 98 - OptimizationTime: 102158376 - PeakMemoryUsage: 50176 - PrepareTime: 337456416 - TotalTime: 449183680 HDFS_SCAN_NODE (id=8) Hdfs split stats (:<# splits>/): 3:1/32.87 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:5 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.99830220713073 - BytesRead: 5108849 - BytesReadDataNodeCache: 0 - BytesReadLocal: 5108849 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 5108849 - DecompressionTime: 248365456 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 5 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 22141627 - PerReadThreadRawHdfsThroughput: 263728591 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 116062976 - RowBatchQueuePutWaitTime: 295570411928 - RowsRead: 616666 - RowsReturned: 616666 - RowsReturnedRate: 1906441 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 509 - ScannerThreadsTotalWallClockTime: 299239028864 - MaterializeTupleTime(*): 3199728472 - ScannerThreadsSysTime: 78987000 - ScannerThreadsUserTime: 1105831000 - ScannerThreadsVoluntaryContextSwitches: 914 - TotalRawHdfsReadTime(*): 19371616 - TotalReadThroughput: 17311 - TotalTime: 323464392 Instance b149ee4c1d799434:2 (host=redacted-host5:22000) Hdfs split stats (:<# splits>/): 1:1/33.74 MB - AverageThreadTokens: 1.9983193277310924 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 24452443 - PerHostPeakMemUsage: 74232755 - RowsProduced: 633334 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 297216870544 - TotalStorageWaitTime: 48532576 - TotalThreadsInvoluntaryContextSwitches: 1063 - TotalThreadsTotalWallClockTime: 599535968368 - TotalThreadsSysTime: 69989000 - TotalThreadsUserTime: 2926555000 - TotalThreadsVoluntaryContextSwitches: 1086 - TotalTime: 300357260696 Fragment Instance Lifecycle Timings - ExecTime: 299924922648 - ExecTreeExecTime: 186811760 - InactiveTotalTime: 0 - OpenTime: 107183560 - ExecTreeOpenTime: 153488 - PrepareTime: 325032416 - ExecTreePrepareTime: 207480 - TotalTime: 0 DataStreamSender (dst_id=16) - BytesSent: 31598717 - InactiveTotalTime: 0 - NetworkThroughput(*): 132609670 - OverallThroughput: 27354055 - PeakMemoryUsage: 213248 - RowsReturned: 633334 - SerializeBatchTime: 915724000 - TotalTime: 1155174856 - TransmitDataRPCTime: 238283656 - UncompressedRowBatchSize: 58687566 CodeGen - CodegenTime: 8727672 - CompileTime: 8297248 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 5 - NumInstructions: 98 - OptimizationTime: 72626408 - PeakMemoryUsage: 50176 - PrepareTime: 148453400 - TotalTime: 230414536 HDFS_SCAN_NODE (id=8) Hdfs split stats (:<# splits>/): 1:1/33.74 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:5 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 5241086 - BytesReadDataNodeCache: 0 - BytesReadLocal: 5241086 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 5241086 - DecompressionTime: 74228368 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 5 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 24238772 - PerReadThreadRawHdfsThroughput: 883555052 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 111968968 - RowBatchQueuePutWaitTime: 297561836592 - RowsRead: 633334 - RowsReturned: 633334 - RowsReturnedRate: 3410806 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 331 - ScannerThreadsTotalWallClockTime: 299516458256 - MaterializeTupleTime(*): 1704625912 - ScannerThreadsSysTime: 23996000 - ScannerThreadsUserTime: 1025844000 - ScannerThreadsVoluntaryContextSwitches: 644 - TotalRawHdfsReadTime(*): 5931816 - TotalReadThroughput: 17612 - TotalTime: 185684536 Instance b149ee4c1d799434:5 (host=redacted-host3:22000) Hdfs split stats (:<# splits>/): 10:1/33.75 MB - AverageThreadTokens: 1.9983050847457626 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 24452448 - PerHostPeakMemUsage: 72516979 - RowsProduced: 633333 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 296943528624 - TotalStorageWaitTime: 58510048 - TotalThreadsInvoluntaryContextSwitches: 1221 - TotalThreadsTotalWallClockTime: 599365220568 - TotalThreadsSysTime: 97984000 - TotalThreadsUserTime: 2866564000 - TotalThreadsVoluntaryContextSwitches: 1182 - TotalTime: 300356808728 Fragment Instance Lifecycle Timings - ExecTime: 299852107760 - ExecTreeExecTime: 264439072 - InactiveTotalTime: 0 - OpenTime: 99493152 - ExecTreeOpenTime: 158472 - PrepareTime: 405120600 - ExecTreePrepareTime: 172336 - TotalTime: 0 DataStreamSender (dst_id=16) - BytesSent: 31863149 - InactiveTotalTime: 0 - NetworkThroughput(*): 140792926 - OverallThroughput: 27934995 - PeakMemoryUsage: 213248 - RowsReturned: 633333 - SerializeBatchTime: 912859352 - TotalTime: 1140617648 - TransmitDataRPCTime: 226312144 - UncompressedRowBatchSize: 58686030 CodeGen - CodegenTime: 22059856 - CompileTime: 4517216 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 5 - NumInstructions: 98 - OptimizationTime: 45841480 - PeakMemoryUsage: 50176 - PrepareTime: 278724744 - TotalTime: 330149808 HDFS_SCAN_NODE (id=8) Hdfs split stats (:<# splits>/): 10:1/33.75 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:5 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 5240284 - BytesReadDataNodeCache: 0 - BytesReadLocal: 5240284 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 5240284 - DecompressionTime: 79853384 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 5 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 24238777 - PerReadThreadRawHdfsThroughput: 414450373 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 184291216 - RowBatchQueuePutWaitTime: 297460677072 - RowsRead: 633333 - RowsReturned: 633333 - RowsReturnedRate: 2408349 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 374 - ScannerThreadsTotalWallClockTime: 299422752232 - MaterializeTupleTime(*): 1641793456 - ScannerThreadsSysTime: 46992000 - ScannerThreadsUserTime: 1006846000 - ScannerThreadsVoluntaryContextSwitches: 710 - TotalRawHdfsReadTime(*): 12643936 - TotalReadThroughput: 17755 - TotalTime: 262973856 Averaged Fragment F05 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:4m58s max:4m58s mean: 4m58s stddev:59.162ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 8 - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 3312189 - PerHostPeakMemUsage: 67353364 - RowsProduced: 2369 - TotalNetworkReceiveTime: 297599651487 - TotalNetworkSendTime: 7936448 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 76 - TotalThreadsTotalWallClockTime: 297962325051 - TotalThreadsSysTime: 10623500 - TotalThreadsUserTime: 159975625 - TotalThreadsVoluntaryContextSwitches: 169 - TotalTime: 298817297069 Fragment Instance Lifecycle Timings - ExecTime: 29885010 - ExecTreeExecTime: 12305164 - InactiveTotalTime: 0 - OpenTime: 297950297565 - ExecTreeOpenTime: 297640713802 - PrepareTime: 836984262 - ExecTreePrepareTime: 384437335 - TotalTime: 0 DataStreamSender (dst_id=17) - BytesSent: 39881 - InactiveTotalTime: 0 - NetworkThroughput(*): 69475059 - OverallThroughput: 17660894 - PeakMemoryUsage: 122696 - RowsReturned: 2369 - SerializeBatchTime: 1628351 - TotalTime: 2318604 - TransmitDataRPCTime: 583275 - UncompressedRowBatchSize: 68743 CodeGen - CodegenTime: 82917514 - CompileTime: 52681302 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 42 - NumInstructions: 750 - OptimizationTime: 206163818 - PeakMemoryUsage: 384000 - PrepareTime: 224463902 - TotalTime: 498099068 AGGREGATION_NODE (id=15) - BuildTime: 29395324 - GetNewBlockTime: 191351224 - GetResultsTime: 10179125 - HTResizeTime: 4697320 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 3241593 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 2369 - RowsReturnedRate: 7 - SpilledPartitions: 0 - TotalTime: 298037406898 - UnpinTime: 5809 EXCHANGE_NODE (id=14) - BytesReceived: 474002 - ConvertRowBatchTime: 6201427 - DeserializeRowBatchTimer: 18552994 - FirstBatchArrivalWaitTime: 287321713637 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 26239 - RowsReturnedRate: 87 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 297606262986 Fragment F05 Instance b149ee4c1d799434:13 (host=redacted-host7:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 3312384 - PerHostPeakMemUsage: 70120109 - RowsProduced: 2269 - TotalNetworkReceiveTime: 298156639768 - TotalNetworkSendTime: 5148432 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 100 - TotalThreadsTotalWallClockTime: 298518971168 - TotalThreadsSysTime: 17998000 - TotalThreadsUserTime: 151977000 - TotalThreadsVoluntaryContextSwitches: 150 - TotalTime: 298926025056 Fragment Instance Lifecycle Timings - ExecTime: 31546984 - ExecTreeExecTime: 18282008 - InactiveTotalTime: 0 - OpenTime: 298488862992 - ExecTreeOpenTime: 298217279376 - PrepareTime: 405491080 - ExecTreePrepareTime: 112537680 - TotalTime: 0 DataStreamSender (dst_id=17) - BytesSent: 38328 - InactiveTotalTime: 0 - NetworkThroughput(*): 75916271 - OverallThroughput: 18557899 - PeakMemoryUsage: 122696 - RowsReturned: 2269 - SerializeBatchTime: 1469424 - TotalTime: 2065320 - TransmitDataRPCTime: 504872 - UncompressedRowBatchSize: 65829 CodeGen - CodegenTime: 21315528 - CompileTime: 39652112 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 42 - NumInstructions: 750 - OptimizationTime: 182737640 - PeakMemoryUsage: 384000 - PrepareTime: 244089840 - TotalTime: 484899952 AGGREGATION_NODE (id=15) ExecOption: Codegen Enabled - BuildTime: 26247784 - GetNewBlockTime: 65658856 - GetResultsTime: 15808552 - HTResizeTime: 3772904 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 3242103 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 2269 - RowsReturnedRate: 7 - SpilledPartitions: 0 - TotalTime: 298348027776 - UnpinTime: 7152 EXCHANGE_NODE (id=14) - BytesReceived: 476175 - ConvertRowBatchTime: 5689952 - DeserializeRowBatchTimer: 4624656 - FirstBatchArrivalWaitTime: 287880504056 - InactiveTotalTime: 298156135008 - PeakMemoryUsage: 0 - RowsReturned: 26367 - RowsReturnedRate: 88 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 298162705752 Instance b149ee4c1d799434:17 (host=redacted-host8:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 3310804 - PerHostPeakMemUsage: 29514737 - RowsProduced: 2316 - TotalNetworkReceiveTime: 296930973760 - TotalNetworkSendTime: 6957000 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 65 - TotalThreadsTotalWallClockTime: 297574132976 - TotalThreadsSysTime: 8999000 - TotalThreadsUserTime: 161976000 - TotalThreadsVoluntaryContextSwitches: 212 - TotalTime: 298881342600 Fragment Instance Lifecycle Timings - ExecTime: 29476688 - ExecTreeExecTime: 12186224 - InactiveTotalTime: 0 - OpenTime: 297549288384 - ExecTreeOpenTime: 296966705568 - PrepareTime: 1302373104 - ExecTreePrepareTime: 597507032 - TotalTime: 0 DataStreamSender (dst_id=17) - BytesSent: 39039 - InactiveTotalTime: 0 - NetworkThroughput(*): 70337498 - OverallThroughput: 20492999 - PeakMemoryUsage: 122696 - RowsReturned: 2316 - SerializeBatchTime: 1232400 - TotalTime: 1904992 - TransmitDataRPCTime: 555024 - UncompressedRowBatchSize: 67192 CodeGen - CodegenTime: 291936968 - CompileTime: 64389808 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 42 - NumInstructions: 750 - OptimizationTime: 451672744 - PeakMemoryUsage: 384000 - PrepareTime: 260191624 - TotalTime: 777136584 AGGREGATION_NODE (id=15) ExecOption: Codegen Enabled - BuildTime: 27880616 - GetNewBlockTime: 157140032 - GetResultsTime: 9474512 - HTResizeTime: 4048112 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 3238007 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 2316 - RowsReturnedRate: 7 - SpilledPartitions: 0 - TotalTime: 297576349912 - UnpinTime: 5120 EXCHANGE_NODE (id=14) - BytesReceived: 474318 - ConvertRowBatchTime: 6084584 - DeserializeRowBatchTimer: 113305208 - FirstBatchArrivalWaitTime: 286651804336 - InactiveTotalTime: 296906500280 - PeakMemoryUsage: 0 - RowsReturned: 26243 - RowsReturnedRate: 88 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 296937467088 Instance b149ee4c1d799434:1a (host=redacted-host5:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 3308288 - PerHostPeakMemUsage: 74232755 - RowsProduced: 2375 - TotalNetworkReceiveTime: 298379120064 - TotalNetworkSendTime: 9123768 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 94 - TotalThreadsTotalWallClockTime: 298638905080 - TotalThreadsSysTime: 9999000 - TotalThreadsUserTime: 151977000 - TotalThreadsVoluntaryContextSwitches: 123 - TotalTime: 298856000440 Fragment Instance Lifecycle Timings - ExecTime: 26006048 - ExecTreeExecTime: 7319024 - InactiveTotalTime: 0 - OpenTime: 298627575488 - ExecTreeOpenTime: 298413177024 - PrepareTime: 202286952 - ExecTreePrepareTime: 12510240 - TotalTime: 0 DataStreamSender (dst_id=17) - BytesSent: 39883 - InactiveTotalTime: 0 - NetworkThroughput(*): 60624992 - OverallThroughput: 18734686 - PeakMemoryUsage: 122696 - RowsReturned: 2375 - SerializeBatchTime: 1378376 - TotalTime: 2128832 - TransmitDataRPCTime: 657864 - UncompressedRowBatchSize: 68903 CodeGen - CodegenTime: 12911448 - CompileTime: 48973976 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 42 - NumInstructions: 750 - OptimizationTime: 137904384 - PeakMemoryUsage: 384000 - PrepareTime: 144427792 - TotalTime: 333020584 AGGREGATION_NODE (id=15) ExecOption: Codegen Enabled - BuildTime: 25817400 - GetNewBlockTime: 9401128 - GetResultsTime: 6246800 - HTResizeTime: 3430824 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 3238007 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 2375 - RowsReturnedRate: 7 - SpilledPartitions: 0 - TotalTime: 298432970216 - UnpinTime: 4104 EXCHANGE_NODE (id=14) - BytesReceived: 470623 - ConvertRowBatchTime: 6150056 - DeserializeRowBatchTimer: 4121792 - FirstBatchArrivalWaitTime: 288093182088 - InactiveTotalTime: 298378470072 - PeakMemoryUsage: 0 - RowsReturned: 26064 - RowsReturnedRate: 87 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 298385658808 Instance b149ee4c1d799434:14 (host=redacted-host3:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 3308288 - PerHostPeakMemUsage: 72516979 - RowsProduced: 2499 - TotalNetworkReceiveTime: 298061581816 - TotalNetworkSendTime: 10890240 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 72 - TotalThreadsTotalWallClockTime: 298326500680 - TotalThreadsSysTime: 12998000 - TotalThreadsUserTime: 156976000 - TotalThreadsVoluntaryContextSwitches: 206 - TotalTime: 298847334472 Fragment Instance Lifecycle Timings - ExecTime: 33341936 - ExecTreeExecTime: 11629808 - InactiveTotalTime: 0 - OpenTime: 298295043656 - ExecTreeOpenTime: 298101714048 - PrepareTime: 518819488 - ExecTreePrepareTime: 209355936 - TotalTime: 0 DataStreamSender (dst_id=17) - BytesSent: 42001 - InactiveTotalTime: 0 - NetworkThroughput(*): 65818258 - OverallThroughput: 17626090 - PeakMemoryUsage: 122696 - RowsReturned: 2499 - SerializeBatchTime: 1599992 - TotalTime: 2382888 - TransmitDataRPCTime: 638136 - UncompressedRowBatchSize: 72499 CodeGen - CodegenTime: 67518200 - CompileTime: 30746120 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 42 - NumInstructions: 750 - OptimizationTime: 139291688 - PeakMemoryUsage: 384000 - PrepareTime: 188997176 - TotalTime: 370391288 AGGREGATION_NODE (id=15) ExecOption: Codegen Enabled - BuildTime: 31694336 - GetNewBlockTime: 102692168 - GetResultsTime: 9374168 - HTResizeTime: 5274464 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 3238007 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 2499 - RowsReturnedRate: 8 - SpilledPartitions: 0 - TotalTime: 298322650136 - UnpinTime: 6160 EXCHANGE_NODE (id=14) - BytesReceived: 476417 - ConvertRowBatchTime: 6170168 - DeserializeRowBatchTimer: 6046328 - FirstBatchArrivalWaitTime: 287790054936 - InactiveTotalTime: 298061051240 - PeakMemoryUsage: 0 - RowsReturned: 26374 - RowsReturnedRate: 88 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 298068169752 Instance b149ee4c1d799434:15 (host=redacted-host1:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 3312413 - PerHostPeakMemUsage: 74501349 - RowsProduced: 2429 - TotalNetworkReceiveTime: 297900735336 - TotalNetworkSendTime: 7186072 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 66 - TotalThreadsTotalWallClockTime: 298236596064 - TotalThreadsSysTime: 11998000 - TotalThreadsUserTime: 170974000 - TotalThreadsVoluntaryContextSwitches: 149 - TotalTime: 298821467744 Fragment Instance Lifecycle Timings - ExecTime: 30496248 - ExecTreeExecTime: 14153624 - InactiveTotalTime: 0 - OpenTime: 298257482152 - ExecTreeOpenTime: 297948507152 - PrepareTime: 533377032 - ExecTreePrepareTime: 163767392 - TotalTime: 0 DataStreamSender (dst_id=17) - BytesSent: 40857 - InactiveTotalTime: 0 - NetworkThroughput(*): 74952669 - OverallThroughput: 15423788 - PeakMemoryUsage: 122696 - RowsReturned: 2429 - SerializeBatchTime: 1989848 - TotalTime: 2648960 - TransmitDataRPCTime: 545104 - UncompressedRowBatchSize: 70469 CodeGen - CodegenTime: 120536072 - CompileTime: 61584480 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 42 - NumInstructions: 750 - OptimizationTime: 148626632 - PeakMemoryUsage: 384000 - PrepareTime: 256203584 - TotalTime: 470529808 AGGREGATION_NODE (id=15) ExecOption: Codegen Enabled - BuildTime: 36698808 - GetNewBlockTime: 110036032 - GetResultsTime: 11995776 - HTResizeTime: 5993208 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 3242132 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 2429 - RowsReturnedRate: 8 - SpilledPartitions: 0 - TotalTime: 298126372008 - UnpinTime: 6360 EXCHANGE_NODE (id=14) - BytesReceived: 473713 - ConvertRowBatchTime: 7614168 - DeserializeRowBatchTimer: 5445176 - FirstBatchArrivalWaitTime: 287631779152 - InactiveTotalTime: 297899385824 - PeakMemoryUsage: 0 - RowsReturned: 26225 - RowsReturnedRate: 88 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 297908840672 Instance b149ee4c1d799434:18 (host=redacted-host6:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 3308288 - PerHostPeakMemUsage: 70264786 - RowsProduced: 2357 - TotalNetworkReceiveTime: 297234085196 - TotalNetworkSendTime: 8387012 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 53 - TotalThreadsTotalWallClockTime: 297545648408 - TotalThreadsSysTime: 8998000 - TotalThreadsUserTime: 144977000 - TotalThreadsVoluntaryContextSwitches: 155 - TotalTime: 298766287300 Fragment Instance Lifecycle Timings - ExecTime: 29795552 - ExecTreeExecTime: 10791160 - InactiveTotalTime: 0 - OpenTime: 297524015272 - ExecTreeOpenTime: 297268604584 - PrepareTime: 1212379116 - ExecTreePrepareTime: 759760272 - TotalTime: 0 DataStreamSender (dst_id=17) - BytesSent: 39685 - InactiveTotalTime: 0 - NetworkThroughput(*): 55624236 - OverallThroughput: 12083112 - PeakMemoryUsage: 122696 - RowsReturned: 2357 - SerializeBatchTime: 2485628 - TotalTime: 3284336 - TransmitDataRPCTime: 713448 - UncompressedRowBatchSize: 68381 CodeGen - CodegenTime: 74915600 - CompileTime: 40862384 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 42 - NumInstructions: 750 - OptimizationTime: 187564268 - PeakMemoryUsage: 384000 - PrepareTime: 250198248 - TotalTime: 545333280 AGGREGATION_NODE (id=15) ExecOption: Codegen Enabled - BuildTime: 27247108 - GetNewBlockTime: 374607632 - GetResultsTime: 8907336 - HTResizeTime: 4764576 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 3238007 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 2357 - RowsReturnedRate: 7 - SpilledPartitions: 0 - TotalTime: 298039115424 - UnpinTime: 7312 EXCHANGE_NODE (id=14) - BytesReceived: 475228 - ConvertRowBatchTime: 5755684 - DeserializeRowBatchTimer: 6333676 - FirstBatchArrivalWaitTime: 286953768600 - InactiveTotalTime: 297233181216 - PeakMemoryUsage: 0 - RowsReturned: 26311 - RowsReturnedRate: 88 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 297240240620 Instance b149ee4c1d799434:19 (host=redacted-host4:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 3328759 - PerHostPeakMemUsage: 74579634 - RowsProduced: 2332 - TotalNetworkReceiveTime: 297759915672 - TotalNetworkSendTime: 5838264 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 65 - TotalThreadsTotalWallClockTime: 298078203856 - TotalThreadsSysTime: 8999000 - TotalThreadsUserTime: 166974000 - TotalThreadsVoluntaryContextSwitches: 149 - TotalTime: 298763490968 Fragment Instance Lifecycle Timings - ExecTime: 27841328 - ExecTreeExecTime: 13644944 - InactiveTotalTime: 0 - OpenTime: 298056234000 - ExecTreeOpenTime: 297795348080 - PrepareTime: 679288128 - ExecTreePrepareTime: 168341584 - TotalTime: 0 DataStreamSender (dst_id=17) - BytesSent: 39261 - InactiveTotalTime: 0 - NetworkThroughput(*): 67593485 - OverallThroughput: 18648046 - PeakMemoryUsage: 122696 - RowsReturned: 2332 - SerializeBatchTime: 1449424 - TotalTime: 2105368 - TransmitDataRPCTime: 580840 - UncompressedRowBatchSize: 67656 CodeGen - CodegenTime: 63290664 - CompileTime: 47324696 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 42 - NumInstructions: 750 - OptimizationTime: 184307344 - PeakMemoryUsage: 384000 - PrepareTime: 267098536 - TotalTime: 508203952 AGGREGATION_NODE (id=15) ExecOption: Codegen Enabled - BuildTime: 27725088 - GetNewBlockTime: 100046336 - GetResultsTime: 10648120 - HTResizeTime: 4437584 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 3258478 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 2332 - RowsReturnedRate: 7 - SpilledPartitions: 0 - TotalTime: 297977285224 - UnpinTime: 4224 EXCHANGE_NODE (id=14) - BytesReceived: 473843 - ConvertRowBatchTime: 5878576 - DeserializeRowBatchTimer: 4592248 - FirstBatchArrivalWaitTime: 287475760000 - InactiveTotalTime: 297759399896 - PeakMemoryUsage: 0 - RowsReturned: 26216 - RowsReturnedRate: 88 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 297766191104 Instance b149ee4c1d799434:16 (host=redacted-host2:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 3308288 - PerHostPeakMemUsage: 73096566 - RowsProduced: 2379 - TotalNetworkReceiveTime: 296374160288 - TotalNetworkSendTime: 9960796 - TotalStorageWaitTime: 0 - TotalThreadsInvoluntaryContextSwitches: 94 - TotalThreadsTotalWallClockTime: 296779642176 - TotalThreadsSysTime: 4999000 - TotalThreadsUserTime: 173974000 - TotalThreadsVoluntaryContextSwitches: 215 - TotalTime: 298676427972 Fragment Instance Lifecycle Timings - ExecTime: 30575300 - ExecTreeExecTime: 10434520 - InactiveTotalTime: 0 - OpenTime: 296803878576 - ExecTreeOpenTime: 296414374588 - PrepareTime: 1841859200 - ExecTreePrepareTime: 1051718548 - TotalTime: 0 DataStreamSender (dst_id=17) - BytesSent: 39996 - InactiveTotalTime: 0 - NetworkThroughput(*): 84933066 - OverallThroughput: 19720532 - PeakMemoryUsage: 122696 - RowsReturned: 2379 - SerializeBatchTime: 1421720 - TotalTime: 2028140 - TransmitDataRPCTime: 470912 - UncompressedRowBatchSize: 69019 CodeGen - CodegenTime: 10915636 - CompileTime: 87916844 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 42 - NumInstructions: 750 - OptimizationTime: 217205844 - PeakMemoryUsage: 384000 - PrepareTime: 184504420 - TotalTime: 495277100 AGGREGATION_NODE (id=15) ExecOption: Codegen Enabled - BuildTime: 31851452 - GetNewBlockTime: 611227608 - GetResultsTime: 8977736 - HTResizeTime: 5856888 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 3238007 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 2379 - RowsReturnedRate: 7 - SpilledPartitions: 0 - TotalTime: 297476484492 - UnpinTime: 6040 EXCHANGE_NODE (id=14) - BytesReceived: 471704 - ConvertRowBatchTime: 6268228 - DeserializeRowBatchTimer: 3954868 - FirstBatchArrivalWaitTime: 286096855932 - InactiveTotalTime: 296372470860 - PeakMemoryUsage: 0 - RowsReturned: 26116 - RowsReturnedRate: 88 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 296380830092 Averaged Fragment F01 split sizes: min: 2.78 GB, max: 2.80 GB, avg: 2.79 GB, stddev: 7.21 MB completion times: min:4m47s max:4m57s mean: 4m53s stddev:3s587ms execution rates: min:9.59 MB/sec max:9.92 MB/sec mean:9.75 MB/sec stddev:129.71 KB/sec num instances: 8 - AverageThreadTokens: 1.982240000269773 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 26683785 - PerHostPeakMemUsage: 67353364 - RowsProduced: 26239 - TotalNetworkReceiveTime: 35005 - TotalNetworkSendTime: 14010723 - TotalStorageWaitTime: 113603265141 - TotalThreadsInvoluntaryContextSwitches: 42047 - TotalThreadsTotalWallClockTime: 580119391582 - TotalThreadsSysTime: 2799073625 - TotalThreadsUserTime: 58156408375 - TotalThreadsVoluntaryContextSwitches: 92137 - TotalTime: 293199209397 Fragment Instance Lifecycle Timings - ExecTime: 272588429118 - ExecTreeExecTime: 272532284741 - InactiveTotalTime: 0 - OpenTime: 19682168519 - ExecTreeOpenTime: 15456491771 - PrepareTime: 928498642 - ExecTreePrepareTime: 187524821 - TotalTime: 0 DataStreamSender (dst_id=14) - BytesSent: 474002 - InactiveTotalTime: 0 - NetworkThroughput(*): 96144325 - OverallThroughput: 26264328 - PeakMemoryUsage: 271296 - RowsReturned: 26239 - SerializeBatchTime: 13063579 - TotalTime: 18127189 - TransmitDataRPCTime: 4931621 - UncompressedRowBatchSize: 761120 CodeGen - CodegenTime: 25579168 - CompileTime: 1334281498 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 121 - NumInstructions: 7493 - OptimizationTime: 2805924895 - PeakMemoryUsage: 3836416 - PrepareTime: 360590545 - TotalTime: 4522601988 AGGREGATION_NODE (id=7) - GetNewBlockTime: 36225146 - GetResultsTime: 7875774 - HTResizeTime: 45386772 - HashBuckets: 67584 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 13936086 - PinTime: 0 - ReductionFactorEstimate: 478.0824438635656 - ReductionFactorThresholdToExpand: 1.0999999999999999 - RowsPassedThrough: 0 - RowsReturned: 26239 - RowsReturnedRate: 90 - StreamingTime: 10310534943 - TotalTime: 288176224127 - UnpinTime: 0 HASH_JOIN_NODE (id=6) - BuildRows: 229 - BuildTime: 31737393 - InactiveTotalTime: 0 - LocalTime: 1420741931 - PeakMemoryUsage: 11215971 - ProbeRows: 992224 - ProbeRowsPartitioned: 0 - ProbeTime: 133645236 - RowsReturned: 272206 - RowsReturnedRate: 980 - TotalTime: 277745740733 Hash Join Builder (join_node_id=6) - BuildRowsPartitionTime: 247497 - BuildRowsPartitioned: 229 - GetNewBlockTime: 27788684 - HashBuckets: 496 - HashCollisions: 0 - HashTablesBuildTime: 1176829 - InactiveTotalTime: 0 - LargestPartitionPercent: 8 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1078144 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=13) - BytesReceived: 4099 - ConvertRowBatchTime: 67416 - DeserializeRowBatchTimer: 10813931 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 229 - RowsReturnedRate: 2526058 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 94583 HASH_JOIN_NODE (id=5) - BuildRows: 251 - BuildTime: 9782253 - InactiveTotalTime: 0 - LocalTime: 646297648 - PeakMemoryUsage: 3212998 - ProbeRows: 4354750 - ProbeRowsPartitioned: 0 - ProbeTime: 494890173 - RowsReturned: 992224 - RowsReturnedRate: 3593 - TotalTime: 276324904218 Hash Join Builder (join_node_id=5) - BuildRowsPartitionTime: 75476 - BuildRowsPartitioned: 251 - GetNewBlockTime: 7234864 - HashBuckets: 480 - HashCollisions: 0 - HashTablesBuildTime: 304510 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1077888 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=12) - BytesReceived: 2832 - ConvertRowBatchTime: 68339 - DeserializeRowBatchTimer: 19244830 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 251 - RowsReturnedRate: 2819161 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 92823 HASH_JOIN_NODE (id=4) - BuildRows: 520 - BuildTime: 26698994 - InactiveTotalTime: 0 - LocalTime: 6316084197 - PeakMemoryUsage: 2070272 - ProbeRows: 61776723 - ProbeRowsPartitioned: 0 - ProbeTime: 5507623769 - RowsReturned: 4354750 - RowsReturnedRate: 15799 - TotalTime: 275678513746 Hash Join Builder (join_node_id=4) - BuildRowsPartitionTime: 261493 - BuildRowsPartitioned: 520 - GetNewBlockTime: 24468922 - HashBuckets: 1152 - HashCollisions: 0 - HashTablesBuildTime: 504376 - InactiveTotalTime: 0 - LargestPartitionPercent: 9 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1088640 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=11) - BytesReceived: 7274 - ConvertRowBatchTime: 135065 - DeserializeRowBatchTimer: 2389208 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 520 - RowsReturnedRate: 3439532 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 162918 HDFS_SCAN_NODE (id=0) - AverageHdfsReadThreadConcurrency: 0.0015453649051517136 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 181226173 - BytesReadDataNodeCache: 0 - BytesReadLocal: 181226173 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 181226173 - DecompressionTime: 5473482243 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 4 - NumDisksAccessed: 12 - NumRowGroups: 136 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 13559856 - PerReadThreadRawHdfsThroughput: 407502299 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 267155212529 - RowBatchQueuePutWaitTime: 539720823 - RowsRead: 61776723 - RowsReturned: 61776723 - RowsReturnedRate: 229405 - ScanRangesComplete: 136 - ScannerThreadsInvoluntaryContextSwitches: 39003 - ScannerThreadsTotalWallClockTime: 287882919878 - MaterializeTupleTime(*): 132365470821 - ScannerThreadsSysTime: 1785852875 - ScannerThreadsUserTime: 48620483000 - ScannerThreadsVoluntaryContextSwitches: 28625 - TotalRawHdfsReadTime(*): 449003149 - TotalReadThroughput: 627544 - TotalTime: 269362266631 Fragment F01 Instance b149ee4c1d799434:d (host=redacted-host1:22000) Hdfs split stats (:<# splits>/): 8:7/145.56 MB 10:10/166.30 MB 2:12/250.88 MB 7:11/224.67 MB 6:11/295.95 MB 1:13/336.57 MB 4:14/245.80 MB 3:11/252.27 MB 0:12/280.17 MB 11:12/249.98 MB 9:9/183.13 MB 5:12/235.13 MB - AverageThreadTokens: 1.9794520547945205 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 29607584 - PerHostPeakMemUsage: 74501349 - RowsProduced: 20327 - TotalNetworkReceiveTime: 35240 - TotalNetworkSendTime: 87163336 - TotalStorageWaitTime: 131800531912 - TotalThreadsInvoluntaryContextSwitches: 37730 - TotalThreadsTotalWallClockTime: 587418172432 - TotalThreadsSysTime: 2863563000 - TotalThreadsUserTime: 57569247000 - TotalThreadsVoluntaryContextSwitches: 93895 - TotalTime: 297403060416 Fragment Instance Lifecycle Timings - ExecTime: 285443252840 - ExecTreeExecTime: 285315519104 - InactiveTotalTime: 0 - OpenTime: 10700448112 - ExecTreeOpenTime: 6072657640 - PrepareTime: 1259267040 - ExecTreePrepareTime: 272551160 - TotalTime: 0 DataStreamSender (dst_id=14) - BytesSent: 366985 - InactiveTotalTime: 0 - NetworkThroughput(*): 92270159 - OverallThroughput: 22047751 - PeakMemoryUsage: 271296 - RowsReturned: 20327 - SerializeBatchTime: 12510000 - TotalTime: 16645008 - TransmitDataRPCTime: 3977288 - UncompressedRowBatchSize: 589611 CodeGen - CodegenTime: 14791800 - CompileTime: 1712678480 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 121 - NumInstructions: 7493 - OptimizationTime: 2860261320 - PeakMemoryUsage: 3836416 - PrepareTime: 593912304 - TotalTime: 5179612968 AGGREGATION_NODE (id=7) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 112347384 - GetResultsTime: 7310880 - HTResizeTime: 13880768 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 14411844 - PinTime: 0 - ReductionFactorEstimate: 523.3504079782412 - ReductionFactorThresholdToExpand: 1.1 - RowsPassedThrough: 0 - RowsReturned: 20327 - RowsReturnedRate: 69 - StreamingTime: 10461907992 - TotalTime: 291660636192 - UnpinTime: 0 HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled - BuildRows: 229 - BuildTime: 55942184 - InactiveTotalTime: 0 - LocalTime: 954495528 - PeakMemoryUsage: 11157638 - ProbeRows: 767174 - ProbeRowsPartitioned: 0 - ProbeTime: 102514976 - RowsReturned: 210629 - RowsReturnedRate: 749 - TotalTime: 280971959960 Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 143248 - BuildRowsPartitioned: 229 - GetNewBlockTime: 54956208 - HashBuckets: 496 - HashCollisions: 0 - HashTablesBuildTime: 434224 - InactiveTotalTime: 0 - LargestPartitionPercent: 8 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1078144 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=13) - BytesReceived: 4099 - ConvertRowBatchTime: 65872 - DeserializeRowBatchTimer: 229504 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 229 - RowsReturnedRate: 2443865 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 93704 HASH_JOIN_NODE (id=5) ExecOption: Probe Side Codegen Enabled - BuildRows: 251 - BuildTime: 739072 - InactiveTotalTime: 0 - LocalTime: 582531696 - PeakMemoryUsage: 3223710 - ProbeRows: 4365887 - ProbeRowsPartitioned: 0 - ProbeTime: 467878448 - RowsReturned: 767174 - RowsReturnedRate: 2739 - TotalTime: 280017370728 Hash Join Builder (join_node_id=5) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 73440 - BuildRowsPartitioned: 251 - GetNewBlockTime: 66528 - HashBuckets: 480 - HashCollisions: 0 - HashTablesBuildTime: 320632 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1077888 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=12) - BytesReceived: 2832 - ConvertRowBatchTime: 70472 - DeserializeRowBatchTimer: 48726392 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 251 - RowsReturnedRate: 2664317 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 94208 HASH_JOIN_NODE (id=4) ExecOption: Probe Side Codegen Enabled - BuildRows: 520 - BuildTime: 19131048 - InactiveTotalTime: 0 - LocalTime: 6301759392 - PeakMemoryUsage: 2070272 - ProbeRows: 61976722 - ProbeRowsPartitioned: 0 - ProbeTime: 5581634512 - RowsReturned: 4365887 - RowsReturnedRate: 15623 - TotalTime: 279434744824 Hash Join Builder (join_node_id=4) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 252032 - BuildRowsPartitioned: 520 - GetNewBlockTime: 9760176 - HashBuckets: 1152 - HashCollisions: 0 - HashTablesBuildTime: 352568 - InactiveTotalTime: 0 - LargestPartitionPercent: 9 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1088640 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=11) - BytesReceived: 7274 - ConvertRowBatchTime: 119824 - DeserializeRowBatchTimer: 286384 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 520 - RowsReturnedRate: 3638195 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 142928 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 8:7/145.56 MB 10:10/166.30 MB 2:12/250.88 MB 7:11/224.67 MB 6:11/295.95 MB 1:13/336.57 MB 4:14/245.80 MB 3:11/252.27 MB 0:12/280.17 MB 11:12/249.98 MB 9:9/183.13 MB 5:12/235.13 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 134 out of 134 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:536 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 181506059 - BytesReadDataNodeCache: 0 - BytesReadLocal: 181506059 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 181506059 - DecompressionTime: 4404848776 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 4 - NumDisksAccessed: 12 - NumRowGroups: 134 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 13863990 - PerReadThreadRawHdfsThroughput: 350267946 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 271077843952 - RowBatchQueuePutWaitTime: 1510001752 - RowsRead: 61976722 - RowsReturned: 61976722 - RowsReturnedRate: 226910 - ScanRangesComplete: 134 - ScannerThreadsInvoluntaryContextSwitches: 34379 - ScannerThreadsTotalWallClockTime: 291283646072 - MaterializeTupleTime(*): 121255302376 - ScannerThreadsSysTime: 1809724000 - ScannerThreadsUserTime: 48059693000 - ScannerThreadsVoluntaryContextSwitches: 29385 - TotalRawHdfsReadTime(*): 518192032 - TotalReadThroughput: 621357 - TotalTime: 273132842504 Instance b149ee4c1d799434:10 (host=redacted-host6:22000) Hdfs split stats (:<# splits>/): 3:7/145.12 MB 1:12/234.99 MB 6:11/194.16 MB 7:8/215.53 MB 4:17/384.47 MB 8:8/186.35 MB 0:17/327.23 MB 2:8/187.35 MB 5:12/237.58 MB 10:13/290.33 MB 9:13/308.22 MB 11:8/142.34 MB - AverageThreadTokens: 1.9845890410958904 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 23185153 - PerHostPeakMemUsage: 70264786 - RowsProduced: 23521 - TotalNetworkReceiveTime: 29292 - TotalNetworkSendTime: 4028104 - TotalStorageWaitTime: 114134877928 - TotalThreadsInvoluntaryContextSwitches: 40572 - TotalThreadsTotalWallClockTime: 589333738592 - TotalThreadsSysTime: 2969548000 - TotalThreadsUserTime: 58792062000 - TotalThreadsVoluntaryContextSwitches: 93990 - TotalTime: 297384154612 Fragment Instance Lifecycle Timings - ExecTime: 292575511048 - ExecTreeExecTime: 292532749092 - InactiveTotalTime: 0 - OpenTime: 4128579892 - ExecTreeOpenTime: 244312976 - PrepareTime: 679975320 - ExecTreePrepareTime: 72658364 - TotalTime: 0 DataStreamSender (dst_id=14) - BytesSent: 424990 - InactiveTotalTime: 0 - NetworkThroughput(*): 90037659 - OverallThroughput: 23311080 - PeakMemoryUsage: 271296 - RowsReturned: 23521 - SerializeBatchTime: 13375304 - TotalTime: 18231244 - TransmitDataRPCTime: 4720136 - UncompressedRowBatchSize: 682269 CodeGen - CodegenTime: 72434116 - CompileTime: 643352528 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 121 - NumInstructions: 7493 - OptimizationTime: 3118759884 - PeakMemoryUsage: 3836416 - PrepareTime: 249333156 - TotalTime: 4060214712 AGGREGATION_NODE (id=7) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 3426908 - GetResultsTime: 7371452 - HTResizeTime: 7775952 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 13418783 - PinTime: 0 - ReductionFactorEstimate: 531.5751784480774 - ReductionFactorThresholdToExpand: 1.1 - RowsPassedThrough: 0 - RowsReturned: 23521 - RowsReturnedRate: 80 - StreamingTime: 12633944764 - TotalTime: 292849642540 - UnpinTime: 0 HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled - BuildRows: 229 - BuildTime: 11976052 - InactiveTotalTime: 0 - LocalTime: 220728428 - PeakMemoryUsage: 11411386 - ProbeRows: 893706 - ProbeRowsPartitioned: 0 - ProbeTime: 111647568 - RowsReturned: 244245 - RowsReturnedRate: 871 - TotalTime: 280126906372 Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 1002416 - BuildRowsPartitioned: 229 - GetNewBlockTime: 3698564 - HashBuckets: 496 - HashCollisions: 0 - HashTablesBuildTime: 6547396 - InactiveTotalTime: 0 - LargestPartitionPercent: 8 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1078144 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=13) - BytesReceived: 4099 - ConvertRowBatchTime: 54432 - DeserializeRowBatchTimer: 9343924 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 229 - RowsReturnedRate: 2990180 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 76584 HASH_JOIN_NODE (id=5) ExecOption: Probe Side Codegen Enabled - BuildRows: 251 - BuildTime: 700156 - InactiveTotalTime: 0 - LocalTime: 587360088 - PeakMemoryUsage: 3198998 - ProbeRows: 4354796 - ProbeRowsPartitioned: 0 - ProbeTime: 495419820 - RowsReturned: 893706 - RowsReturnedRate: 3192 - TotalTime: 279906101360 Hash Join Builder (join_node_id=5) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 53908 - BuildRowsPartitioned: 251 - GetNewBlockTime: 150132 - HashBuckets: 480 - HashCollisions: 0 - HashTablesBuildTime: 241464 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1077888 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=12) - BytesReceived: 2832 - ConvertRowBatchTime: 56588 - DeserializeRowBatchTimer: 45020 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 251 - RowsReturnedRate: 3227050 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 77780 HASH_JOIN_NODE (id=4) ExecOption: Probe Side Codegen Enabled - BuildRows: 520 - BuildTime: 3026908 - InactiveTotalTime: 0 - LocalTime: 6477951140 - PeakMemoryUsage: 2070272 - ProbeRows: 61686307 - ProbeRowsPartitioned: 0 - ProbeTime: 5660559220 - RowsReturned: 4354796 - RowsReturnedRate: 15590 - TotalTime: 279318663492 Hash Join Builder (join_node_id=4) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 202852 - BuildRowsPartitioned: 520 - GetNewBlockTime: 2137660 - HashBuckets: 1152 - HashCollisions: 0 - HashTablesBuildTime: 430040 - InactiveTotalTime: 0 - LargestPartitionPercent: 9 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1088640 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=11) - BytesReceived: 7274 - ConvertRowBatchTime: 112008 - DeserializeRowBatchTimer: 140312 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 520 - RowsReturnedRate: 3801614 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 136784 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 3:7/145.12 MB 1:12/234.99 MB 6:11/194.16 MB 7:8/215.53 MB 4:17/384.47 MB 8:8/186.35 MB 0:17/327.23 MB 2:8/187.35 MB 5:12/237.58 MB 10:13/290.33 MB 9:13/308.22 MB 11:8/142.34 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 134 out of 134 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:536 - AverageHdfsReadThreadConcurrency: 0.0034782608695652175 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 180876028 - BytesReadDataNodeCache: 0 - BytesReadLocal: 180876028 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180876028 - DecompressionTime: 6289460480 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 4 - NumDisksAccessed: 12 - NumRowGroups: 134 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 9376766 - PerReadThreadRawHdfsThroughput: 470137557 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 269375858412 - RowBatchQueuePutWaitTime: 0 - RowsRead: 61686307 - RowsReturned: 61686307 - RowsReturnedRate: 226089 - ScanRangesComplete: 134 - ScannerThreadsInvoluntaryContextSwitches: 37455 - ScannerThreadsTotalWallClockTime: 292722867876 - MaterializeTupleTime(*): 137233382172 - ScannerThreadsSysTime: 1907709000 - ScannerThreadsUserTime: 49162526000 - ScannerThreadsVoluntaryContextSwitches: 30050 - TotalRawHdfsReadTime(*): 384730012 - TotalReadThroughput: 619213 - TotalTime: 272840575568 Instance b149ee4c1d799434:11 (host=redacted-host4:22000) Hdfs split stats (:<# splits>/): 1:16/327.02 MB 0:16/312.59 MB 9:8/156.09 MB 6:13/276.39 MB 10:12/220.43 MB 2:9/169.15 MB 7:12/267.22 MB 4:15/286.00 MB 11:6/132.65 MB 5:17/372.34 MB 8:8/158.27 MB 3:9/169.67 MB - AverageThreadTokens: 1.9828178694158076 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 27501387 - PerHostPeakMemUsage: 74579634 - RowsProduced: 29128 - TotalNetworkReceiveTime: 31712 - TotalNetworkSendTime: 4000744 - TotalStorageWaitTime: 143728492320 - TotalThreadsInvoluntaryContextSwitches: 35699 - TotalThreadsTotalWallClockTime: 587372144816 - TotalThreadsSysTime: 2771578000 - TotalThreadsUserTime: 57468263000 - TotalThreadsVoluntaryContextSwitches: 92826 - TotalTime: 296971046128 Fragment Instance Lifecycle Timings - ExecTime: 231514682864 - ExecTreeExecTime: 231460946456 - InactiveTotalTime: 0 - OpenTime: 64261137064 - ExecTreeOpenTime: 60261740752 - PrepareTime: 1195003664 - ExecTreePrepareTime: 325700304 - TotalTime: 0 DataStreamSender (dst_id=14) - BytesSent: 526293 - InactiveTotalTime: 0 - NetworkThroughput(*): 89909778 - OverallThroughput: 23986387 - PeakMemoryUsage: 271296 - RowsReturned: 29128 - SerializeBatchTime: 15940304 - TotalTime: 21941320 - TransmitDataRPCTime: 5853568 - UncompressedRowBatchSize: 844904 CodeGen - CodegenTime: 19062384 - CompileTime: 1396151880 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 121 - NumInstructions: 7493 - OptimizationTime: 2485106904 - PeakMemoryUsage: 3836416 - PrepareTime: 493036096 - TotalTime: 4375247184 AGGREGATION_NODE (id=7) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 47521088 - GetResultsTime: 10291544 - HTResizeTime: 47010768 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 13814021 - PinTime: 0 - ReductionFactorEstimate: 503.87707569549275 - ReductionFactorThresholdToExpand: 1.1 - RowsPassedThrough: 0 - RowsReturned: 29128 - RowsReturnedRate: 99 - StreamingTime: 5545211072 - TotalTime: 292048298136 - UnpinTime: 0 HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled - BuildRows: 229 - BuildTime: 20916368 - InactiveTotalTime: 0 - LocalTime: 3365911336 - PeakMemoryUsage: 11166246 - ProbeRows: 1106942 - ProbeRowsPartitioned: 0 - ProbeTime: 148336240 - RowsReturned: 302033 - RowsReturnedRate: 1054 - TotalTime: 286289616656 Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 129096 - BuildRowsPartitioned: 229 - GetNewBlockTime: 11417112 - HashBuckets: 496 - HashCollisions: 0 - HashTablesBuildTime: 320280 - InactiveTotalTime: 0 - LargestPartitionPercent: 8 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1078144 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=13) - BytesReceived: 4099 - ConvertRowBatchTime: 55032 - DeserializeRowBatchTimer: 106264 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 229 - RowsReturnedRate: 2893460 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 79144 HASH_JOIN_NODE (id=5) ExecOption: Probe Side Codegen Enabled - BuildRows: 251 - BuildTime: 18031824 - InactiveTotalTime: 0 - LocalTime: 674557136 - PeakMemoryUsage: 3215808 - ProbeRows: 4333369 - ProbeRowsPartitioned: 0 - ProbeTime: 497279312 - RowsReturned: 1106942 - RowsReturnedRate: 3912 - TotalTime: 282923626176 Hash Join Builder (join_node_id=5) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 76944 - BuildRowsPartitioned: 251 - GetNewBlockTime: 17354520 - HashBuckets: 480 - HashCollisions: 0 - HashTablesBuildTime: 262528 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1077888 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=12) - BytesReceived: 2832 - ConvertRowBatchTime: 58864 - DeserializeRowBatchTimer: 43944 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 251 - RowsReturnedRate: 3049373 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 82312 HASH_JOIN_NODE (id=4) ExecOption: Probe Side Codegen Enabled - BuildRows: 520 - BuildTime: 1081816 - InactiveTotalTime: 0 - LocalTime: 6300109168 - PeakMemoryUsage: 2070272 - ProbeRows: 61491771 - ProbeRowsPartitioned: 0 - ProbeTime: 5484315832 - RowsReturned: 4333369 - RowsReturnedRate: 15353 - TotalTime: 282248986728 Hash Join Builder (join_node_id=4) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 217328 - BuildRowsPartitioned: 520 - GetNewBlockTime: 174056 - HashBuckets: 1152 - HashCollisions: 0 - HashTablesBuildTime: 453568 - InactiveTotalTime: 0 - LargestPartitionPercent: 9 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1088640 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=11) - BytesReceived: 7274 - ConvertRowBatchTime: 115848 - DeserializeRowBatchTimer: 7301512 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 520 - RowsReturnedRate: 3836619 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 135536 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 1:16/327.02 MB 0:16/312.59 MB 9:8/156.09 MB 6:13/276.39 MB 10:12/220.43 MB 2:9/169.15 MB 7:12/267.22 MB 4:15/286.00 MB 11:6/132.65 MB 5:17/372.34 MB 8:8/158.27 MB 3:9/169.67 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 141 out of 141 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:564 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 180592498 - BytesReadDataNodeCache: 0 - BytesReadLocal: 180592498 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180592498 - DecompressionTime: 4502998408 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 4 - NumDisksAccessed: 12 - NumRowGroups: 141 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 13945728 - PerReadThreadRawHdfsThroughput: 431920606 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 274323828960 - RowBatchQueuePutWaitTime: 2318000 - RowsRead: 61491771 - RowsReturned: 61491771 - RowsReturnedRate: 222837 - ScanRangesComplete: 141 - ScannerThreadsInvoluntaryContextSwitches: 32230 - ScannerThreadsTotalWallClockTime: 291647470616 - MaterializeTupleTime(*): 114488198376 - ScannerThreadsSysTime: 1753733000 - ScannerThreadsUserTime: 47975706000 - ScannerThreadsVoluntaryContextSwitches: 28385 - TotalRawHdfsReadTime(*): 418115032 - TotalReadThroughput: 620415 - TotalTime: 275948742024 Instance b149ee4c1d799434:b (host=redacted-host7:22000) Hdfs split stats (:<# splits>/): 9:9/227.87 MB 0:14/274.43 MB 8:12/266.82 MB 2:16/343.09 MB 11:7/145.25 MB 1:16/271.10 MB 6:8/214.95 MB 4:14/259.78 MB 3:8/172.24 MB 10:10/168.20 MB 7:13/277.63 MB 5:12/236.21 MB - AverageThreadTokens: 1.986159169550173 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 25168011 - PerHostPeakMemUsage: 70120109 - RowsProduced: 25160 - TotalNetworkReceiveTime: 34920 - TotalNetworkSendTime: 3565680 - TotalStorageWaitTime: 107013798040 - TotalThreadsInvoluntaryContextSwitches: 44320 - TotalThreadsTotalWallClockTime: 584050834376 - TotalThreadsSysTime: 2818571000 - TotalThreadsUserTime: 58990032000 - TotalThreadsVoluntaryContextSwitches: 91167 - TotalTime: 294468574488 Fragment Instance Lifecycle Timings - ExecTime: 289983190328 - ExecTreeExecTime: 289949954072 - InactiveTotalTime: 0 - OpenTime: 4001319464 - ExecTreeOpenTime: 240620024 - PrepareTime: 483986424 - ExecTreePrepareTime: 23336080 - TotalTime: 0 DataStreamSender (dst_id=14) - BytesSent: 454173 - InactiveTotalTime: 0 - NetworkThroughput(*): 100170312 - OverallThroughput: 31161559 - PeakMemoryUsage: 271296 - RowsReturned: 25160 - SerializeBatchTime: 9901960 - TotalTime: 14574784 - TransmitDataRPCTime: 4534008 - UncompressedRowBatchSize: 729800 CodeGen - CodegenTime: 14503096 - CompileTime: 814770992 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 121 - NumInstructions: 7493 - OptimizationTime: 2842082408 - PeakMemoryUsage: 3836416 - PrepareTime: 169446696 - TotalTime: 3827244416 AGGREGATION_NODE (id=7) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 3926856 - GetResultsTime: 5479624 - HTResizeTime: 103400000 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 14758139 - PinTime: 0 - ReductionFactorEstimate: 525.6044178969909 - ReductionFactorThresholdToExpand: 1.1 - RowsPassedThrough: 0 - RowsReturned: 25160 - RowsReturnedRate: 86 - StreamingTime: 11467938184 - TotalTime: 290213850600 - UnpinTime: 0 HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled - BuildRows: 229 - BuildTime: 13074072 - InactiveTotalTime: 0 - LocalTime: 354034064 - PeakMemoryUsage: 11109294 - ProbeRows: 956505 - ProbeRowsPartitioned: 0 - ProbeTime: 134750784 - RowsReturned: 261288 - RowsReturnedRate: 937 - TotalTime: 278716628208 Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 142840 - BuildRowsPartitioned: 229 - GetNewBlockTime: 4464744 - HashBuckets: 496 - HashCollisions: 0 - HashTablesBuildTime: 440872 - InactiveTotalTime: 0 - LargestPartitionPercent: 8 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1078144 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=13) - BytesReceived: 4099 - ConvertRowBatchTime: 77584 - DeserializeRowBatchTimer: 2723504 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 229 - RowsReturnedRate: 2133964 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 107312 HASH_JOIN_NODE (id=5) ExecOption: Probe Side Codegen Enabled - BuildRows: 251 - BuildTime: 25724752 - InactiveTotalTime: 0 - LocalTime: 671731672 - PeakMemoryUsage: 3169314 - ProbeRows: 4354897 - ProbeRowsPartitioned: 0 - ProbeTime: 505466088 - RowsReturned: 956505 - RowsReturnedRate: 3436 - TotalTime: 278362486832 Hash Join Builder (join_node_id=5) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 81480 - BuildRowsPartitioned: 251 - GetNewBlockTime: 24960920 - HashBuckets: 480 - HashCollisions: 0 - HashTablesBuildTime: 272240 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1077888 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=12) - BytesReceived: 2832 - ConvertRowBatchTime: 63264 - DeserializeRowBatchTimer: 98032 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 251 - RowsReturnedRate: 2867915 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 87520 HASH_JOIN_NODE (id=4) ExecOption: Probe Side Codegen Enabled - BuildRows: 520 - BuildTime: 1118944 - InactiveTotalTime: 0 - LocalTime: 6185661464 - PeakMemoryUsage: 2070272 - ProbeRows: 61724663 - ProbeRowsPartitioned: 0 - ProbeTime: 5498451784 - RowsReturned: 4354897 - RowsReturnedRate: 15682 - TotalTime: 277690667640 Hash Join Builder (join_node_id=4) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 238192 - BuildRowsPartitioned: 520 - GetNewBlockTime: 129736 - HashBuckets: 1152 - HashCollisions: 0 - HashTablesBuildTime: 434928 - InactiveTotalTime: 0 - LargestPartitionPercent: 9 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1088640 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=11) - BytesReceived: 7274 - ConvertRowBatchTime: 129440 - DeserializeRowBatchTimer: 162256 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 520 - RowsReturnedRate: 3426643 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 151752 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 9:9/227.87 MB 0:14/274.43 MB 8:12/266.82 MB 2:16/343.09 MB 11:7/145.25 MB 1:16/271.10 MB 6:8/214.95 MB 4:14/259.78 MB 3:8/172.24 MB 10:10/168.20 MB 7:13/277.63 MB 5:12/236.21 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 139 out of 139 Hdfs Read Thread Concurrency Bucket: 0:99.65% 1:0.3509% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:556 - AverageHdfsReadThreadConcurrency: 0.0017543859649122807 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 181206489 - BytesReadDataNodeCache: 0 - BytesReadLocal: 181206489 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 181206489 - DecompressionTime: 5733281848 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 4 - NumDisksAccessed: 12 - NumRowGroups: 139 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 14565537 - PerReadThreadRawHdfsThroughput: 372545481 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 269315273888 - RowBatchQueuePutWaitTime: 151463416 - RowsRead: 61724663 - RowsReturned: 61724663 - RowsReturnedRate: 227342 - ScanRangesComplete: 139 - ScannerThreadsInvoluntaryContextSwitches: 41487 - ScannerThreadsTotalWallClockTime: 290129822888 - MaterializeTupleTime(*): 138964532992 - ScannerThreadsSysTime: 1798726000 - ScannerThreadsUserTime: 49481477000 - ScannerThreadsVoluntaryContextSwitches: 27925 - TotalRawHdfsReadTime(*): 486400984 - TotalReadThroughput: 624031 - TotalTime: 271504854424 Instance b149ee4c1d799434:f (host=redacted-host8:22000) Hdfs split stats (:<# splits>/): 5:8/156.15 MB 2:10/211.50 MB 7:13/262.90 MB 9:14/288.51 MB 0:13/321.55 MB 10:10/152.83 MB 4:8/157.08 MB 1:10/152.99 MB 6:14/391.73 MB 11:7/129.74 MB 3:19/380.47 MB 8:12/252.57 MB - AverageThreadTokens: 1.9722703639514731 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 27059666 - PerHostPeakMemUsage: 29514737 - RowsProduced: 32416 - TotalNetworkReceiveTime: 30704 - TotalNetworkSendTime: 3031944 - TotalStorageWaitTime: 96751411528 - TotalThreadsInvoluntaryContextSwitches: 44404 - TotalThreadsTotalWallClockTime: 572881719864 - TotalThreadsSysTime: 2716587000 - TotalThreadsUserTime: 58353128000 - TotalThreadsVoluntaryContextSwitches: 90392 - TotalTime: 291235725392 Fragment Instance Lifecycle Timings - ExecTime: 265768644064 - ExecTreeExecTime: 265714582432 - InactiveTotalTime: 0 - OpenTime: 23988020224 - ExecTreeOpenTime: 17566422456 - PrepareTime: 1478977592 - ExecTreePrepareTime: 453845336 - TotalTime: 0 DataStreamSender (dst_id=14) - BytesSent: 585420 - InactiveTotalTime: 0 - NetworkThroughput(*): 99138368 - OverallThroughput: 27184563 - PeakMemoryUsage: 271296 - RowsReturned: 32416 - SerializeBatchTime: 15506712 - TotalTime: 21535016 - TransmitDataRPCTime: 5905080 - UncompressedRowBatchSize: 940284 CodeGen - CodegenTime: 10113608 - CompileTime: 2708876168 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 121 - NumInstructions: 7493 - OptimizationTime: 3588057728 - PeakMemoryUsage: 3836416 - PrepareTime: 457013232 - TotalTime: 6770582688 AGGREGATION_NODE (id=7) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 59215120 - GetResultsTime: 10245752 - HTResizeTime: 44606064 - HashBuckets: 81920 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 13481014 - PinTime: 0 - ReductionFactorEstimate: 142.55434075634156 - ReductionFactorThresholdToExpand: 1.1 - RowsPassedThrough: 0 - RowsReturned: 32416 - RowsReturnedRate: 114 - StreamingTime: 10695556416 - TotalTime: 283734769664 - UnpinTime: 0 HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled - BuildRows: 229 - BuildTime: 17403096 - InactiveTotalTime: 0 - LocalTime: 1540507752 - PeakMemoryUsage: 11467927 - ProbeRows: 1221986 - ProbeRowsPartitioned: 0 - ProbeTime: 178767024 - RowsReturned: 337313 - RowsReturnedRate: 1237 - TotalTime: 272558109928 Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 132480 - BuildRowsPartitioned: 229 - GetNewBlockTime: 16578464 - HashBuckets: 496 - HashCollisions: 0 - HashTablesBuildTime: 344184 - InactiveTotalTime: 0 - LargestPartitionPercent: 8 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1078144 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=13) - BytesReceived: 4099 - ConvertRowBatchTime: 67592 - DeserializeRowBatchTimer: 73428392 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 229 - RowsReturnedRate: 2513831 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 91096 HASH_JOIN_NODE (id=5) ExecOption: Probe Side Codegen Enabled - BuildRows: 251 - BuildTime: 897888 - InactiveTotalTime: 0 - LocalTime: 646404712 - PeakMemoryUsage: 3281688 - ProbeRows: 4354781 - ProbeRowsPartitioned: 0 - ProbeTime: 506840464 - RowsReturned: 1221986 - RowsReturnedRate: 4508 - TotalTime: 271017511080 Hash Join Builder (join_node_id=5) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 62624 - BuildRowsPartitioned: 251 - GetNewBlockTime: 158880 - HashBuckets: 480 - HashCollisions: 0 - HashTablesBuildTime: 390760 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1077888 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=12) - BytesReceived: 2832 - ConvertRowBatchTime: 67752 - DeserializeRowBatchTimer: 102253848 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 251 - RowsReturnedRate: 2809114 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 89352 HASH_JOIN_NODE (id=4) ExecOption: Probe Side Codegen Enabled - BuildRows: 520 - BuildTime: 22400864 - InactiveTotalTime: 0 - LocalTime: 6230999360 - PeakMemoryUsage: 2070272 - ProbeRows: 61742572 - ProbeRowsPartitioned: 0 - ProbeTime: 5441822504 - RowsReturned: 4354781 - RowsReturnedRate: 16106 - TotalTime: 270371017016 Hash Join Builder (join_node_id=4) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 270424 - BuildRowsPartitioned: 520 - GetNewBlockTime: 21212552 - HashBuckets: 1152 - HashCollisions: 0 - HashTablesBuildTime: 559984 - InactiveTotalTime: 0 - LargestPartitionPercent: 9 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1088640 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=11) - BytesReceived: 7274 - ConvertRowBatchTime: 106848 - DeserializeRowBatchTimer: 2552352 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 520 - RowsReturnedRate: 4059962 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 128080 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 5:8/156.15 MB 2:10/211.50 MB 7:13/262.90 MB 9:14/288.51 MB 0:13/321.55 MB 10:10/152.83 MB 4:8/157.08 MB 1:10/152.99 MB 6:14/391.73 MB 11:7/129.74 MB 3:19/380.47 MB 8:12/252.57 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 138 out of 138 Hdfs Read Thread Concurrency Bucket: 0:99.64% 1:0.3565% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:552 - AverageHdfsReadThreadConcurrency: 0.0017825311942959 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 181196984 - BytesReadDataNodeCache: 0 - BytesReadLocal: 181196984 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 181196984 - DecompressionTime: 6171325112 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 4 - NumDisksAccessed: 12 - NumRowGroups: 138 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 14564804 - PerReadThreadRawHdfsThroughput: 385898555 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 261945169640 - RowBatchQueuePutWaitTime: 508711304 - RowsRead: 61742572 - RowsReturned: 61742572 - RowsReturnedRate: 233749 - ScanRangesComplete: 138 - ScannerThreadsInvoluntaryContextSwitches: 41727 - ScannerThreadsTotalWallClockTime: 283166791832 - MaterializeTupleTime(*): 140099037888 - ScannerThreadsSysTime: 1761732000 - ScannerThreadsUserTime: 48706595000 - ScannerThreadsVoluntaryContextSwitches: 27503 - TotalRawHdfsReadTime(*): 469545640 - TotalReadThroughput: 628473 - TotalTime: 264139889576 Instance b149ee4c1d799434:c (host=redacted-host3:22000) Hdfs split stats (:<# splits>/): 5:12/279.98 MB 6:12/264.26 MB 7:8/142.35 MB 0:12/264.94 MB 8:14/319.68 MB 2:13/233.88 MB 4:11/194.32 MB 1:12/222.73 MB 11:14/333.26 MB 10:11/223.09 MB 3:10/182.97 MB 9:12/207.76 MB - AverageThreadTokens: 1.9877408056042032 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 25493059 - PerHostPeakMemUsage: 72516979 - RowsProduced: 29540 - TotalNetworkReceiveTime: 46024 - TotalNetworkSendTime: 2985568 - TotalStorageWaitTime: 111840150576 - TotalThreadsInvoluntaryContextSwitches: 43739 - TotalThreadsTotalWallClockTime: 577305777128 - TotalThreadsSysTime: 2769578000 - TotalThreadsUserTime: 58402121000 - TotalThreadsVoluntaryContextSwitches: 91448 - TotalTime: 290729190640 Fragment Instance Lifecycle Timings - ExecTime: 286721697744 - ExecTreeExecTime: 286677074416 - InactiveTotalTime: 0 - OpenTime: 3726156760 - ExecTreeOpenTime: 344592208 - PrepareTime: 281239664 - ExecTreePrepareTime: 14997984 - TotalTime: 0 DataStreamSender (dst_id=14) - BytesSent: 533688 - InactiveTotalTime: 0 - NetworkThroughput(*): 98436490 - OverallThroughput: 29633278 - PeakMemoryUsage: 271296 - RowsReturned: 29540 - SerializeBatchTime: 12468808 - TotalTime: 18009752 - TransmitDataRPCTime: 5421648 - UncompressedRowBatchSize: 856852 CodeGen - CodegenTime: 6320776 - CompileTime: 929892496 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 121 - NumInstructions: 7493 - OptimizationTime: 2427991720 - PeakMemoryUsage: 3836416 - PrepareTime: 88512560 - TotalTime: 3447108048 AGGREGATION_NODE (id=7) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 2278928 - GetResultsTime: 7499736 - HTResizeTime: 39465224 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 13656837 - PinTime: 0 - ReductionFactorEstimate: 551.045465309022 - ReductionFactorThresholdToExpand: 1.1 - RowsPassedThrough: 0 - RowsReturned: 29540 - RowsReturnedRate: 102 - StreamingTime: 12333429192 - TotalTime: 287036597336 - UnpinTime: 0 HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled - BuildRows: 229 - BuildTime: 62891840 - InactiveTotalTime: 0 - LocalTime: 263271696 - PeakMemoryUsage: 11039749 - ProbeRows: 1112198 - ProbeRowsPartitioned: 0 - ProbeTime: 140787672 - RowsReturned: 306610 - RowsReturnedRate: 1116 - TotalTime: 274683899008 Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 204864 - BuildRowsPartitioned: 229 - GetNewBlockTime: 61546680 - HashBuckets: 496 - HashCollisions: 0 - HashTablesBuildTime: 611912 - InactiveTotalTime: 0 - LargestPartitionPercent: 8 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1078144 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=13) - BytesReceived: 4099 - ConvertRowBatchTime: 112536 - DeserializeRowBatchTimer: 112904 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 229 - RowsReturnedRate: 1557229 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 147056 HASH_JOIN_NODE (id=5) ExecOption: Probe Side Codegen Enabled - BuildRows: 251 - BuildTime: 1144976 - InactiveTotalTime: 0 - LocalTime: 582153752 - PeakMemoryUsage: 3249622 - ProbeRows: 4367594 - ProbeRowsPartitioned: 0 - ProbeTime: 491186584 - RowsReturned: 1112198 - RowsReturnedRate: 4052 - TotalTime: 274420480256 Hash Join Builder (join_node_id=5) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 118328 - BuildRowsPartitioned: 251 - GetNewBlockTime: 202472 - HashBuckets: 480 - HashCollisions: 0 - HashTablesBuildTime: 455080 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1077888 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=12) - BytesReceived: 2832 - ConvertRowBatchTime: 116904 - DeserializeRowBatchTimer: 2714096 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 251 - RowsReturnedRate: 1675299 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 149824 HASH_JOIN_NODE (id=4) ExecOption: Probe Side Codegen Enabled - BuildRows: 520 - BuildTime: 58368392 - InactiveTotalTime: 0 - LocalTime: 6229003072 - PeakMemoryUsage: 2070272 - ProbeRows: 61962462 - ProbeRowsPartitioned: 0 - ProbeTime: 5395885920 - RowsReturned: 4367594 - RowsReturnedRate: 15949 - TotalTime: 273838176680 Hash Join Builder (join_node_id=4) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 427424 - BuildRowsPartitioned: 520 - GetNewBlockTime: 56333232 - HashBuckets: 1152 - HashCollisions: 0 - HashTablesBuildTime: 899928 - InactiveTotalTime: 0 - LargestPartitionPercent: 9 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1088640 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=11) - BytesReceived: 7274 - ConvertRowBatchTime: 249104 - DeserializeRowBatchTimer: 8337680 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 520 - RowsReturnedRate: 1654954 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 314208 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 5:12/279.98 MB 6:12/264.26 MB 7:8/142.35 MB 0:12/264.94 MB 8:14/319.68 MB 2:13/233.88 MB 4:11/194.32 MB 1:12/222.73 MB 11:14/333.26 MB 10:11/223.09 MB 3:10/182.97 MB 9:12/207.76 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 141 out of 141 Hdfs Read Thread Concurrency Bucket: 0:99.82% 1:0.1773% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:564 - AverageHdfsReadThreadConcurrency: 0.0017730496453900709 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 181872584 - BytesReadDataNodeCache: 0 - BytesReadLocal: 181872584 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 181872584 - DecompressionTime: 5348822760 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 4 - NumDisksAccessed: 12 - NumRowGroups: 141 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 14542828 - PerReadThreadRawHdfsThroughput: 398593733 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 265800135200 - RowBatchQueuePutWaitTime: 137723376 - RowsRead: 61962462 - RowsReturned: 61962462 - RowsReturnedRate: 231541 - ScanRangesComplete: 141 - ScannerThreadsInvoluntaryContextSwitches: 40714 - ScannerThreadsTotalWallClockTime: 286862274304 - MaterializeTupleTime(*): 136722390608 - ScannerThreadsSysTime: 1786728000 - ScannerThreadsUserTime: 48955557000 - ScannerThreadsVoluntaryContextSwitches: 28579 - TotalRawHdfsReadTime(*): 456285608 - TotalReadThroughput: 636735 - TotalTime: 267608859400 Instance b149ee4c1d799434:e (host=redacted-host2:22000) Hdfs split stats (:<# splits>/): 2:11/223.74 MB 0:12/264.11 MB 10:5/61.57 MB 3:14/303.91 MB 4:14/290.72 MB 6:8/157.20 MB 1:13/291.60 MB 7:9/156.04 MB 9:15/415.36 MB 8:9/255.22 MB 11:7/130.97 MB 5:15/315.72 MB - AverageThreadTokens: 1.9771929824561403 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 28221144 - PerHostPeakMemUsage: 73096566 - RowsProduced: 26520 - TotalNetworkReceiveTime: 36388 - TotalNetworkSendTime: 2762712 - TotalStorageWaitTime: 109746783976 - TotalThreadsInvoluntaryContextSwitches: 42145 - TotalThreadsTotalWallClockTime: 571616855488 - TotalThreadsSysTime: 2754580000 - TotalThreadsUserTime: 57146312000 - TotalThreadsVoluntaryContextSwitches: 92683 - TotalTime: 289913356380 Fragment Instance Lifecycle Timings - ExecTime: 280029866164 - ExecTreeExecTime: 279982702828 - InactiveTotalTime: 0 - OpenTime: 8265140556 - ExecTreeOpenTime: 3515607448 - PrepareTime: 1618247876 - ExecTreePrepareTime: 308267192 - TotalTime: 0 DataStreamSender (dst_id=14) - BytesSent: 479457 - InactiveTotalTime: 0 - NetworkThroughput(*): 99597626 - OverallThroughput: 27101587 - PeakMemoryUsage: 271296 - RowsReturned: 26520 - SerializeBatchTime: 12746792 - TotalTime: 17691104 - TransmitDataRPCTime: 4813940 - UncompressedRowBatchSize: 769264 CodeGen - CodegenTime: 47748124 - CompileTime: 1571318492 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 121 - NumInstructions: 7493 - OptimizationTime: 3063559920 - PeakMemoryUsage: 3836416 - PrepareTime: 637238480 - TotalTime: 5351236000 AGGREGATION_NODE (id=7) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 53101556 - GetResultsTime: 7262136 - HTResizeTime: 46986412 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 14284374 - PinTime: 0 - ReductionFactorEstimate: 522.4024943310658 - ReductionFactorThresholdToExpand: 1.1 - RowsPassedThrough: 0 - RowsReturned: 26520 - RowsReturnedRate: 93 - StreamingTime: 9805159348 - TotalTime: 283806506040 - UnpinTime: 0 HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled - BuildRows: 229 - BuildTime: 32191984 - InactiveTotalTime: 0 - LocalTime: 470388740 - PeakMemoryUsage: 11061048 - ProbeRows: 1002095 - ProbeRowsPartitioned: 0 - ProbeTime: 127809024 - RowsReturned: 274404 - RowsReturnedRate: 1000 - TotalTime: 274136323500 Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 115576 - BuildRowsPartitioned: 229 - GetNewBlockTime: 31272056 - HashBuckets: 496 - HashCollisions: 0 - HashTablesBuildTime: 340956 - InactiveTotalTime: 0 - LargestPartitionPercent: 8 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1078144 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=13) - BytesReceived: 4099 - ConvertRowBatchTime: 56400 - DeserializeRowBatchTimer: 454656 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 229 - RowsReturnedRate: 2698816 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 84852 HASH_JOIN_NODE (id=5) ExecOption: Probe Side Codegen Enabled - BuildRows: 251 - BuildTime: 30342000 - InactiveTotalTime: 0 - LocalTime: 806900468 - PeakMemoryUsage: 3137609 - ProbeRows: 4368740 - ProbeRowsPartitioned: 0 - ProbeTime: 487971424 - RowsReturned: 1002095 - RowsReturnedRate: 3661 - TotalTime: 273665849908 Hash Join Builder (join_node_id=5) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 77776 - BuildRowsPartitioned: 251 - GetNewBlockTime: 14809888 - HashBuckets: 480 - HashCollisions: 0 - HashTablesBuildTime: 266764 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1077888 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=12) - BytesReceived: 2832 - ConvertRowBatchTime: 49824 - DeserializeRowBatchTimer: 38904 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 251 - RowsReturnedRate: 3401452 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 73792 HASH_JOIN_NODE (id=4) ExecOption: Probe Side Codegen Enabled - BuildRows: 520 - BuildTime: 77722568 - InactiveTotalTime: 0 - LocalTime: 6538549484 - PeakMemoryUsage: 2070272 - ProbeRows: 61980494 - ProbeRowsPartitioned: 0 - ProbeTime: 5474384872 - RowsReturned: 4368740 - RowsReturnedRate: 16010 - TotalTime: 272858875648 Hash Join Builder (join_node_id=4) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 236024 - BuildRowsPartitioned: 520 - GetNewBlockTime: 76371052 - HashBuckets: 1152 - HashCollisions: 0 - HashTablesBuildTime: 477896 - InactiveTotalTime: 0 - LargestPartitionPercent: 9 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1088640 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=11) - BytesReceived: 7274 - ConvertRowBatchTime: 114504 - DeserializeRowBatchTimer: 150148 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 520 - RowsReturnedRate: 3759289 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 138324 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 2:11/223.74 MB 0:12/264.11 MB 10:5/61.57 MB 3:14/303.91 MB 4:14/290.72 MB 6:8/157.20 MB 1:13/291.60 MB 7:9/156.04 MB 9:15/415.36 MB 8:9/255.22 MB 11:7/130.97 MB 5:15/315.72 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 132 out of 132 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:528 - AverageHdfsReadThreadConcurrency: 0.0017953321364452424 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 181771354 - BytesReadDataNodeCache: 0 - BytesReadLocal: 181771354 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 181771354 - DecompressionTime: 5168181536 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 4 - NumDisksAccessed: 12 - NumRowGroups: 132 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 13829257 - PerReadThreadRawHdfsThroughput: 462429525 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 263887282548 - RowBatchQueuePutWaitTime: 523623464 - RowsRead: 61980494 - RowsReturned: 61980494 - RowsReturnedRate: 232729 - ScanRangesComplete: 132 - ScannerThreadsInvoluntaryContextSwitches: 39168 - ScannerThreadsTotalWallClockTime: 283324566172 - MaterializeTupleTime(*): 130492508836 - ScannerThreadsSysTime: 1756732000 - ScannerThreadsUserTime: 47691749000 - ScannerThreadsVoluntaryContextSwitches: 29143 - TotalRawHdfsReadTime(*): 393079040 - TotalReadThroughput: 637606 - TotalTime: 266320187840 Instance b149ee4c1d799434:12 (host=redacted-host5:22000) Hdfs split stats (:<# splits>/): 9:9/183.22 MB 10:9/155.38 MB 1:13/262.48 MB 6:14/317.38 MB 2:9/183.93 MB 11:8/199.99 MB 3:6/162.73 MB 4:10/211.75 MB 7:11/238.69 MB 0:15/301.75 MB 5:15/259.20 MB 8:15/375.56 MB - AverageThreadTokens: 1.9876977152899824 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 27234276 - PerHostPeakMemUsage: 74232755 - RowsProduced: 23304 - TotalNetworkReceiveTime: 35760 - TotalNetworkSendTime: 4547696 - TotalStorageWaitTime: 93810074848 - TotalThreadsInvoluntaryContextSwitches: 47771 - TotalThreadsTotalWallClockTime: 570975889960 - TotalThreadsSysTime: 2728584000 - TotalThreadsUserTime: 58530102000 - TotalThreadsVoluntaryContextSwitches: 90698 - TotalTime: 287488567120 Fragment Instance Lifecycle Timings - ExecTime: 248670587896 - ExecTreeExecTime: 248624749528 - InactiveTotalTime: 0 - OpenTime: 38386546080 - ExecTreeOpenTime: 35405980664 - PrepareTime: 431291560 - ExecTreePrepareTime: 28842152 - TotalTime: 0 DataStreamSender (dst_id=14) - BytesSent: 421015 - InactiveTotalTime: 0 - NetworkThroughput(*): 99594209 - OverallThroughput: 25688425 - PeakMemoryUsage: 271296 - RowsReturned: 23304 - SerializeBatchTime: 12058752 - TotalTime: 16389288 - TransmitDataRPCTime: 4227304 - UncompressedRowBatchSize: 675976 CodeGen - CodegenTime: 19659440 - CompileTime: 897210952 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 121 - NumInstructions: 7493 - OptimizationTime: 2061579280 - PeakMemoryUsage: 3836416 - PrepareTime: 196231840 - TotalTime: 3169569888 AGGREGATION_NODE (id=7) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 7983328 - GetResultsTime: 7545072 - HTResizeTime: 59968992 - HashBuckets: 65536 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 13663682 - PinTime: 0 - ReductionFactorEstimate: 524.2501704932939 - ReductionFactorThresholdToExpand: 1.1 - RowsPassedThrough: 0 - RowsReturned: 23304 - RowsReturnedRate: 82 - StreamingTime: 9541132576 - TotalTime: 284059492512 - UnpinTime: 0 HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled - BuildRows: 229 - BuildTime: 39503552 - InactiveTotalTime: 0 - LocalTime: 4196597904 - PeakMemoryUsage: 11314481 - ProbeRows: 877189 - ProbeRowsPartitioned: 0 - ProbeTime: 124548600 - RowsReturned: 241127 - RowsReturnedRate: 878 - TotalTime: 274482482232 Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 109456 - BuildRowsPartitioned: 229 - GetNewBlockTime: 38375648 - HashBuckets: 496 - HashCollisions: 0 - HashTablesBuildTime: 374808 - InactiveTotalTime: 0 - LargestPartitionPercent: 8 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1078144 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=13) - BytesReceived: 4099 - ConvertRowBatchTime: 49880 - DeserializeRowBatchTimer: 112304 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 229 - RowsReturnedRate: 2977119 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 76920 HASH_JOIN_NODE (id=5) ExecOption: Probe Side Codegen Enabled - BuildRows: 251 - BuildTime: 677360 - InactiveTotalTime: 0 - LocalTime: 618741664 - PeakMemoryUsage: 3227242 - ProbeRows: 4337939 - ProbeRowsPartitioned: 0 - ProbeTime: 507079248 - RowsReturned: 877189 - RowsReturnedRate: 3245 - TotalTime: 270285807408 Hash Join Builder (join_node_id=5) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 59312 - BuildRowsPartitioned: 251 - GetNewBlockTime: 175576 - HashBuckets: 480 - HashCollisions: 0 - HashTablesBuildTime: 226616 - InactiveTotalTime: 0 - LargestPartitionPercent: 7 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1077888 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=12) - BytesReceived: 2832 - ConvertRowBatchTime: 63048 - DeserializeRowBatchTimer: 38408 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 251 - RowsReturnedRate: 2858769 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 87800 HASH_JOIN_NODE (id=4) ExecOption: Probe Side Codegen Enabled - BuildRows: 520 - BuildTime: 30741416 - InactiveTotalTime: 0 - LocalTime: 6264640496 - PeakMemoryUsage: 2070272 - ProbeRows: 61648795 - ProbeRowsPartitioned: 0 - ProbeTime: 5523935512 - RowsReturned: 4337939 - RowsReturnedRate: 16086 - TotalTime: 269666977944 Hash Join Builder (join_node_id=4) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 247672 - BuildRowsPartitioned: 520 - GetNewBlockTime: 29632912 - HashBuckets: 1152 - HashCollisions: 0 - HashTablesBuildTime: 426096 - InactiveTotalTime: 0 - LargestPartitionPercent: 9 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1088640 - PinTime: 0 - RepartitionTime: 0 - SpilledPartitions: 0 - TotalTime: 0 - UnpinTime: 0 EXCHANGE_NODE (id=11) - BytesReceived: 7274 - ConvertRowBatchTime: 132944 - DeserializeRowBatchTimer: 183024 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 520 - RowsReturnedRate: 3338983 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 155736 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 9:9/183.22 MB 10:9/155.38 MB 1:13/262.48 MB 6:14/317.38 MB 2:9/183.93 MB 11:8/199.99 MB 3:6/162.73 MB 4:10/211.75 MB 7:11/238.69 MB 0:15/301.75 MB 5:15/259.20 MB 8:15/375.56 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 134 out of 134 Hdfs Read Thread Concurrency Bucket: 0:99.82% 1:0.1779% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:536 - AverageHdfsReadThreadConcurrency: 0.0017793594306049821 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 180787391 - BytesReadDataNodeCache: 0 - BytesReadLocal: 180787391 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180787391 - DecompressionTime: 6168939024 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 4 - NumDisksAccessed: 12 - NumRowGroups: 134 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 13789944 - PerReadThreadRawHdfsThroughput: 388224992 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 261516307632 - RowBatchQueuePutWaitTime: 1483925272 - RowsRead: 61648795 - RowsReturned: 61648795 - RowsReturnedRate: 234048 - ScanRangesComplete: 134 - ScannerThreadsInvoluntaryContextSwitches: 44868 - ScannerThreadsTotalWallClockTime: 283925919264 - MaterializeTupleTime(*): 139668413320 - ScannerThreadsSysTime: 1711739000 - ScannerThreadsUserTime: 48930561000 - ScannerThreadsVoluntaryContextSwitches: 28037 - TotalRawHdfsReadTime(*): 465676848 - TotalReadThroughput: 632526 - TotalTime: 263402181712 Averaged Fragment F02 split sizes: min: 41.12 KB, max: 41.12 KB, avg: 41.12 KB, stddev: 0 completion times: min:3s093ms max:3s093ms mean: 3s093ms stddev:0.000ns execution rates: min:13.29 KB/sec max:13.29 KB/sec mean:13.29 KB/sec stddev:0.41 B/sec num instances: 1 - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 632001 - PerHostPeakMemUsage: 30343001 - RowsProduced: 520 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 1091667688 - TotalStorageWaitTime: 19040048 - TotalThreadsInvoluntaryContextSwitches: 66 - TotalThreadsTotalWallClockTime: 1853263576 - TotalThreadsSysTime: 4998000 - TotalThreadsUserTime: 134979000 - TotalThreadsVoluntaryContextSwitches: 101 - TotalTime: 2324311896 Fragment Instance Lifecycle Timings - ExecTime: 1543658992 - ExecTreeExecTime: 141524400 - InactiveTotalTime: 0 - OpenTime: 184705688 - ExecTreeOpenTime: 254104 - PrepareTime: 595845344 - ExecTreePrepareTime: 2386824 - TotalTime: 0 DataStreamSender (dst_id=11) - BytesSent: 58192 - InactiveTotalTime: 0 - NetworkThroughput(*): 73636897 - OverallThroughput: 42001813 - PeakMemoryUsage: 31744 - RowsReturned: 520 - SerializeBatchTime: 517064 - TotalTime: 1385464 - TransmitDataRPCTime: 790256 - UncompressedRowBatchSize: 178912 CodeGen - CodegenTime: 33279232 - CompileTime: 24666208 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 42 - NumInstructions: 723 - OptimizationTime: 136485512 - PeakMemoryUsage: 370176 - PrepareTime: 347109416 - TotalTime: 509185080 HDFS_SCAN_NODE (id=3) - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 83074 - BytesReadDataNodeCache: 0 - BytesReadLocal: 83074 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 83074 - DecompressionTime: 31640 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 4 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 589003 - PerReadThreadRawHdfsThroughput: 208514889 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 125743672 - RowBatchQueuePutWaitTime: 0 - RowsRead: 7200 - RowsReturned: 520 - RowsReturnedRate: 3497 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 1 - ScannerThreadsTotalWallClockTime: 134565464 - MaterializeTupleTime(*): 84799944 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 9998000 - ScannerThreadsVoluntaryContextSwitches: 45 - TotalRawHdfsReadTime(*): 398408 - TotalReadThroughput: 0 - TotalTime: 148670584 Fragment F02 Instance b149ee4c1d799434:8 (host=redacted-host4:22000) Hdfs split stats (:<# splits>/): 7:1/41.12 KB - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 632001 - PerHostPeakMemUsage: 30343001 - RowsProduced: 520 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 1091667688 - TotalStorageWaitTime: 19040048 - TotalThreadsInvoluntaryContextSwitches: 66 - TotalThreadsTotalWallClockTime: 1853263576 - TotalThreadsSysTime: 4998000 - TotalThreadsUserTime: 134979000 - TotalThreadsVoluntaryContextSwitches: 101 - TotalTime: 2324311896 Fragment Instance Lifecycle Timings - ExecTime: 1543658992 - ExecTreeExecTime: 141524400 - InactiveTotalTime: 0 - OpenTime: 184705688 - ExecTreeOpenTime: 254104 - PrepareTime: 595845344 - ExecTreePrepareTime: 2386824 - TotalTime: 0 DataStreamSender (dst_id=11) - BytesSent: 58192 - InactiveTotalTime: 0 - NetworkThroughput(*): 73636897 - OverallThroughput: 42001813 - PeakMemoryUsage: 31744 - RowsReturned: 520 - SerializeBatchTime: 517064 - TotalTime: 1385464 - TransmitDataRPCTime: 790256 - UncompressedRowBatchSize: 178912 CodeGen - CodegenTime: 33279232 - CompileTime: 24666208 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 42 - NumInstructions: 723 - OptimizationTime: 136485512 - PeakMemoryUsage: 370176 - PrepareTime: 347109416 - TotalTime: 509185080 HDFS_SCAN_NODE (id=3) Hdfs split stats (:<# splits>/): 7:1/41.12 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:4 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 83074 - BytesReadDataNodeCache: 0 - BytesReadLocal: 83074 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 83074 - DecompressionTime: 31640 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 4 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 589003 - PerReadThreadRawHdfsThroughput: 208514889 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 125743672 - RowBatchQueuePutWaitTime: 0 - RowsRead: 7200 - RowsReturned: 520 - RowsReturnedRate: 3497 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 1 - ScannerThreadsTotalWallClockTime: 134565464 - MaterializeTupleTime(*): 84799944 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 9998000 - ScannerThreadsVoluntaryContextSwitches: 45 - TotalRawHdfsReadTime(*): 398408 - TotalReadThroughput: 0 - TotalTime: 148670584 Averaged Fragment F03 split sizes: min: 2.15 MB, max: 2.15 MB, avg: 2.15 MB, stddev: 0 completion times: min:2s915ms max:2s915ms mean: 2s915ms stddev:0.000ns execution rates: min:755.13 KB/sec max:755.13 KB/sec mean:755.13 KB/sec stddev:0.42 B/sec num instances: 1 - AverageThreadTokens: 1.25 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 3270928 - PerHostPeakMemUsage: 35247033 - RowsProduced: 251 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 452040844 - TotalStorageWaitTime: 113855380 - TotalThreadsInvoluntaryContextSwitches: 129 - TotalThreadsTotalWallClockTime: 1601561372 - TotalThreadsSysTime: 5998000 - TotalThreadsUserTime: 213967000 - TotalThreadsVoluntaryContextSwitches: 107 - TotalTime: 2122898248 Fragment Instance Lifecycle Timings - ExecTime: 988644412 - ExecTreeExecTime: 246453756 - InactiveTotalTime: 0 - OpenTime: 342173928 - ExecTreeOpenTime: 150000 - PrepareTime: 791949796 - ExecTreePrepareTime: 801696 - TotalTime: 0 DataStreamSender (dst_id=12) - BytesSent: 22656 - InactiveTotalTime: 0 - NetworkThroughput(*): 95767884 - OverallThroughput: 379051 - PeakMemoryUsage: 61632 - RowsReturned: 251 - SerializeBatchTime: 161060 - TotalTime: 59770172 - TransmitDataRPCTime: 236572 - UncompressedRowBatchSize: 42200 CodeGen - CodegenTime: 44206272 - CompileTime: 71079968 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 42 - NumInstructions: 794 - OptimizationTime: 236995608 - PeakMemoryUsage: 406528 - PrepareTime: 351370008 - TotalTime: 660368524 HDFS_SCAN_NODE (id=1) - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 480185 - BytesReadDataNodeCache: 0 - BytesReadLocal: 480185 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 480185 - DecompressionTime: 18372236 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 3197432 - PerReadThreadRawHdfsThroughput: 231678403 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 238386668 - RowBatchQueuePutWaitTime: 0 - RowsRead: 73049 - RowsReturned: 251 - RowsReturnedRate: 1008 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 28 - ScannerThreadsTotalWallClockTime: 278748364 - MaterializeTupleTime(*): 48160444 - ScannerThreadsSysTime: 1999000 - ScannerThreadsUserTime: 55991000 - ScannerThreadsVoluntaryContextSwitches: 30 - TotalRawHdfsReadTime(*): 2072636 - TotalReadThroughput: 0 - TotalTime: 248802752 Fragment F03 Instance b149ee4c1d799434:9 (host=redacted-host6:22000) Hdfs split stats (:<# splits>/): 1:1/2.15 MB - AverageThreadTokens: 1.25 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 3270928 - PerHostPeakMemUsage: 35247033 - RowsProduced: 251 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 452040844 - TotalStorageWaitTime: 113855380 - TotalThreadsInvoluntaryContextSwitches: 129 - TotalThreadsTotalWallClockTime: 1601561372 - TotalThreadsSysTime: 5998000 - TotalThreadsUserTime: 213967000 - TotalThreadsVoluntaryContextSwitches: 107 - TotalTime: 2122898248 Fragment Instance Lifecycle Timings - ExecTime: 988644412 - ExecTreeExecTime: 246453756 - InactiveTotalTime: 0 - OpenTime: 342173928 - ExecTreeOpenTime: 150000 - PrepareTime: 791949796 - ExecTreePrepareTime: 801696 - TotalTime: 0 DataStreamSender (dst_id=12) - BytesSent: 22656 - InactiveTotalTime: 0 - NetworkThroughput(*): 95767884 - OverallThroughput: 379051 - PeakMemoryUsage: 61632 - RowsReturned: 251 - SerializeBatchTime: 161060 - TotalTime: 59770172 - TransmitDataRPCTime: 236572 - UncompressedRowBatchSize: 42200 CodeGen - CodegenTime: 44206272 - CompileTime: 71079968 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 42 - NumInstructions: 794 - OptimizationTime: 236995608 - PeakMemoryUsage: 406528 - PrepareTime: 351370008 - TotalTime: 660368524 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 1:1/2.15 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:3 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 480185 - BytesReadDataNodeCache: 0 - BytesReadLocal: 480185 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 480185 - DecompressionTime: 18372236 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 3197432 - PerReadThreadRawHdfsThroughput: 231678403 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 238386668 - RowBatchQueuePutWaitTime: 0 - RowsRead: 73049 - RowsReturned: 251 - RowsReturnedRate: 1008 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 28 - ScannerThreadsTotalWallClockTime: 278748364 - MaterializeTupleTime(*): 48160444 - ScannerThreadsSysTime: 1999000 - ScannerThreadsUserTime: 55991000 - ScannerThreadsVoluntaryContextSwitches: 30 - TotalRawHdfsReadTime(*): 2072636 - TotalReadThroughput: 0 - TotalTime: 248802752 Averaged Fragment F04 split sizes: min: 71.36 KB, max: 71.36 KB, avg: 71.36 KB, stddev: 0 completion times: min:2s979ms max:2s979ms mean: 2s979ms stddev:0.000ns execution rates: min:23.95 KB/sec max:23.95 KB/sec mean:23.95 KB/sec stddev:0.38 B/sec num instances: 1 - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 310208 - PerHostPeakMemUsage: 30897977 - RowsProduced: 229 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 1143041872 - TotalStorageWaitTime: 8153384 - TotalThreadsInvoluntaryContextSwitches: 86 - TotalThreadsTotalWallClockTime: 1732156632 - TotalThreadsSysTime: 2999000 - TotalThreadsUserTime: 183972000 - TotalThreadsVoluntaryContextSwitches: 102 - TotalTime: 2263088080 Fragment Instance Lifecycle Timings - ExecTime: 1423816104 - ExecTreeExecTime: 14781352 - InactiveTotalTime: 0 - OpenTime: 299194880 - ExecTreeOpenTime: 197544 - PrepareTime: 539964880 - ExecTreePrepareTime: 282800 - TotalTime: 0 DataStreamSender (dst_id=13) - BytesSent: 32792 - InactiveTotalTime: 0 - NetworkThroughput(*): 95058092 - OverallThroughput: 61577982 - PeakMemoryUsage: 41920 - RowsReturned: 229 - SerializeBatchTime: 132784 - TotalTime: 532528 - TransmitDataRPCTime: 344968 - UncompressedRowBatchSize: 81288 CodeGen - CodegenTime: 6060080 - CompileTime: 54283480 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 33 - NumInstructions: 508 - OptimizationTime: 233902528 - PeakMemoryUsage: 260096 - PrepareTime: 272077912 - TotalTime: 582496040 HDFS_SCAN_NODE (id=2) - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 78030 - BytesReadDataNodeCache: 0 - BytesReadLocal: 78030 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 78030 - DecompressionTime: 8048 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 143360 - PerReadThreadRawHdfsThroughput: 1540633391 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 10127352 - RowBatchQueuePutWaitTime: 0 - RowsRead: 804 - RowsReturned: 229 - RowsReturnedRate: 14627 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 11330352 - MaterializeTupleTime(*): 456432 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 2999000 - ScannerThreadsVoluntaryContextSwitches: 13 - TotalRawHdfsReadTime(*): 50648 - TotalReadThroughput: 0 - TotalTime: 15655712 Fragment F04 Instance b149ee4c1d799434:a (host=redacted-host4:22000) Hdfs split stats (:<# splits>/): 9:1/71.36 KB - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 310208 - PerHostPeakMemUsage: 30897977 - RowsProduced: 229 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 1143041872 - TotalStorageWaitTime: 8153384 - TotalThreadsInvoluntaryContextSwitches: 86 - TotalThreadsTotalWallClockTime: 1732156632 - TotalThreadsSysTime: 2999000 - TotalThreadsUserTime: 183972000 - TotalThreadsVoluntaryContextSwitches: 102 - TotalTime: 2263088080 Fragment Instance Lifecycle Timings - ExecTime: 1423816104 - ExecTreeExecTime: 14781352 - InactiveTotalTime: 0 - OpenTime: 299194880 - ExecTreeOpenTime: 197544 - PrepareTime: 539964880 - ExecTreePrepareTime: 282800 - TotalTime: 0 DataStreamSender (dst_id=13) - BytesSent: 32792 - InactiveTotalTime: 0 - NetworkThroughput(*): 95058092 - OverallThroughput: 61577982 - PeakMemoryUsage: 41920 - RowsReturned: 229 - SerializeBatchTime: 132784 - TotalTime: 532528 - TransmitDataRPCTime: 344968 - UncompressedRowBatchSize: 81288 CodeGen - CodegenTime: 6060080 - CompileTime: 54283480 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2021884 - NumFunctions: 33 - NumInstructions: 508 - OptimizationTime: 233902528 - PeakMemoryUsage: 260096 - PrepareTime: 272077912 - TotalTime: 582496040 HDFS_SCAN_NODE (id=2) Hdfs split stats (:<# splits>/): 9:1/71.36 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% File Formats: PARQUET/SNAPPY:2 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 78030 - BytesReadDataNodeCache: 0 - BytesReadLocal: 78030 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 78030 - DecompressionTime: 8048 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 143360 - PerReadThreadRawHdfsThroughput: 1540633391 - RemoteScanRanges: 0 - RowBatchQueueGetWaitTime: 10127352 - RowBatchQueuePutWaitTime: 0 - RowsRead: 804 - RowsReturned: 229 - RowsReturnedRate: 14627 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 11330352 - MaterializeTupleTime(*): 456432 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 2999000 - ScannerThreadsVoluntaryContextSwitches: 13 - TotalRawHdfsReadTime(*): 50648 - TotalReadThroughput: 0 - TotalTime: 15655712