Query (id=844a0200d7876345:20bb38b900000000): Summary: Session ID: b34dadc0b4cd9634:e123e3ca7d888199 Session Type: BEESWAX Start Time: 2018-03-12 23:22:57.366843000 End Time: 2018-03-13 07:24:36.478143000 Query Type: DML Query State: EXCEPTION Query Status: Kudu error(s) reported, first error: Timed out: Failed to write batch of 38590 ops to tablet b8431200388d486995a4426c88bc06a2 after 1 attempt(s): Failed to write to server: a260dca5a9c846e99cb621881a7b86b8 (vc1515.foo.com:7050): Write RPC to 10.17.221.25:7050 timed out after 180.000s (SENT) Impala Version: impalad version 2.11.0-SNAPSHOT RELEASE (build d0bf63a83dc4bd5effd84b52217827d5c4c5b1e3) User: systest@foo.COM Connected User: systest@foo.COM Delegated User: Network Address: 10.17.187.36:51187 Default Db: tpch_30000_kudu Sql Statement: insert INTO TABLE lineitem SELECT L_SHIPDATE, L_ORDERKEY, L_LINENUMBER, L_PARTKEY, L_SUPPKEY, L_QUANTITY, L_EXTENDEDPRICE, L_DISCOUNT, L_TAX, L_RETURNFLAG, L_LINESTATUS, L_COMMITDATE, L_RECEIPTDATE, L_SHIPINSTRUCT, L_SHIPMODE, L_COMMENT FROM tpch_30000_parquet.lineitem Coordinator: va1026.foo.com:22000 Query Options (set by configuration): Query Options (set by configuration and planner): MT_DOP=0 Plan: ---------------- Max Per-Host Resource Reservation: Memory=4.00MB Per-Host Resource Estimates: Memory=1.41GB F01:PLAN FRAGMENT [KUDU(KuduPartition(L_ORDERKEY))] hosts=129 instances=129 | Per-Host Resources: mem-estimate=128.00MB mem-reservation=4.00MB INSERT INTO KUDU [tpch_30000_kudu.lineitem] | mem-estimate=0B mem-reservation=0B | 02:PARTIAL SORT | order by: KuduPartition(L_ORDERKEY) ASC NULLS LAST, l_shipdate ASC NULLS LAST, l_orderkey ASC NULLS LAST, l_linenumber ASC NULLS LAST | materialized: KuduPartition(L_ORDERKEY) | mem-estimate=128.00MB mem-reservation=4.00MB spill-buffer=2.00MB | tuple-ids=2 row-size=260B cardinality=179999978268 | 01:EXCHANGE [KUDU(KuduPartition(L_ORDERKEY))] | mem-estimate=0B mem-reservation=0B | tuple-ids=0 row-size=256B cardinality=179999978268 | F00:PLAN FRAGMENT [RANDOM] hosts=129 instances=129 Per-Host Resources: mem-estimate=1.29GB mem-reservation=0B 00:SCAN HDFS [tpch_30000_parquet.lineitem, RANDOM] partitions=2526/2526 files=28976 size=6.89TB stored statistics: table: rows=179999978268 size=unavailable partitions: 2526/2526 rows=179999978268 columns: all extrapolated-rows=disabled mem-estimate=1.29GB mem-reservation=0B tuple-ids=0 row-size=256B cardinality=179999978268 ---------------- Estimated Per-Host Mem: 1518338048 Per Host Min Reservation: va1026.foo.com:22000(4.00 MB) va1027.foo.com:22000(4.00 MB) va1028.foo.com:22000(4.00 MB) va1029.foo.com:22000(4.00 MB) va1030.foo.com:22000(4.00 MB) va1031.foo.com:22000(4.00 MB) va1032.foo.com:22000(4.00 MB) va1033.foo.com:22000(4.00 MB) va1034.foo.com:22000(4.00 MB) va1035.foo.com:22000(4.00 MB) va1036.foo.com:22000(4.00 MB) va1037.foo.com:22000(4.00 MB) va1039.foo.com:22000(4.00 MB) va1041.foo.com:22000(4.00 MB) va1042.foo.com:22000(4.00 MB) va1043.foo.com:22000(4.00 MB) va1044.foo.com:22000(4.00 MB) va1045.foo.com:22000(4.00 MB) va1046.foo.com:22000(4.00 MB) vc1501.foo.com:22000(4.00 MB) vc1502.foo.com:22000(4.00 MB) vc1503.foo.com:22000(4.00 MB) vc1504.foo.com:22000(4.00 MB) vc1505.foo.com:22000(4.00 MB) vc1507.foo.com:22000(4.00 MB) vc1508.foo.com:22000(4.00 MB) vc1509.foo.com:22000(4.00 MB) vc1510.foo.com:22000(4.00 MB) vc1511.foo.com:22000(4.00 MB) vc1513.foo.com:22000(4.00 MB) vc1515.foo.com:22000(4.00 MB) vc1516.foo.com:22000(4.00 MB) vc1517.foo.com:22000(4.00 MB) vc1518.foo.com:22000(4.00 MB) vc1519.foo.com:22000(4.00 MB) vc1520.foo.com:22000(4.00 MB) vc1522.foo.com:22000(4.00 MB) vc1523.foo.com:22000(4.00 MB) vc1524.foo.com:22000(4.00 MB) vc1525.foo.com:22000(4.00 MB) vc1526.foo.com:22000(4.00 MB) vc1527.foo.com:22000(4.00 MB) vc1528.foo.com:22000(4.00 MB) vc1529.foo.com:22000(4.00 MB) vc1531.foo.com:22000(4.00 MB) vc1532.foo.com:22000(4.00 MB) vc1533.foo.com:22000(4.00 MB) vc1534.foo.com:22000(4.00 MB) vc1535.foo.com:22000(4.00 MB) vc1536.foo.com:22000(4.00 MB) vc1537.foo.com:22000(4.00 MB) vc1538.foo.com:22000(4.00 MB) vc1539.foo.com:22000(4.00 MB) vc1540.foo.com:22000(4.00 MB) vc1541.foo.com:22000(4.00 MB) vc1542.foo.com:22000(4.00 MB) vd1101.foo.com:22000(4.00 MB) vd1102.foo.com:22000(4.00 MB) vd1103.foo.com:22000(4.00 MB) vd1104.foo.com:22000(4.00 MB) vd1105.foo.com:22000(4.00 MB) vd1107.foo.com:22000(4.00 MB) vd1108.foo.com:22000(4.00 MB) vd1109.foo.com:22000(4.00 MB) vd1110.foo.com:22000(4.00 MB) vd1111.foo.com:22000(4.00 MB) vd1112.foo.com:22000(4.00 MB) vd1113.foo.com:22000(4.00 MB) vd1114.foo.com:22000(4.00 MB) vd1115.foo.com:22000(4.00 MB) vd1116.foo.com:22000(4.00 MB) vd1117.foo.com:22000(4.00 MB) vd1118.foo.com:22000(4.00 MB) vd1119.foo.com:22000(4.00 MB) vd1120.foo.com:22000(4.00 MB) vd1121.foo.com:22000(4.00 MB) vd1122.foo.com:22000(4.00 MB) vd1123.foo.com:22000(4.00 MB) vd1124.foo.com:22000(4.00 MB) vd1125.foo.com:22000(4.00 MB) vd1126.foo.com:22000(4.00 MB) vd1127.foo.com:22000(4.00 MB) vd1129.foo.com:22000(4.00 MB) vd1130.foo.com:22000(4.00 MB) vd1131.foo.com:22000(4.00 MB) vd1132.foo.com:22000(4.00 MB) vd1133.foo.com:22000(4.00 MB) vd1134.foo.com:22000(4.00 MB) vd1135.foo.com:22000(4.00 MB) vd1136.foo.com:22000(4.00 MB) vd1138.foo.com:22000(4.00 MB) vd1139.foo.com:22000(4.00 MB) vd1140.foo.com:22000(4.00 MB) vd1141.foo.com:22000(4.00 MB) vd1142.foo.com:22000(4.00 MB) vd1326.foo.com:22000(4.00 MB) vd1327.foo.com:22000(4.00 MB) vd1328.foo.com:22000(4.00 MB) vd1329.foo.com:22000(4.00 MB) vd1330.foo.com:22000(4.00 MB) vd1331.foo.com:22000(4.00 MB) vd1332.foo.com:22000(4.00 MB) vd1333.foo.com:22000(4.00 MB) vd1334.foo.com:22000(4.00 MB) vd1335.foo.com:22000(4.00 MB) vd1336.foo.com:22000(4.00 MB) vd1337.foo.com:22000(4.00 MB) vd1338.foo.com:22000(4.00 MB) vd1339.foo.com:22000(4.00 MB) vd1340.foo.com:22000(4.00 MB) vd1341.foo.com:22000(4.00 MB) vd1342.foo.com:22000(4.00 MB) ve1125.foo.com:22000(4.00 MB) ve1126.foo.com:22000(4.00 MB) ve1127.foo.com:22000(4.00 MB) ve1128.foo.com:22000(4.00 MB) ve1129.foo.com:22000(4.00 MB) ve1130.foo.com:22000(4.00 MB) ve1131.foo.com:22000(4.00 MB) ve1132.foo.com:22000(4.00 MB) ve1134.foo.com:22000(4.00 MB) ve1135.foo.com:22000(4.00 MB) ve1136.foo.com:22000(4.00 MB) ve1137.foo.com:22000(4.00 MB) ve1138.foo.com:22000(4.00 MB) ve1139.foo.com:22000(4.00 MB) ve1140.foo.com:22000(4.00 MB) ve1141.foo.com:22000(4.00 MB) ve1142.foo.com:22000(4.00 MB) Request Pool: root.default Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ---------------------------------------------------------------------------------------------------------------------------- 02:PARTIAL SORT 129 19m2s 1h42m 8.63B 180.00B 161.38 GB 128.00 MB 01:EXCHANGE 129 49s226ms 53s226ms 77.10B 180.00B 19.54 MB 0 KUDU(KuduPartition(L_ORDERKEY)) 00:SCAN HDFS 129 10s019ms 21s585ms 77.10B 180.00B 14.01 GB 1.29 GB tpch_30000_parquet.lineitem Errors: Error in Kudu table 'impala::tpch_30000_kudu.lineitem': Timed out: Failed to write batch of 38590 ops to tablet b8431200388d486995a4426c88bc06a2 after 1 attempt(s): Failed to write to server: a260dca5a9c846e99cb621881a7b86b8 (vc1515.foo.com:7050): Write RPC to 10.17.221.25:7050 timed out after 180.000s (SENT) (1 of 38590 similar) Query Compilation: 131.797ms - Metadata of all 2 tables cached: 1.053ms (1.053ms) - Analysis finished: 2.971ms (1.918ms) - Value transfer graph computed: 3.204ms (232.783us) - Single node plan created: 112.664ms (109.460ms) - Runtime filters computed: 112.748ms (83.895us) - Distributed plan created: 113.671ms (923.018us) - Lineage info computed: 114.099ms (428.030us) - Planning finished: 131.797ms (17.697ms) Query Timeline: 8h1m - Query submitted: 58.075us (58.075us) - Planning finished: 243.661ms (243.603ms) - Submit for admission: 320.031ms (76.370ms) - Completed admission: 320.145ms (113.463us) - Ready to start on 129 backends: 337.957ms (17.812ms) - All 129 execution backends (258 fragment instances) started: 626.366ms (288.409ms) - Released admission control resources: 8h1m (8h1m) - Request finished: 8h1m (361.424ms) - Unregister query: 8h1m (129.233ms) - ComputeScanRangeAssignmentTimer: 48.666ms ImpalaServer: - ClientFetchWaitTimer: 0.000ns - RowMaterializationTimer: 0.000ns Execution Profile 844a0200d7876345:20bb38b900000000:(Total: 8h1m, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- Backend startup latencies: Count: 129, min / max: 25ms / 142ms, 25th %-ile: 80ms, 50th %-ile: 103ms, 75th %-ile: 120ms, 90th %-ile: 131ms, 95th %-ile: 135ms, 99.9th %-ile: 142ms Per Node Peak Memory Usage: va1026.foo.com:22000(160.62 GB) va1027.foo.com:22000(175.34 GB) va1028.foo.com:22000(175.25 GB) va1029.foo.com:22000(175.11 GB) va1030.foo.com:22000(175.33 GB) va1031.foo.com:22000(175.39 GB) va1032.foo.com:22000(160.65 GB) va1033.foo.com:22000(175.26 GB) va1034.foo.com:22000(160.71 GB) va1035.foo.com:22000(160.64 GB) va1036.foo.com:22000(160.73 GB) va1037.foo.com:22000(175.24 GB) va1039.foo.com:22000(175.34 GB) va1041.foo.com:22000(160.70 GB) va1042.foo.com:22000(160.69 GB) va1043.foo.com:22000(175.36 GB) va1044.foo.com:22000(175.37 GB) va1045.foo.com:22000(160.66 GB) va1046.foo.com:22000(175.17 GB) vc1501.foo.com:22000(175.35 GB) vc1502.foo.com:22000(175.37 GB) vc1503.foo.com:22000(160.70 GB) vc1504.foo.com:22000(160.71 GB) vc1505.foo.com:22000(175.41 GB) vc1507.foo.com:22000(175.26 GB) vc1508.foo.com:22000(160.57 GB) vc1509.foo.com:22000(160.73 GB) vc1510.foo.com:22000(160.57 GB) vc1511.foo.com:22000(175.34 GB) vc1513.foo.com:22000(160.65 GB) vc1515.foo.com:22000(160.60 GB) vc1516.foo.com:22000(160.69 GB) vc1517.foo.com:22000(160.68 GB) vc1518.foo.com:22000(160.62 GB) vc1519.foo.com:22000(175.35 GB) vc1520.foo.com:22000(160.59 GB) vc1522.foo.com:22000(160.54 GB) vc1523.foo.com:22000(160.58 GB) vc1524.foo.com:22000(175.31 GB) vc1525.foo.com:22000(160.62 GB) vc1526.foo.com:22000(160.70 GB) vc1527.foo.com:22000(160.68 GB) vc1528.foo.com:22000(160.63 GB) vc1529.foo.com:22000(160.71 GB) vc1531.foo.com:22000(175.28 GB) vc1532.foo.com:22000(175.34 GB) vc1533.foo.com:22000(175.40 GB) vc1534.foo.com:22000(175.18 GB) vc1535.foo.com:22000(160.71 GB) vc1536.foo.com:22000(160.71 GB) vc1537.foo.com:22000(175.25 GB) vc1538.foo.com:22000(160.70 GB) vc1539.foo.com:22000(160.68 GB) vc1540.foo.com:22000(160.69 GB) vc1541.foo.com:22000(175.38 GB) vc1542.foo.com:22000(160.66 GB) vd1101.foo.com:22000(175.27 GB) vd1102.foo.com:22000(160.55 GB) vd1103.foo.com:22000(175.37 GB) vd1104.foo.com:22000(160.68 GB) vd1105.foo.com:22000(175.28 GB) vd1107.foo.com:22000(175.40 GB) vd1108.foo.com:22000(160.68 GB) vd1109.foo.com:22000(160.70 GB) vd1110.foo.com:22000(175.25 GB) vd1111.foo.com:22000(160.68 GB) vd1112.foo.com:22000(175.39 GB) vd1113.foo.com:22000(175.24 GB) vd1114.foo.com:22000(160.72 GB) vd1115.foo.com:22000(160.73 GB) vd1116.foo.com:22000(175.39 GB) vd1117.foo.com:22000(175.36 GB) vd1118.foo.com:22000(175.36 GB) vd1119.foo.com:22000(160.72 GB) vd1120.foo.com:22000(160.71 GB) vd1121.foo.com:22000(160.69 GB) vd1122.foo.com:22000(160.65 GB) vd1123.foo.com:22000(175.28 GB) vd1124.foo.com:22000(175.37 GB) vd1125.foo.com:22000(160.67 GB) vd1126.foo.com:22000(160.70 GB) vd1127.foo.com:22000(159.44 GB) vd1129.foo.com:22000(174.09 GB) vd1130.foo.com:22000(159.44 GB) vd1131.foo.com:22000(159.40 GB) vd1132.foo.com:22000(174.09 GB) vd1133.foo.com:22000(159.45 GB) vd1134.foo.com:22000(159.44 GB) vd1135.foo.com:22000(159.46 GB) vd1136.foo.com:22000(174.06 GB) vd1138.foo.com:22000(174.09 GB) vd1139.foo.com:22000(174.01 GB) vd1140.foo.com:22000(159.48 GB) vd1141.foo.com:22000(174.13 GB) vd1142.foo.com:22000(159.43 GB) vd1326.foo.com:22000(174.11 GB) vd1327.foo.com:22000(174.03 GB) vd1328.foo.com:22000(159.42 GB) vd1329.foo.com:22000(159.46 GB) vd1330.foo.com:22000(174.13 GB) vd1331.foo.com:22000(174.11 GB) vd1332.foo.com:22000(159.41 GB) vd1333.foo.com:22000(174.12 GB) vd1334.foo.com:22000(174.11 GB) vd1335.foo.com:22000(174.06 GB) vd1336.foo.com:22000(159.42 GB) vd1337.foo.com:22000(159.44 GB) vd1338.foo.com:22000(159.35 GB) vd1339.foo.com:22000(159.48 GB) vd1340.foo.com:22000(159.41 GB) vd1341.foo.com:22000(159.47 GB) vd1342.foo.com:22000(159.47 GB) ve1125.foo.com:22000(159.24 GB) ve1126.foo.com:22000(159.41 GB) ve1127.foo.com:22000(159.42 GB) ve1128.foo.com:22000(159.42 GB) ve1129.foo.com:22000(159.43 GB) ve1130.foo.com:22000(159.45 GB) ve1131.foo.com:22000(159.41 GB) ve1132.foo.com:22000(159.44 GB) ve1134.foo.com:22000(159.46 GB) ve1135.foo.com:22000(159.39 GB) ve1136.foo.com:22000(174.06 GB) ve1137.foo.com:22000(159.44 GB) ve1138.foo.com:22000(159.45 GB) ve1139.foo.com:22000(159.40 GB) ve1140.foo.com:22000(159.44 GB) ve1141.foo.com:22000(159.46 GB) ve1142.foo.com:22000(159.43 GB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns - NumBackends: 129 (129) - NumFragmentInstances: 258 (258) - NumFragments: 2 (2) Averaged Fragment F01:(Total: 3m45s, non-child: 0.000ns, % non-child: 0.00%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:8h1m max:8h1m mean: 8h1m stddev:99.532ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 129 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 152.44 GB (163678305103) - PeakReservation: 152.40 GB (163638299497) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 165.88 GB (178117160539) - RowsProduced: 66.90M (66895770) - TotalNetworkReceiveTime: 5h46m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 4.61K (4614) - TotalThreadsTotalWallClockTime: 3m44s - TotalThreadsSysTime: 2s109ms - TotalThreadsUserTime: 31s117ms - TotalThreadsVoluntaryContextSwitches: 22.40K (22403) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 3m43s - ExecTreeExecTime: 1h29m - OpenTime: 999.782ms - ExecTreeOpenTime: 940.114ms - PrepareTime: 209.226ms - ExecTreePrepareTime: 172.232us KuduTableSink:(Total: 12m1s, non-child: 12m1s, % non-child: 100.00%) - KuduApplyTimer: 9m50s - NumRowErrors: 299 (299) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 19.88 K/sec - TotalNumRows: 66.90M (66895595) SORT_NODE (id=2):(Total: 1h29m, non-child: 0.000ns, % non-child: 0.00%) - InMemorySortTime: 7m21s - PeakMemoryUsage: 152.40 GB (163638309467) - RowsReturned: 66.90M (66895770) - RowsReturnedRate: 3.02 K/sec - RunsCreated: 1 (1) - SortDataSize: 35.03 GB (37610383555) Buffer pool: - AllocTime: 31m28s - CumulativeAllocationBytes: 152.40 GB (163639258659) - CumulativeAllocations: 78.03K (78029) - PeakReservation: 152.40 GB (163638299497) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 152.40 GB (163638299497) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h46m, non-child: 5h46m, % non-child: 100.00%) - ConvertRowBatchTime: 20s406ms - PeakMemoryUsage: 19.53 MB (20482778) - RowsReturned: 597.66M (597655328) - RowsReturnedRate: 29.07 K/sec Buffer pool: - AllocTime: 3h8m - CumulativeAllocationBytes: 271.42 GB (291430353642) - CumulativeAllocations: 14.23M (14229997) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: - FirstBatchArrivalWaitTime: 940.034ms - TotalBytesReceived: 154.06 GB (165423556871) - TotalGetBatchTime: 5h46m - DataArrivalTimer: 5h46m SenderSide: - DeserializeRowBatchTime: 3h19m - NumBatchesArrived: 7.12M (7115004) - NumBatchesDeferred: 91.40K (91401) - NumBatchesEnqueued: 7.11M (7114998) - NumBatchesReceived: 7.11M (7114998) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 58.319ms, non-child: 58.319ms, % non-child: 100.00%) - CodegenTime: 1.146ms - CompileTime: 5.269ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 30.202ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.261ms Fragment F01: Instance 844a0200d7876345:20bb38b90000008b (host=vd1132.foo.com:22000):(Total: 8h1m, non-child: 615.134ms, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 8h1m - Prepare Finished: 217.057ms (217.057ms) - Open Finished: 1s201ms (983.996ms) - First Batch Produced: 7h17m (7h17m) - First Batch Sent: 7h17m (10.530ms) - ExecInternal Finished: 8h1m (43m46s) MemoryUsage(8m32s): 22.82 GB, 40.35 GB, 49.70 GB, 57.96 GB, 65.53 GB, 72.20 GB, 77.96 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.36 GB, 102.67 GB, 106.88 GB, 111.03 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.71 GB, 128.94 GB, 132.06 GB, 135.09 GB, 137.96 GB, 140.71 GB, 143.56 GB, 146.32 GB, 148.97 GB, 151.66 GB, 154.33 GB, 157.00 GB, 159.70 GB, 161.38 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.09 GB (186932769177) - RowsProduced: 219.95M (219946790) - TotalNetworkReceiveTime: 5h39m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 594.44K (594443) - TotalThreadsTotalWallClockTime: 8h1m - TotalThreadsSysTime: 4m31s - TotalThreadsUserTime: 1h6m - TotalThreadsVoluntaryContextSwitches: 2.89M (2889162) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 8h1m - ExecTreeExecTime: 7h17m - OpenTime: 983.990ms - ExecTreeOpenTime: 918.150ms - PrepareTime: 216.760ms - ExecTreePrepareTime: 201.732us KuduTableSink:(Total: 43m45s, non-child: 43m45s, % non-child: 100.00%) - KuduApplyTimer: 36m24s - NumRowErrors: 38.59K (38590) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 83.77 K/sec - TotalNumRows: 219.95M (219946790) SORT_NODE (id=2):(Total: 7h17m, non-child: 1h38m, % non-child: 22.45%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 34m17s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 219.95M (219946790) - RowsReturnedRate: 8.37 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276680764) Buffer pool: - AllocTime: 42m56s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h39m, non-child: 51s055ms, % non-child: 0.25%) - ConvertRowBatchTime: 20s865ms - PeakMemoryUsage: 19.53 MB (20482410) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 31.07 K/sec Buffer pool: - AllocTime: 5h33m - CumulativeAllocationBytes: 287.43 GB (308621721600) - CumulativeAllocations: 15.07M (15069420) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.02 GB, 40.75 GB, 50.20 GB, 58.56 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.65 GB, 99.40 GB, 103.75 GB, 108.01 GB, 112.20 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.31 GB, 133.46 GB, 136.52 GB, 139.43 GB, 142.21 GB, 145.09 GB, 147.87 GB, 150.56 GB, 153.28 GB, 155.97 GB, 158.68 GB, 161.40 GB, 163.11 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 918.089ms - TotalBytesReceived: 163.15 GB (175182114552) - TotalGetBatchTime: 5h39m - DataArrivalTimer: 5h38m SenderSide: - DeserializeRowBatchTime: 5h52m - NumBatchesArrived: 7.53M (7534710) - NumBatchesDeferred: 334.63K (334635) - NumBatchesEnqueued: 7.53M (7534710) - NumBatchesReceived: 7.53M (7534710) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 64.554ms, non-child: 64.554ms, % non-child: 100.00%) - CodegenTime: 1.123ms - CompileTime: 4.721ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 35.467ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.720ms Instance 844a0200d7876345:20bb38b9000000fa (host=vd1332.foo.com:22000):(Total: 1s327ms, non-child: 96.447ms, % non-child: 7.27%) Fragment Instance Lifecycle Event Timeline: 1s327ms - Prepare Finished: 95.286ms (95.286ms) - Open Finished: 1s327ms (1s232ms) MemoryUsage(8m32s): 20.73 GB, 36.69 GB, 45.18 GB, 52.70 GB, 59.58 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.35 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.97 GB, 111.18 GB, 114.29 GB, 117.23 GB, 120.06 GB, 122.82 GB, 125.44 GB, 127.93 GB, 130.53 GB, 133.03 GB, 135.44 GB, 137.89 GB, 140.31 GB, 142.75 GB, 145.20 GB, 146.73 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.77 GB (157594763702) - PeakReservation: 146.73 GB (157554835456) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.41 GB (171166687857) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h59m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 1s232ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 53.991ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s232ms - ExecTreeOpenTime: 1s174ms - PrepareTime: 94.870ms - ExecTreePrepareTime: 175.355us KuduTableSink:(Total: 13.314us, non-child: 13.314us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 1s175ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.73 GB (157554843648) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 25m3s - CumulativeAllocationBytes: 146.73 GB (157554835456) - CumulativeAllocations: 75.13K (75128) - PeakReservation: 146.73 GB (157554835456) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.73 GB (157554835456) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h59m, non-child: 49s585ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s532ms - PeakMemoryUsage: 19.53 MB (20483070) - RowsReturned: 575.43M (575427636) - RowsReturnedRate: 26.65 K/sec Buffer pool: - AllocTime: 1h55m - CumulativeAllocationBytes: 261.32 GB (280589475840) - CumulativeAllocations: 13.70M (13700658) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.92 GB, 37.05 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.32 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.37 GB, 94.32 GB, 98.19 GB, 102.01 GB, 105.60 GB, 109.10 GB, 112.36 GB, 115.50 GB, 118.47 GB, 121.33 GB, 124.12 GB, 126.76 GB, 129.29 GB, 131.91 GB, 134.44 GB, 136.88 GB, 139.36 GB, 141.80 GB, 144.26 GB, 146.74 GB, 148.29 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB - FirstBatchArrivalWaitTime: 1s174ms - TotalBytesReceived: 148.33 GB (159270015915) - TotalGetBatchTime: 5h59m - DataArrivalTimer: 5h59m SenderSide: - DeserializeRowBatchTime: 2h4m - NumBatchesArrived: 6.85M (6850329) - NumBatchesDeferred: 18.65K (18652) - NumBatchesEnqueued: 6.85M (6850329) - NumBatchesReceived: 6.85M (6850329) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 55.504ms, non-child: 55.504ms, % non-child: 100.00%) - CodegenTime: 1.098ms - CompileTime: 4.518ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.216ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.256ms Instance 844a0200d7876345:20bb38b900000098 (host=va1031.foo.com:22000):(Total: 1s310ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h14m - Prepare Finished: 87.952ms (87.952ms) - Open Finished: 1s310ms (1s222ms) - First Batch Produced: 7h14m (7h14m) - First Batch Sent: 7h14m (11.032ms) MemoryUsage(8m32s): 22.79 GB, 40.34 GB, 49.69 GB, 57.96 GB, 65.52 GB, 72.20 GB, 77.96 GB, 83.40 GB, 88.58 GB, 93.67 GB, 98.37 GB, 102.68 GB, 106.88 GB, 111.03 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.09 GB, 137.97 GB, 140.71 GB, 143.57 GB, 146.32 GB, 148.97 GB, 151.67 GB, 154.33 GB, 157.01 GB, 159.70 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173326178038) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.39 GB (188323580396) - RowsProduced: 278.42M (278417032) - TotalNetworkReceiveTime: 5h51m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s222ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 56.992ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h14m - OpenTime: 1s222ms - ExecTreeOpenTime: 1s165ms - PrepareTime: 87.598ms - ExecTreePrepareTime: 171.161us KuduTableSink:(Total: 46m58s, non-child: 46m58s, % non-child: 100.00%) - KuduApplyTimer: 38m45s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 98.77 K/sec - TotalNumRows: 278.42M (278416008) SORT_NODE (id=2):(Total: 7h14m, non-child: 1h22m, % non-child: 18.96%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 30m53s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 278.42M (278417032) - RowsReturnedRate: 10.68 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276762739) Buffer pool: - AllocTime: 31m14s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h52m, non-child: 51s190ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s982ms - PeakMemoryUsage: 19.53 MB (20481886) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 29.96 K/sec Buffer pool: - AllocTime: 2h47m - CumulativeAllocationBytes: 287.43 GB (308628398080) - CumulativeAllocations: 15.07M (15069746) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.75 GB, 50.20 GB, 58.56 GB, 66.21 GB, 72.95 GB, 78.78 GB, 84.27 GB, 89.51 GB, 94.66 GB, 99.41 GB, 103.76 GB, 108.01 GB, 112.21 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.32 GB, 133.46 GB, 136.52 GB, 139.43 GB, 142.21 GB, 145.10 GB, 147.88 GB, 150.56 GB, 153.28 GB, 155.98 GB, 158.68 GB, 161.41 GB, 163.11 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 1s165ms - TotalBytesReceived: 163.15 GB (175185988014) - TotalGetBatchTime: 5h51m - DataArrivalTimer: 5h51m SenderSide: - DeserializeRowBatchTime: 2h56m - NumBatchesArrived: 7.53M (7534929) - NumBatchesDeferred: 128.21K (128212) - NumBatchesEnqueued: 7.53M (7534873) - NumBatchesReceived: 7.53M (7534873) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.169ms, non-child: 56.169ms, % non-child: 100.00%) - CodegenTime: 1.159ms - CompileTime: 5.163ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 30.803ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.636ms Instance 844a0200d7876345:20bb38b9000000e4 (host=va1026.foo.com:22000):(Total: 1s291ms, non-child: 258.894ms, % non-child: 20.05%) Fragment Instance Lifecycle Event Timeline: 1s291ms - Prepare Finished: 258.176ms (258.176ms) - Open Finished: 1s291ms (1s033ms) MemoryUsage(8m32s): 20.73 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.43 GB, 137.87 GB, 140.29 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157571695030) - PeakReservation: 146.71 GB (157531766784) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.62 GB (172464205556) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 6h16m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 1s033ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 55.992ms - TotalThreadsVoluntaryContextSwitches: 9 (9) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s033ms - ExecTreeOpenTime: 975.361ms - PrepareTime: 257.673ms - ExecTreePrepareTime: 132.498us KuduTableSink:(Total: 9.468us, non-child: 9.468us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 975.491ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157531774976) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 2m40s - CumulativeAllocationBytes: 146.71 GB (157531766784) - CumulativeAllocations: 75.12K (75117) - PeakReservation: 146.71 GB (157531766784) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157531766784) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 6h16m, non-child: 53s226ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s880ms - PeakMemoryUsage: 19.53 MB (20482957) - RowsReturned: 575.35M (575345568) - RowsReturnedRate: 25.46 K/sec Buffer pool: - AllocTime: 1h47m - CumulativeAllocationBytes: 261.28 GB (280549457920) - CumulativeAllocations: 13.70M (13698704) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.64 GB, 53.24 GB, 60.19 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.18 GB, 102.00 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 975.255ms - TotalBytesReceived: 148.31 GB (159247013047) - TotalGetBatchTime: 6h16m - DataArrivalTimer: 6h15m SenderSide: - DeserializeRowBatchTime: 1h56m - NumBatchesArrived: 6.85M (6849352) - NumBatchesDeferred: 13.11K (13110) - NumBatchesEnqueued: 6.85M (6849352) - NumBatchesReceived: 6.85M (6849352) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.748ms, non-child: 56.748ms, % non-child: 100.00%) - CodegenTime: 967.576us - CompileTime: 5.104ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.275ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.849ms Instance 844a0200d7876345:20bb38b9000000da (host=vc1542.foo.com:22000):(Total: 1s276ms, non-child: 149.606ms, % non-child: 11.72%) Fragment Instance Lifecycle Event Timeline: 1s277ms - Prepare Finished: 148.539ms (148.539ms) - Open Finished: 1s277ms (1s128ms) MemoryUsage(8m32s): 20.73 GB, 36.67 GB, 45.17 GB, 52.68 GB, 59.56 GB, 65.62 GB, 70.86 GB, 75.81 GB, 80.52 GB, 85.14 GB, 89.41 GB, 93.33 GB, 97.15 GB, 100.92 GB, 104.48 GB, 107.94 GB, 111.16 GB, 114.26 GB, 117.21 GB, 120.03 GB, 122.79 GB, 125.41 GB, 127.90 GB, 130.50 GB, 133.00 GB, 135.41 GB, 137.86 GB, 140.28 GB, 142.71 GB, 145.16 GB, 146.70 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157557014966) - PeakReservation: 146.70 GB (157517086720) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.66 GB (172511153594) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h53m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s128ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 52.992ms - TotalThreadsVoluntaryContextSwitches: 17 (17) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s128ms - ExecTreeOpenTime: 1s071ms - PrepareTime: 148.187ms - ExecTreePrepareTime: 173.652us KuduTableSink:(Total: 49.401us, non-child: 49.401us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 1s071ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.70 GB (157517094912) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 31m50s - CumulativeAllocationBytes: 146.70 GB (157517086720) - CumulativeAllocations: 75.11K (75110) - PeakReservation: 146.70 GB (157517086720) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.70 GB (157517086720) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 48s853ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s289ms - PeakMemoryUsage: 19.53 MB (20483562) - RowsReturned: 575.30M (575298780) - RowsReturnedRate: 27.10 K/sec Buffer pool: - AllocTime: 2h53m - CumulativeAllocationBytes: 261.26 GB (280526643200) - CumulativeAllocations: 13.70M (13697590) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.91 GB, 37.03 GB, 45.62 GB, 53.22 GB, 60.17 GB, 66.30 GB, 71.60 GB, 76.59 GB, 81.36 GB, 86.03 GB, 90.35 GB, 94.30 GB, 98.17 GB, 101.98 GB, 105.58 GB, 109.08 GB, 112.33 GB, 115.47 GB, 118.45 GB, 121.30 GB, 124.09 GB, 126.74 GB, 129.26 GB, 131.88 GB, 134.41 GB, 136.85 GB, 139.32 GB, 141.77 GB, 144.23 GB, 146.71 GB, 148.26 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB - FirstBatchArrivalWaitTime: 1s071ms - TotalBytesReceived: 148.30 GB (159234237617) - TotalGetBatchTime: 5h53m - DataArrivalTimer: 5h52m SenderSide: - DeserializeRowBatchTime: 3h4m - NumBatchesArrived: 6.85M (6848795) - NumBatchesDeferred: 99.67K (99667) - NumBatchesEnqueued: 6.85M (6848795) - NumBatchesReceived: 6.85M (6848795) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 55.186ms, non-child: 55.186ms, % non-child: 100.00%) - CodegenTime: 1.062ms - CompileTime: 5.605ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 26.037ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.011ms Instance 844a0200d7876345:20bb38b9000000b0 (host=va1043.foo.com:22000):(Total: 1s267ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h14m - Prepare Finished: 168.250ms (168.250ms) - Open Finished: 1s267ms (1s099ms) - First Batch Produced: 7h14m (7h14m) - First Batch Sent: 7h14m (10.215ms) MemoryUsage(8m32s): 22.79 GB, 40.34 GB, 49.69 GB, 57.95 GB, 65.52 GB, 72.19 GB, 77.95 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.36 GB, 102.67 GB, 106.87 GB, 111.02 GB, 114.94 GB, 118.74 GB, 122.28 GB, 125.70 GB, 128.93 GB, 132.05 GB, 135.08 GB, 137.96 GB, 140.70 GB, 143.56 GB, 146.31 GB, 148.96 GB, 151.66 GB, 154.32 GB, 157.00 GB, 159.69 GB, 161.38 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.36 GB (188286936910) - RowsProduced: 256.15M (256150238) - TotalNetworkReceiveTime: 5h50m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s099ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 52.992ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h14m - OpenTime: 1s099ms - ExecTreeOpenTime: 1s044ms - PrepareTime: 167.969ms - ExecTreePrepareTime: 161.740us KuduTableSink:(Total: 46m11s, non-child: 46m11s, % non-child: 100.00%) - KuduApplyTimer: 38m6s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 92.41 K/sec - TotalNumRows: 256.15M (256149214) SORT_NODE (id=2):(Total: 7h14m, non-child: 1h24m, % non-child: 19.43%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 31m20s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 256.15M (256150238) - RowsReturnedRate: 9.81 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276218217) Buffer pool: - AllocTime: 31m57s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h50m, non-child: 47s431ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s952ms - PeakMemoryUsage: 19.53 MB (20481613) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.10 K/sec Buffer pool: - AllocTime: 3h56m - CumulativeAllocationBytes: 287.42 GB (308613488640) - CumulativeAllocations: 15.07M (15069018) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.75 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.27 GB, 89.50 GB, 94.65 GB, 99.40 GB, 103.75 GB, 108.00 GB, 112.20 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.03 GB, 130.31 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.09 GB, 147.87 GB, 150.55 GB, 153.27 GB, 155.97 GB, 158.67 GB, 161.40 GB, 163.10 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 1s044ms - TotalBytesReceived: 163.15 GB (175176984236) - TotalGetBatchTime: 5h49m - DataArrivalTimer: 5h49m SenderSide: - DeserializeRowBatchTime: 4h11m - NumBatchesArrived: 7.53M (7534509) - NumBatchesDeferred: 153.22K (153215) - NumBatchesEnqueued: 7.53M (7534509) - NumBatchesReceived: 7.53M (7534509) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 53.606ms, non-child: 53.606ms, % non-child: 100.00%) - CodegenTime: 975.524us - CompileTime: 4.679ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.277ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 20.102ms Instance 844a0200d7876345:20bb38b900000096 (host=va1027.foo.com:22000):(Total: 1s266ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h17m - Prepare Finished: 161.550ms (161.550ms) - Open Finished: 1s266ms (1s104ms) - First Batch Produced: 7h17m (7h17m) - First Batch Sent: 7h17m (55.289ms) MemoryUsage(8m32s): 22.80 GB, 40.35 GB, 49.69 GB, 57.95 GB, 65.52 GB, 72.19 GB, 77.96 GB, 83.39 GB, 88.58 GB, 93.66 GB, 98.36 GB, 102.67 GB, 106.88 GB, 111.03 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.70 GB, 128.94 GB, 132.05 GB, 135.08 GB, 137.96 GB, 140.71 GB, 143.56 GB, 146.31 GB, 148.97 GB, 151.66 GB, 154.33 GB, 157.00 GB, 159.70 GB, 161.38 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.34 GB (188273043643) - RowsProduced: 248.70M (248699872) - TotalNetworkReceiveTime: 5h50m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s104ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 52.992ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h17m - OpenTime: 1s104ms - ExecTreeOpenTime: 1s048ms - PrepareTime: 161.033ms - ExecTreePrepareTime: 181.498us KuduTableSink:(Total: 43m38s, non-child: 43m38s, % non-child: 100.00%) - KuduApplyTimer: 36m19s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 94.99 K/sec - TotalNumRows: 248.70M (248698848) SORT_NODE (id=2):(Total: 7h17m, non-child: 1h26m, % non-child: 19.78%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 33m55s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 248.70M (248699872) - RowsReturnedRate: 9.47 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276046823) Buffer pool: - AllocTime: 32m46s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h50m, non-child: 46s743ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s477ms - PeakMemoryUsage: 19.53 MB (20482074) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.05 K/sec Buffer pool: - AllocTime: 3h54m - CumulativeAllocationBytes: 287.42 GB (308617011200) - CumulativeAllocations: 15.07M (15069190) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.01 GB, 40.75 GB, 50.19 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.65 GB, 99.39 GB, 103.75 GB, 108.00 GB, 112.20 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.04 GB, 130.31 GB, 133.46 GB, 136.52 GB, 139.43 GB, 142.20 GB, 145.09 GB, 147.87 GB, 150.55 GB, 153.28 GB, 155.97 GB, 158.68 GB, 161.40 GB, 163.10 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 1s048ms - TotalBytesReceived: 163.15 GB (175178811634) - TotalGetBatchTime: 5h50m - DataArrivalTimer: 5h50m SenderSide: - DeserializeRowBatchTime: 4h8m - NumBatchesArrived: 7.53M (7534595) - NumBatchesDeferred: 151.85K (151855) - NumBatchesEnqueued: 7.53M (7534595) - NumBatchesReceived: 7.53M (7534595) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 55.070ms, non-child: 55.070ms, % non-child: 100.00%) - CodegenTime: 917.487us - CompileTime: 5.209ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 29.513ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.700ms Instance 844a0200d7876345:20bb38b9000000b2 (host=va1039.foo.com:22000):(Total: 1s260ms, non-child: 176.721ms, % non-child: 14.02%) Fragment Instance Lifecycle Event Timeline: 1s261ms - Prepare Finished: 175.889ms (175.889ms) - Open Finished: 1s261ms (1s085ms) MemoryUsage(8m32s): 22.80 GB, 40.34 GB, 49.69 GB, 57.95 GB, 65.52 GB, 72.19 GB, 77.95 GB, 83.38 GB, 88.57 GB, 93.66 GB, 98.35 GB, 102.66 GB, 106.87 GB, 111.02 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.70 GB, 128.93 GB, 132.05 GB, 135.08 GB, 137.96 GB, 140.70 GB, 143.56 GB, 146.31 GB, 148.96 GB, 151.66 GB, 154.32 GB, 157.00 GB, 159.69 GB, 161.38 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325500854) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.34 GB (188271562026) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h49m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1s085ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 62.990ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s085ms - ExecTreeOpenTime: 1s020ms - PrepareTime: 175.456ms - ExecTreePrepareTime: 152.204us KuduTableSink:(Total: 11.831us, non-child: 11.831us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 1s020ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173285580800) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 34m17s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h49m, non-child: 50s953ms, % non-child: 0.24%) - ConvertRowBatchTime: 21s087ms - PeakMemoryUsage: 19.53 MB (20481634) - RowsReturned: 632.88M (632883720) - RowsReturnedRate: 30.18 K/sec Buffer pool: - AllocTime: 2h41m - CumulativeAllocationBytes: 287.41 GB (308606156800) - CumulativeAllocations: 15.07M (15068660) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.01 GB, 40.74 GB, 50.19 GB, 58.55 GB, 66.19 GB, 72.93 GB, 78.76 GB, 84.25 GB, 89.49 GB, 94.64 GB, 99.39 GB, 103.74 GB, 108.00 GB, 112.19 GB, 116.14 GB, 120.00 GB, 123.57 GB, 127.03 GB, 130.30 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.08 GB, 147.87 GB, 150.55 GB, 153.27 GB, 155.97 GB, 158.67 GB, 161.40 GB, 163.10 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 1s020ms - TotalBytesReceived: 163.14 GB (175173323606) - TotalGetBatchTime: 5h49m - DataArrivalTimer: 5h48m SenderSide: - DeserializeRowBatchTime: 2h51m - NumBatchesArrived: 7.53M (7534330) - NumBatchesDeferred: 130.72K (130721) - NumBatchesEnqueued: 7.53M (7534330) - NumBatchesReceived: 7.53M (7534330) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 63.010ms, non-child: 63.010ms, % non-child: 100.00%) - CodegenTime: 1.371ms - CompileTime: 6.253ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 33.366ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.715ms Instance 844a0200d7876345:20bb38b9000000c8 (host=vc1523.foo.com:22000):(Total: 1s260ms, non-child: 220.418ms, % non-child: 17.49%) Fragment Instance Lifecycle Event Timeline: 1s260ms - Prepare Finished: 219.654ms (219.654ms) - Open Finished: 1s260ms (1s041ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.19 GB, 52.70 GB, 59.58 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.49 GB, 107.96 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157571695030) - PeakReservation: 146.71 GB (157531766784) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.58 GB (172426092605) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h57m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 10 (10) - TotalThreadsTotalWallClockTime: 1s041ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 58.991ms - TotalThreadsVoluntaryContextSwitches: 6 (6) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s041ms - ExecTreeOpenTime: 980.802ms - PrepareTime: 219.235ms - ExecTreePrepareTime: 155.722us KuduTableSink:(Total: 12.733us, non-child: 12.733us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 980.955ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157531774976) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 27m36s - CumulativeAllocationBytes: 146.71 GB (157531766784) - CumulativeAllocations: 75.12K (75117) - PeakReservation: 146.71 GB (157531766784) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157531766784) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h57m, non-child: 48s595ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s507ms - PeakMemoryUsage: 19.53 MB (20483634) - RowsReturned: 575.35M (575353464) - RowsReturnedRate: 26.80 K/sec Buffer pool: - AllocTime: 2h3m - CumulativeAllocationBytes: 261.29 GB (280553308160) - CumulativeAllocations: 13.70M (13698892) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.19 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.59 GB, 109.09 GB, 112.35 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 980.732ms - TotalBytesReceived: 148.31 GB (159249736575) - TotalGetBatchTime: 5h57m - DataArrivalTimer: 5h57m SenderSide: - DeserializeRowBatchTime: 2h11m - NumBatchesArrived: 6.85M (6849446) - NumBatchesDeferred: 43.77K (43772) - NumBatchesEnqueued: 6.85M (6849446) - NumBatchesReceived: 6.85M (6849446) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 59.003ms, non-child: 59.003ms, % non-child: 100.00%) - CodegenTime: 1.138ms - CompileTime: 4.762ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.202ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 25.450ms Instance 844a0200d7876345:20bb38b9000000ae (host=vd1123.foo.com:22000):(Total: 1s258ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h18m - Prepare Finished: 173.589ms (173.589ms) - Open Finished: 1s258ms (1s085ms) - First Batch Produced: 7h18m (7h18m) - First Batch Sent: 7h18m (10.557ms) MemoryUsage(8m32s): 22.80 GB, 40.35 GB, 49.70 GB, 57.96 GB, 65.53 GB, 72.20 GB, 77.96 GB, 83.40 GB, 88.58 GB, 93.67 GB, 98.36 GB, 102.67 GB, 106.87 GB, 111.02 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.70 GB, 128.94 GB, 132.05 GB, 135.08 GB, 137.96 GB, 140.71 GB, 143.56 GB, 146.31 GB, 148.97 GB, 151.66 GB, 154.33 GB, 157.00 GB, 159.70 GB, 161.38 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.28 GB (188204366857) - RowsProduced: 223.15M (223147270) - TotalNetworkReceiveTime: 5h46m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1s085ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 59.991ms - TotalThreadsVoluntaryContextSwitches: 7 (7) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h18m - OpenTime: 1s085ms - ExecTreeOpenTime: 1s021ms - PrepareTime: 172.944ms - ExecTreePrepareTime: 162.140us KuduTableSink:(Total: 42m35s, non-child: 42m35s, % non-child: 100.00%) - KuduApplyTimer: 34m6s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 87.31 K/sec - TotalNumRows: 223.15M (223146246) SORT_NODE (id=2):(Total: 7h18m, non-child: 1h32m, % non-child: 20.98%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 35m - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 223.15M (223147270) - RowsReturnedRate: 8.48 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276571948) Buffer pool: - AllocTime: 36m55s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h46m, non-child: 50s880ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s725ms - PeakMemoryUsage: 19.53 MB (20482295) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.44 K/sec Buffer pool: - AllocTime: 3h22m - CumulativeAllocationBytes: 287.42 GB (308618117120) - CumulativeAllocations: 15.07M (15069244) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.02 GB, 40.75 GB, 50.20 GB, 58.56 GB, 66.20 GB, 72.95 GB, 78.77 GB, 84.27 GB, 89.51 GB, 94.65 GB, 99.39 GB, 103.75 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.03 GB, 130.30 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.09 GB, 147.87 GB, 150.55 GB, 153.28 GB, 155.97 GB, 158.68 GB, 161.40 GB, 163.10 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 1s021ms - TotalBytesReceived: 163.15 GB (175179963120) - TotalGetBatchTime: 5h46m - DataArrivalTimer: 5h45m SenderSide: - DeserializeRowBatchTime: 3h33m - NumBatchesArrived: 7.53M (7534622) - NumBatchesDeferred: 177.35K (177353) - NumBatchesEnqueued: 7.53M (7534622) - NumBatchesReceived: 7.53M (7534622) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.141ms, non-child: 62.141ms, % non-child: 100.00%) - CodegenTime: 1.169ms - CompileTime: 5.929ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 34.652ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 21.036ms Instance 844a0200d7876345:20bb38b9000000c0 (host=vd1120.foo.com:22000):(Total: 1s258ms, non-child: 189.204ms, % non-child: 15.04%) Fragment Instance Lifecycle Event Timeline: 1s258ms - Prepare Finished: 188.355ms (188.355ms) - Open Finished: 1s258ms (1s070ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.33 GB, 97.16 GB, 100.93 GB, 104.48 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.04 GB, 122.80 GB, 125.41 GB, 127.91 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.72 GB, 145.17 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157571695030) - PeakReservation: 146.71 GB (157531766784) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.71 GB (172557075337) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h50m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1s069ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 64.989ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s069ms - ExecTreeOpenTime: 1s004ms - PrepareTime: 187.941ms - ExecTreePrepareTime: 165.267us KuduTableSink:(Total: 12.719us, non-child: 12.719us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 1s004ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157531774976) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 33m37s - CumulativeAllocationBytes: 146.71 GB (157531766784) - CumulativeAllocations: 75.12K (75117) - PeakReservation: 146.71 GB (157531766784) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157531766784) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h50m, non-child: 47s152ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s702ms - PeakMemoryUsage: 19.53 MB (20483765) - RowsReturned: 575.35M (575345400) - RowsReturnedRate: 27.34 K/sec Buffer pool: - AllocTime: 3h55m - CumulativeAllocationBytes: 261.28 GB (280549376000) - CumulativeAllocations: 13.70M (13698700) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.92 GB, 37.04 GB, 45.63 GB, 53.22 GB, 60.18 GB, 66.31 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.35 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.31 GB, 124.10 GB, 126.74 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 1s004ms - TotalBytesReceived: 148.31 GB (159247162854) - TotalGetBatchTime: 5h50m - DataArrivalTimer: 5h49m SenderSide: - DeserializeRowBatchTime: 4h7m - NumBatchesArrived: 6.85M (6849350) - NumBatchesDeferred: 102.36K (102363) - NumBatchesEnqueued: 6.85M (6849350) - NumBatchesReceived: 6.85M (6849350) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 64.281ms, non-child: 64.281ms, % non-child: 100.00%) - CodegenTime: 1.078ms - CompileTime: 5.404ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.938ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 25.267ms Instance 844a0200d7876345:20bb38b9000000fd (host=vc1539.foo.com:22000):(Total: 1s256ms, non-child: 207.024ms, % non-child: 16.48%) Fragment Instance Lifecycle Event Timeline: 1s256ms - Prepare Finished: 206.294ms (206.294ms) - Open Finished: 1s256ms (1s050ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.17 GB, 52.68 GB, 59.56 GB, 65.62 GB, 70.86 GB, 75.81 GB, 80.52 GB, 85.14 GB, 89.41 GB, 93.33 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157565403574) - PeakReservation: 146.71 GB (157525475328) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.68 GB (172533560950) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h54m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1s050ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 62.991ms - TotalThreadsVoluntaryContextSwitches: 18 (18) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s050ms - ExecTreeOpenTime: 986.615ms - PrepareTime: 205.862ms - ExecTreePrepareTime: 150.477us KuduTableSink:(Total: 12.843us, non-child: 12.843us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 986.762ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157525483520) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 29m29s - CumulativeAllocationBytes: 146.71 GB (157525475328) - CumulativeAllocations: 75.11K (75114) - PeakReservation: 146.71 GB (157525475328) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157525475328) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h55m, non-child: 48s138ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s088ms - PeakMemoryUsage: 19.53 MB (20482789) - RowsReturned: 575.33M (575332380) - RowsReturnedRate: 26.99 K/sec Buffer pool: - AllocTime: 2h17m - CumulativeAllocationBytes: 261.28 GB (280543027200) - CumulativeAllocations: 13.70M (13698390) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.92 GB, 37.04 GB, 45.62 GB, 53.22 GB, 60.17 GB, 66.30 GB, 71.60 GB, 76.59 GB, 81.36 GB, 86.03 GB, 90.35 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.90 GB, 134.42 GB, 136.86 GB, 139.34 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.27 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 986.536ms - TotalBytesReceived: 148.31 GB (159243347519) - TotalGetBatchTime: 5h54m - DataArrivalTimer: 5h54m SenderSide: - DeserializeRowBatchTime: 2h29m - NumBatchesArrived: 6.85M (6849195) - NumBatchesDeferred: 52.37K (52372) - NumBatchesEnqueued: 6.85M (6849195) - NumBatchesReceived: 6.85M (6849195) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.296ms, non-child: 62.296ms, % non-child: 100.00%) - CodegenTime: 1.050ms - CompileTime: 6.016ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.564ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.016ms Instance 844a0200d7876345:20bb38b9000000e2 (host=va1045.foo.com:22000):(Total: 1s249ms, non-child: 190.003ms, % non-child: 15.21%) Fragment Instance Lifecycle Event Timeline: 1s249ms - Prepare Finished: 189.154ms (189.154ms) - Open Finished: 1s249ms (1s060ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.17 GB, 52.69 GB, 59.56 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.91 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.72 GB, 145.17 GB, 146.71 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157565403574) - PeakReservation: 146.71 GB (157525475328) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.66 GB (172508912446) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h50m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1s060ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 64.989ms - TotalThreadsVoluntaryContextSwitches: 7 (7) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s060ms - ExecTreeOpenTime: 994.998ms - PrepareTime: 188.757ms - ExecTreePrepareTime: 156.672us KuduTableSink:(Total: 12.022us, non-child: 12.022us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 995.151ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157525483520) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 33m - CumulativeAllocationBytes: 146.71 GB (157525475328) - CumulativeAllocations: 75.11K (75114) - PeakReservation: 146.71 GB (157525475328) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157525475328) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h50m, non-child: 47s034ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s671ms - PeakMemoryUsage: 19.53 MB (20483667) - RowsReturned: 575.33M (575333136) - RowsReturnedRate: 27.34 K/sec Buffer pool: - AllocTime: 2h47m - CumulativeAllocationBytes: 261.28 GB (280543395840) - CumulativeAllocations: 13.70M (13698408) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.91 GB, 37.04 GB, 45.62 GB, 53.22 GB, 60.17 GB, 66.30 GB, 71.60 GB, 76.59 GB, 81.36 GB, 86.04 GB, 90.35 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.74 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.27 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 994.914ms - TotalBytesReceived: 148.31 GB (159243456650) - TotalGetBatchTime: 5h50m - DataArrivalTimer: 5h49m SenderSide: - DeserializeRowBatchTime: 2h58m - NumBatchesArrived: 6.85M (6849204) - NumBatchesDeferred: 65.03K (65033) - NumBatchesEnqueued: 6.85M (6849204) - NumBatchesReceived: 6.85M (6849204) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 64.384ms, non-child: 64.384ms, % non-child: 100.00%) - CodegenTime: 1.156ms - CompileTime: 8.263ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.089ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.481ms Instance 844a0200d7876345:20bb38b9000000f6 (host=vd1119.foo.com:22000):(Total: 1s248ms, non-child: 223.381ms, % non-child: 17.89%) Fragment Instance Lifecycle Event Timeline: 1s249ms - Prepare Finished: 222.795ms (222.795ms) - Open Finished: 1s249ms (1s026ms) MemoryUsage(8m32s): 20.74 GB, 36.70 GB, 45.19 GB, 52.71 GB, 59.58 GB, 65.65 GB, 70.88 GB, 75.82 GB, 80.54 GB, 85.16 GB, 89.43 GB, 93.35 GB, 97.18 GB, 100.95 GB, 104.50 GB, 107.97 GB, 111.18 GB, 114.29 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.43 GB, 127.93 GB, 130.52 GB, 133.02 GB, 135.44 GB, 137.89 GB, 140.31 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.77 GB (157588472246) - PeakReservation: 146.73 GB (157548544000) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.72 GB (172571892017) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h58m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s026ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 46.993ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s026ms - ExecTreeOpenTime: 977.555ms - PrepareTime: 222.185ms - ExecTreePrepareTime: 171.480us KuduTableSink:(Total: 11.457us, non-child: 11.457us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 977.724ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.73 GB (157548552192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 26m23s - CumulativeAllocationBytes: 146.73 GB (157548544000) - CumulativeAllocations: 75.12K (75125) - PeakReservation: 146.73 GB (157548544000) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.73 GB (157548544000) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h59m, non-child: 49s468ms, % non-child: 0.23%) - ConvertRowBatchTime: 21s076ms - PeakMemoryUsage: 19.53 MB (20482626) - RowsReturned: 575.41M (575409828) - RowsReturnedRate: 26.70 K/sec Buffer pool: - AllocTime: 2h50m - CumulativeAllocationBytes: 261.31 GB (280580792320) - CumulativeAllocations: 13.70M (13700234) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.64 GB, 53.24 GB, 60.19 GB, 66.32 GB, 71.62 GB, 76.61 GB, 81.38 GB, 86.05 GB, 90.37 GB, 94.33 GB, 98.20 GB, 102.01 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.76 GB, 129.28 GB, 131.91 GB, 134.44 GB, 136.88 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.74 GB, 148.29 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB - FirstBatchArrivalWaitTime: 977.476ms - TotalBytesReceived: 148.33 GB (159264972895) - TotalGetBatchTime: 5h58m - DataArrivalTimer: 5h58m SenderSide: - DeserializeRowBatchTime: 2h58m - NumBatchesArrived: 6.85M (6850117) - NumBatchesDeferred: 41.39K (41392) - NumBatchesEnqueued: 6.85M (6850117) - NumBatchesReceived: 6.85M (6850117) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 47.675ms, non-child: 47.675ms, % non-child: 100.00%) - CodegenTime: 1.112ms - CompileTime: 4.590ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 26.240ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 16.385ms Instance 844a0200d7876345:20bb38b9000000b1 (host=vd1124.foo.com:22000):(Total: 1s247ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 4h48m - Prepare Finished: 166.329ms (166.329ms) - Open Finished: 1s247ms (1s081ms) - First Batch Produced: 4h48m (4h48m) - First Batch Sent: 4h48m (11.353ms) MemoryUsage(8m32s): 22.80 GB, 40.35 GB, 49.70 GB, 57.96 GB, 65.53 GB, 72.20 GB, 77.96 GB, 83.40 GB, 88.58 GB, 93.67 GB, 98.37 GB, 102.68 GB, 106.88 GB, 111.03 GB, 114.95 GB, 118.76 GB, 122.30 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.10 GB, 137.98 GB, 140.72 GB, 143.58 GB, 146.33 GB, 148.98 GB, 151.68 GB, 154.34 GB, 157.02 GB, 159.71 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.37 GB (188307257381) - RowsProduced: 632.89M (632891204) - TotalNetworkReceiveTime: 3h31m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1s081ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 65.990ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 4h48m - OpenTime: 1s081ms - ExecTreeOpenTime: 1s013ms - PrepareTime: 165.905ms - ExecTreePrepareTime: 147.353us KuduTableSink:(Total: 1h48m, non-child: 1h48m, % non-child: 100.00%) - KuduApplyTimer: 1h29m - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 97.48 K/sec - TotalNumRows: 632.89M (632891204) SORT_NODE (id=2):(Total: 4h48m, non-child: 1h16m, % non-child: 26.65%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 31m14s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 632.89M (632891204) - RowsReturnedRate: 36.54 K/sec - RunsCreated: 2 (2) - SortDataSize: 161.38 GB (173276613681) Buffer pool: - AllocTime: 33m19s - CumulativeAllocationBytes: 161.41 GB (173308641280) - CumulativeAllocations: 82.64K (82640) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 3h31m, non-child: 51s159ms, % non-child: 0.40%) - ConvertRowBatchTime: 21s127ms - PeakMemoryUsage: 19.53 MB (20482164) - RowsReturned: 632.96M (632959992) - RowsReturnedRate: 49.82 K/sec Buffer pool: - AllocTime: 2h46m - CumulativeAllocationBytes: 287.45 GB (308643348480) - CumulativeAllocations: 15.07M (15070476) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.01 GB, 40.75 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.95 GB, 78.77 GB, 84.27 GB, 89.51 GB, 94.65 GB, 99.40 GB, 103.76 GB, 108.01 GB, 112.20 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.32 GB, 133.47 GB, 136.53 GB, 139.44 GB, 142.22 GB, 145.10 GB, 147.89 GB, 150.57 GB, 153.29 GB, 155.98 GB, 158.69 GB, 161.41 GB, 163.12 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB - FirstBatchArrivalWaitTime: 1s013ms - TotalBytesReceived: 163.16 GB (175194321831) - TotalGetBatchTime: 3h31m - DataArrivalTimer: 3h30m SenderSide: - DeserializeRowBatchTime: 2h56m - NumBatchesArrived: 7.54M (7535238) - NumBatchesDeferred: 117.64K (117641) - NumBatchesEnqueued: 7.54M (7535238) - NumBatchesReceived: 7.54M (7535238) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 66.496ms, non-child: 66.496ms, % non-child: 100.00%) - CodegenTime: 1.395ms - CompileTime: 5.802ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 34.157ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 25.899ms Instance 844a0200d7876345:20bb38b9000000a0 (host=va1046.foo.com:22000):(Total: 1s246ms, non-child: 193.250ms, % non-child: 15.50%) Fragment Instance Lifecycle Event Timeline: 1s247ms - Prepare Finished: 192.831ms (192.831ms) - Open Finished: 1s247ms (1s054ms) MemoryUsage(8m32s): 22.80 GB, 40.34 GB, 49.69 GB, 57.95 GB, 65.52 GB, 72.19 GB, 77.96 GB, 83.39 GB, 88.58 GB, 93.66 GB, 98.36 GB, 102.66 GB, 106.87 GB, 111.02 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.70 GB, 128.94 GB, 132.05 GB, 135.08 GB, 137.95 GB, 140.70 GB, 143.56 GB, 146.31 GB, 148.96 GB, 151.65 GB, 154.32 GB, 156.99 GB, 159.69 GB, 161.37 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173321306550) - PeakReservation: 161.38 GB (173281378304) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.17 GB (188092279510) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h46m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1s054ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 62.990ms - TotalThreadsVoluntaryContextSwitches: 5 (5) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s054ms - ExecTreeOpenTime: 989.916ms - PrepareTime: 192.210ms - ExecTreePrepareTime: 170.452us KuduTableSink:(Total: 12.148us, non-child: 12.148us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 990.083ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173281386496) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 36m22s - CumulativeAllocationBytes: 161.38 GB (173281378304) - CumulativeAllocations: 82.63K (82627) - PeakReservation: 161.38 GB (173281378304) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173281378304) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h46m, non-child: 50s084ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s627ms - PeakMemoryUsage: 19.53 MB (20482422) - RowsReturned: 632.87M (632873472) - RowsReturnedRate: 30.43 K/sec Buffer pool: - AllocTime: 4h3m - CumulativeAllocationBytes: 287.41 GB (308601159680) - CumulativeAllocations: 15.07M (15068416) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.01 GB, 40.74 GB, 50.19 GB, 58.55 GB, 66.19 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.64 GB, 99.39 GB, 103.75 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.03 GB, 130.30 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.08 GB, 147.87 GB, 150.55 GB, 153.27 GB, 155.96 GB, 158.67 GB, 161.39 GB, 163.09 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 989.853ms - TotalBytesReceived: 163.14 GB (175170238352) - TotalGetBatchTime: 5h46m - DataArrivalTimer: 5h45m SenderSide: - DeserializeRowBatchTime: 4h15m - NumBatchesArrived: 7.53M (7534208) - NumBatchesDeferred: 164.42K (164420) - NumBatchesEnqueued: 7.53M (7534208) - NumBatchesReceived: 7.53M (7534208) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 63.269ms, non-child: 63.269ms, % non-child: 100.00%) - CodegenTime: 1.133ms - CompileTime: 5.692ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 33.207ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.819ms Instance 844a0200d7876345:20bb38b9000000cb (host=vc1536.foo.com:22000):(Total: 1s243ms, non-child: 190.290ms, % non-child: 15.30%) Fragment Instance Lifecycle Event Timeline: 1s244ms - Prepare Finished: 189.448ms (189.448ms) - Open Finished: 1s244ms (1s054ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.17 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.43 GB, 93.34 GB, 97.17 GB, 100.93 GB, 104.49 GB, 107.96 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157577986486) - PeakReservation: 146.72 GB (157538058240) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.71 GB (172566043126) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h53m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s054ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 47.993ms - TotalThreadsVoluntaryContextSwitches: 14 (14) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s054ms - ExecTreeOpenTime: 1s003ms - PrepareTime: 189.198ms - ExecTreePrepareTime: 175.558us KuduTableSink:(Total: 12.853us, non-child: 12.853us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 1s003ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157538066432) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 31m7s - CumulativeAllocationBytes: 146.72 GB (157538058240) - CumulativeAllocations: 75.12K (75120) - PeakReservation: 146.72 GB (157538058240) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157538058240) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 48s637ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s310ms - PeakMemoryUsage: 19.53 MB (20482825) - RowsReturned: 575.37M (575374128) - RowsReturnedRate: 27.10 K/sec Buffer pool: - AllocTime: 3h31m - CumulativeAllocationBytes: 261.30 GB (280563384320) - CumulativeAllocations: 13.70M (13699384) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.04 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.37 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 1s003ms - TotalBytesReceived: 148.32 GB (159255510782) - TotalGetBatchTime: 5h53m - DataArrivalTimer: 5h53m SenderSide: - DeserializeRowBatchTime: 3h41m - NumBatchesArrived: 6.85M (6849692) - NumBatchesDeferred: 49.45K (49445) - NumBatchesEnqueued: 6.85M (6849692) - NumBatchesReceived: 6.85M (6849692) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 50.072ms, non-child: 50.072ms, % non-child: 100.00%) - CodegenTime: 961.919us - CompileTime: 5.293ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 26.411ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 17.892ms Instance 844a0200d7876345:20bb38b9000000d0 (host=vd1342.foo.com:22000):(Total: 1s242ms, non-child: 215.750ms, % non-child: 17.37%) Fragment Instance Lifecycle Event Timeline: 1s242ms - Prepare Finished: 214.615ms (214.615ms) - Open Finished: 1s242ms (1s027ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.96 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157584277942) - PeakReservation: 146.72 GB (157544349696) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.47 GB (171226604972) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h54m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 1s027ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 56.992ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s027ms - ExecTreeOpenTime: 969.796ms - PrepareTime: 214.228ms - ExecTreePrepareTime: 215.290us KuduTableSink:(Total: 15.465us, non-child: 15.465us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 970.008ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157544357888) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 29m43s - CumulativeAllocationBytes: 146.72 GB (157544349696) - CumulativeAllocations: 75.12K (75123) - PeakReservation: 146.72 GB (157544349696) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157544349696) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h55m, non-child: 48s928ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s280ms - PeakMemoryUsage: 19.54 MB (20483934) - RowsReturned: 575.39M (575393532) - RowsReturnedRate: 27.00 K/sec Buffer pool: - AllocTime: 3h19m - CumulativeAllocationBytes: 261.30 GB (280572846080) - CumulativeAllocations: 13.70M (13699846) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.19 GB, 66.32 GB, 71.62 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.59 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.76 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 969.695ms - TotalBytesReceived: 148.32 GB (159260853106) - TotalGetBatchTime: 5h54m - DataArrivalTimer: 5h54m SenderSide: - DeserializeRowBatchTime: 3h27m - NumBatchesArrived: 6.85M (6849923) - NumBatchesDeferred: 52.03K (52029) - NumBatchesEnqueued: 6.85M (6849923) - NumBatchesReceived: 6.85M (6849923) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.393ms, non-child: 56.393ms, % non-child: 100.00%) - CodegenTime: 1.112ms - CompileTime: 4.893ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.550ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.345ms Instance 844a0200d7876345:20bb38b9000000e1 (host=vc1517.foo.com:22000):(Total: 1s240ms, non-child: 219.788ms, % non-child: 17.72%) Fragment Instance Lifecycle Event Timeline: 1s240ms - Prepare Finished: 219.368ms (219.368ms) - Open Finished: 1s240ms (1s021ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.18 GB, 52.70 GB, 59.58 GB, 65.64 GB, 70.88 GB, 75.83 GB, 80.54 GB, 85.16 GB, 89.44 GB, 93.35 GB, 97.18 GB, 100.95 GB, 104.50 GB, 107.97 GB, 111.18 GB, 114.29 GB, 117.23 GB, 120.06 GB, 122.82 GB, 125.43 GB, 127.93 GB, 130.52 GB, 133.03 GB, 135.44 GB, 137.89 GB, 140.31 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157586375094) - PeakReservation: 146.73 GB (157546446848) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.68 GB (172525689887) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h48m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 1s021ms - TotalThreadsSysTime: 999.000us - TotalThreadsUserTime: 54.991ms - TotalThreadsVoluntaryContextSwitches: 13 (13) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s021ms - ExecTreeOpenTime: 964.679ms - PrepareTime: 218.832ms - ExecTreePrepareTime: 147.821us KuduTableSink:(Total: 11.984us, non-child: 11.984us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 964.824ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.73 GB (157546455040) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 34m26s - CumulativeAllocationBytes: 146.73 GB (157546446848) - CumulativeAllocations: 75.12K (75124) - PeakReservation: 146.73 GB (157546446848) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.73 GB (157546446848) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h48m, non-child: 48s035ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s942ms - PeakMemoryUsage: 19.53 MB (20483636) - RowsReturned: 575.40M (575399076) - RowsReturnedRate: 27.49 K/sec Buffer pool: - AllocTime: 3h43m - CumulativeAllocationBytes: 261.31 GB (280575549440) - CumulativeAllocations: 13.70M (13699978) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.64 GB, 53.24 GB, 60.19 GB, 66.32 GB, 71.62 GB, 76.62 GB, 81.38 GB, 86.06 GB, 90.38 GB, 94.33 GB, 98.20 GB, 102.01 GB, 105.61 GB, 109.11 GB, 112.36 GB, 115.50 GB, 118.48 GB, 121.34 GB, 124.12 GB, 126.76 GB, 129.29 GB, 131.91 GB, 134.44 GB, 136.88 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.74 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 964.595ms - TotalBytesReceived: 148.32 GB (159261727918) - TotalGetBatchTime: 5h48m - DataArrivalTimer: 5h48m SenderSide: - DeserializeRowBatchTime: 4h - NumBatchesArrived: 6.85M (6849989) - NumBatchesDeferred: 110.91K (110913) - NumBatchesEnqueued: 6.85M (6849989) - NumBatchesReceived: 6.85M (6849989) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 55.810ms, non-child: 55.810ms, % non-child: 100.00%) - CodegenTime: 1.049ms - CompileTime: 5.116ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 29.886ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 20.320ms Instance 844a0200d7876345:20bb38b9000000b7 (host=va1042.foo.com:22000):(Total: 1s240ms, non-child: 207.588ms, % non-child: 16.74%) Fragment Instance Lifecycle Event Timeline: 1s241ms - Prepare Finished: 207.026ms (207.026ms) - Open Finished: 1s241ms (1s034ms) MemoryUsage(8m32s): 20.71 GB, 36.67 GB, 45.16 GB, 52.68 GB, 59.55 GB, 65.61 GB, 70.85 GB, 75.79 GB, 80.51 GB, 85.13 GB, 89.40 GB, 93.32 GB, 97.14 GB, 100.91 GB, 104.47 GB, 107.93 GB, 111.15 GB, 114.25 GB, 117.20 GB, 120.03 GB, 122.78 GB, 125.40 GB, 127.89 GB, 130.49 GB, 132.99 GB, 135.40 GB, 137.85 GB, 140.27 GB, 142.71 GB, 145.16 GB, 146.69 GB, 146.71 GB, 146.71 GB, 146.71 GB, 146.71 GB, 146.71 GB, 146.71 GB, 146.71 GB, 146.71 GB, 146.71 GB, 146.71 GB, 146.71 GB, 146.71 GB, 146.71 GB, 146.71 GB, 146.71 GB, 146.71 GB, 146.72 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.73 GB (157548626358) - PeakReservation: 146.69 GB (157508698112) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.69 GB (172544268435) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h50m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s033ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 58.991ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s033ms - ExecTreeOpenTime: 974.276ms - PrepareTime: 206.317ms - ExecTreePrepareTime: 164.850us KuduTableSink:(Total: 13.852us, non-child: 13.852us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 974.438ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.69 GB (157508706304) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 33m23s - CumulativeAllocationBytes: 146.69 GB (157508698112) - CumulativeAllocations: 75.11K (75106) - PeakReservation: 146.69 GB (157508698112) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.69 GB (157508698112) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h50m, non-child: 47s816ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s740ms - PeakMemoryUsage: 19.54 MB (20484209) - RowsReturned: 575.27M (575266020) - RowsReturnedRate: 27.33 K/sec Buffer pool: - AllocTime: 3h51m - CumulativeAllocationBytes: 261.25 GB (280510668800) - CumulativeAllocations: 13.70M (13696810) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.62 GB, 53.22 GB, 60.17 GB, 66.29 GB, 71.59 GB, 76.59 GB, 81.35 GB, 86.02 GB, 90.34 GB, 94.30 GB, 98.17 GB, 101.98 GB, 105.57 GB, 109.07 GB, 112.32 GB, 115.46 GB, 118.44 GB, 121.30 GB, 124.08 GB, 126.73 GB, 129.25 GB, 131.88 GB, 134.40 GB, 136.84 GB, 139.32 GB, 141.77 GB, 144.23 GB, 146.70 GB, 148.25 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB - FirstBatchArrivalWaitTime: 974.184ms - TotalBytesReceived: 148.29 GB (159225249939) - TotalGetBatchTime: 5h50m - DataArrivalTimer: 5h49m SenderSide: - DeserializeRowBatchTime: 4h6m - NumBatchesArrived: 6.85M (6848405) - NumBatchesDeferred: 98.59K (98592) - NumBatchesEnqueued: 6.85M (6848405) - NumBatchesReceived: 6.85M (6848405) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 58.338ms, non-child: 58.338ms, % non-child: 100.00%) - CodegenTime: 1.108ms - CompileTime: 4.763ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 29.100ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.872ms Instance 844a0200d7876345:20bb38b90000009a (host=vc1507.foo.com:22000):(Total: 1s239ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h15m - Prepare Finished: 186.472ms (186.472ms) - Open Finished: 1s240ms (1s053ms) - First Batch Produced: 7h15m (7h15m) - First Batch Sent: 7h15m (10.322ms) MemoryUsage(8m32s): 22.80 GB, 40.35 GB, 49.70 GB, 57.96 GB, 65.52 GB, 72.19 GB, 77.96 GB, 83.40 GB, 88.58 GB, 93.67 GB, 98.36 GB, 102.67 GB, 106.88 GB, 111.02 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.70 GB, 128.94 GB, 132.05 GB, 135.08 GB, 137.96 GB, 140.71 GB, 143.56 GB, 146.31 GB, 148.97 GB, 151.66 GB, 154.32 GB, 157.00 GB, 159.69 GB, 161.38 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.26 GB (188182757334) - RowsProduced: 243.55M (243548838) - TotalNetworkReceiveTime: 5h53m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 1s053ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 61.990ms - TotalThreadsVoluntaryContextSwitches: 12 (12) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h15m - OpenTime: 1s053ms - ExecTreeOpenTime: 988.615ms - PrepareTime: 186.128ms - ExecTreePrepareTime: 148.111us KuduTableSink:(Total: 46m12s, non-child: 46m12s, % non-child: 100.00%) - KuduApplyTimer: 37m23s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 87.86 K/sec - TotalNumRows: 243.55M (243547814) SORT_NODE (id=2):(Total: 7h15m, non-child: 1h21m, % non-child: 18.66%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 31m35s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 243.55M (243548838) - RowsReturnedRate: 9.33 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276638260) Buffer pool: - AllocTime: 29m32s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h54m, non-child: 51s976ms, % non-child: 0.24%) - ConvertRowBatchTime: 21s471ms - PeakMemoryUsage: 19.53 MB (20482123) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 29.80 K/sec Buffer pool: - AllocTime: 2h50m - CumulativeAllocationBytes: 287.42 GB (308610826240) - CumulativeAllocations: 15.07M (15068888) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.01 GB, 40.75 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.65 GB, 99.40 GB, 103.75 GB, 108.01 GB, 112.20 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.04 GB, 130.31 GB, 133.45 GB, 136.52 GB, 139.43 GB, 142.20 GB, 145.09 GB, 147.87 GB, 150.55 GB, 153.27 GB, 155.97 GB, 158.67 GB, 161.40 GB, 163.10 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 988.553ms - TotalBytesReceived: 163.15 GB (175175890487) - TotalGetBatchTime: 5h53m - DataArrivalTimer: 5h53m SenderSide: - DeserializeRowBatchTime: 3h3m - NumBatchesArrived: 7.53M (7534444) - NumBatchesDeferred: 58.33K (58328) - NumBatchesEnqueued: 7.53M (7534444) - NumBatchesReceived: 7.53M (7534444) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 63.854ms, non-child: 63.854ms, % non-child: 100.00%) - CodegenTime: 1.066ms - CompileTime: 5.382ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.339ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 25.373ms Instance 844a0200d7876345:20bb38b900000084 (host=vd1117.foo.com:22000):(Total: 1s238ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h23m - Prepare Finished: 207.050ms (207.050ms) - Open Finished: 1s239ms (1s032ms) - First Batch Produced: 7h23m (7h23m) - First Batch Sent: 7h23m (10.941ms) MemoryUsage(8m32s): 22.79 GB, 40.35 GB, 49.70 GB, 57.96 GB, 65.53 GB, 72.20 GB, 77.96 GB, 83.40 GB, 88.58 GB, 93.67 GB, 98.36 GB, 102.67 GB, 106.88 GB, 111.03 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.70 GB, 128.94 GB, 132.05 GB, 135.08 GB, 137.96 GB, 140.71 GB, 143.56 GB, 146.32 GB, 148.97 GB, 151.66 GB, 154.33 GB, 157.00 GB, 159.70 GB, 161.38 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.36 GB (188290961521) - RowsProduced: 215.22M (215215228) - TotalNetworkReceiveTime: 5h44m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 1s032ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 51.992ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h23m - OpenTime: 1s032ms - ExecTreeOpenTime: 979.066ms - PrepareTime: 206.609ms - ExecTreePrepareTime: 152.685us KuduTableSink:(Total: 37m27s, non-child: 37m27s, % non-child: 100.00%) - KuduApplyTimer: 30m56s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 95.75 K/sec - TotalNumRows: 215.21M (215214204) SORT_NODE (id=2):(Total: 7h23m, non-child: 1h38m, % non-child: 22.30%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 40m13s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 215.22M (215215228) - RowsReturnedRate: 8.08 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276030034) Buffer pool: - AllocTime: 38m31s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h44m, non-child: 50s310ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s770ms - PeakMemoryUsage: 19.53 MB (20481865) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.59 K/sec Buffer pool: - AllocTime: 3h51m - CumulativeAllocationBytes: 287.42 GB (308619427840) - CumulativeAllocations: 15.07M (15069308) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.04 GB, 40.76 GB, 50.21 GB, 58.56 GB, 66.21 GB, 72.95 GB, 78.78 GB, 84.27 GB, 89.51 GB, 94.66 GB, 99.40 GB, 103.76 GB, 108.01 GB, 112.20 GB, 116.16 GB, 120.01 GB, 123.58 GB, 127.04 GB, 130.31 GB, 133.46 GB, 136.52 GB, 139.43 GB, 142.21 GB, 145.09 GB, 147.88 GB, 150.56 GB, 153.28 GB, 155.97 GB, 158.68 GB, 161.40 GB, 163.10 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 978.998ms - TotalBytesReceived: 163.15 GB (175180163919) - TotalGetBatchTime: 5h44m - DataArrivalTimer: 5h44m SenderSide: - DeserializeRowBatchTime: 4h4m - NumBatchesArrived: 7.53M (7534654) - NumBatchesDeferred: 172.92K (172923) - NumBatchesEnqueued: 7.53M (7534654) - NumBatchesReceived: 7.53M (7534654) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 51.836ms, non-child: 51.836ms, % non-child: 100.00%) - CodegenTime: 1.027ms - CompileTime: 4.776ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 25.790ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 20.714ms Instance 844a0200d7876345:20bb38b900000088 (host=vd1105.foo.com:22000):(Total: 1s237ms, non-child: 217.804ms, % non-child: 17.60%) Fragment Instance Lifecycle Event Timeline: 1s238ms - Prepare Finished: 217.159ms (217.159ms) - Open Finished: 1s238ms (1s021ms) MemoryUsage(8m32s): 22.80 GB, 40.34 GB, 49.69 GB, 57.95 GB, 65.51 GB, 72.18 GB, 77.95 GB, 83.38 GB, 88.57 GB, 93.65 GB, 98.35 GB, 102.66 GB, 106.87 GB, 111.02 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.69 GB, 128.93 GB, 132.04 GB, 135.07 GB, 137.95 GB, 140.70 GB, 143.55 GB, 146.31 GB, 148.96 GB, 151.65 GB, 154.32 GB, 156.99 GB, 159.69 GB, 161.37 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173319209398) - PeakReservation: 161.38 GB (173279281152) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.28 GB (188210601299) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h51m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1s021ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 60.991ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s021ms - ExecTreeOpenTime: 957.573ms - PrepareTime: 216.621ms - ExecTreePrepareTime: 179.042us KuduTableSink:(Total: 13.281us, non-child: 13.281us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 957.749ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173279289344) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 32m34s - CumulativeAllocationBytes: 161.38 GB (173279281152) - CumulativeAllocations: 82.63K (82626) - PeakReservation: 161.38 GB (173279281152) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173279281152) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h51m, non-child: 51s181ms, % non-child: 0.24%) - ConvertRowBatchTime: 21s179ms - PeakMemoryUsage: 19.53 MB (20481255) - RowsReturned: 632.86M (632860032) - RowsReturnedRate: 30.00 K/sec Buffer pool: - AllocTime: 3h13m - CumulativeAllocationBytes: 287.40 GB (308594606080) - CumulativeAllocations: 15.07M (15068096) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.01 GB, 40.74 GB, 50.19 GB, 58.54 GB, 66.19 GB, 72.93 GB, 78.76 GB, 84.25 GB, 89.49 GB, 94.64 GB, 99.39 GB, 103.74 GB, 108.00 GB, 112.19 GB, 116.14 GB, 119.99 GB, 123.57 GB, 127.03 GB, 130.30 GB, 133.44 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.08 GB, 147.86 GB, 150.55 GB, 153.27 GB, 155.96 GB, 158.67 GB, 161.39 GB, 163.09 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 957.511ms - TotalBytesReceived: 163.14 GB (175166650920) - TotalGetBatchTime: 5h51m - DataArrivalTimer: 5h50m SenderSide: - DeserializeRowBatchTime: 3h21m - NumBatchesArrived: 7.53M (7534048) - NumBatchesDeferred: 89.78K (89776) - NumBatchesEnqueued: 7.53M (7534048) - NumBatchesReceived: 7.53M (7534048) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.276ms, non-child: 62.276ms, % non-child: 100.00%) - CodegenTime: 960.813us - CompileTime: 5.728ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.864ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.176ms Instance 844a0200d7876345:20bb38b9000000a9 (host=vd1129.foo.com:22000):(Total: 1s236ms, non-child: 227.916ms, % non-child: 18.43%) Fragment Instance Lifecycle Event Timeline: 1s236ms - Prepare Finished: 227.025ms (227.025ms) - Open Finished: 1s236ms (1s009ms) MemoryUsage(8m32s): 22.79 GB, 40.34 GB, 49.68 GB, 57.95 GB, 65.51 GB, 72.18 GB, 77.94 GB, 83.38 GB, 88.56 GB, 93.65 GB, 98.35 GB, 102.66 GB, 106.87 GB, 111.01 GB, 114.93 GB, 118.73 GB, 122.27 GB, 125.69 GB, 128.93 GB, 132.04 GB, 135.07 GB, 137.95 GB, 140.70 GB, 143.55 GB, 146.31 GB, 148.96 GB, 151.65 GB, 154.32 GB, 157.00 GB, 159.69 GB, 161.38 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325500854) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.09 GB (186929979149) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h49m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1s009ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 59.991ms - TotalThreadsVoluntaryContextSwitches: 7 (7) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s009ms - ExecTreeOpenTime: 946.493ms - PrepareTime: 226.651ms - ExecTreePrepareTime: 176.570us KuduTableSink:(Total: 12.973us, non-child: 12.973us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 946.667ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173285580800) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 33m6s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h50m, non-child: 52s340ms, % non-child: 0.25%) - ConvertRowBatchTime: 21s419ms - PeakMemoryUsage: 19.53 MB (20482178) - RowsReturned: 632.89M (632885652) - RowsReturnedRate: 30.10 K/sec Buffer pool: - AllocTime: 3h18m - CumulativeAllocationBytes: 287.41 GB (308607098880) - CumulativeAllocations: 15.07M (15068706) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.75 GB, 50.19 GB, 58.55 GB, 66.19 GB, 72.94 GB, 78.76 GB, 84.25 GB, 89.49 GB, 94.64 GB, 99.39 GB, 103.74 GB, 108.00 GB, 112.19 GB, 116.14 GB, 119.99 GB, 123.57 GB, 127.03 GB, 130.30 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.08 GB, 147.87 GB, 150.55 GB, 153.27 GB, 155.97 GB, 158.67 GB, 161.40 GB, 163.10 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 946.417ms - TotalBytesReceived: 163.14 GB (175173178491) - TotalGetBatchTime: 5h49m - DataArrivalTimer: 5h49m SenderSide: - DeserializeRowBatchTime: 3h28m - NumBatchesArrived: 7.53M (7534353) - NumBatchesDeferred: 127.00K (127002) - NumBatchesEnqueued: 7.53M (7534353) - NumBatchesReceived: 7.53M (7534353) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 61.872ms, non-child: 61.872ms, % non-child: 100.00%) - CodegenTime: 1.316ms - CompileTime: 5.795ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 33.444ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 21.918ms Instance 844a0200d7876345:20bb38b90000008f (host=vc1531.foo.com:22000):(Total: 1s234ms, non-child: 203.622ms, % non-child: 16.50%) Fragment Instance Lifecycle Event Timeline: 1s234ms - Prepare Finished: 202.448ms (202.448ms) - Open Finished: 1s234ms (1s032ms) MemoryUsage(8m32s): 22.80 GB, 40.34 GB, 49.69 GB, 57.95 GB, 65.52 GB, 72.19 GB, 77.95 GB, 83.38 GB, 88.57 GB, 93.66 GB, 98.35 GB, 102.66 GB, 106.87 GB, 111.01 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.70 GB, 128.93 GB, 132.04 GB, 135.07 GB, 137.95 GB, 140.70 GB, 143.55 GB, 146.31 GB, 148.96 GB, 151.65 GB, 154.32 GB, 157.00 GB, 159.69 GB, 161.37 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173323403702) - PeakReservation: 161.38 GB (173283475456) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.28 GB (188201747221) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h53m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 1s032ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 50.993ms - TotalThreadsVoluntaryContextSwitches: 16 (16) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s032ms - ExecTreeOpenTime: 979.145ms - PrepareTime: 202.145ms - ExecTreePrepareTime: 181.249us KuduTableSink:(Total: 14.580us, non-child: 14.580us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 979.324ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173283483648) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 30m18s - CumulativeAllocationBytes: 161.38 GB (173283475456) - CumulativeAllocations: 82.63K (82628) - PeakReservation: 161.38 GB (173283475456) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173283475456) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h54m, non-child: 51s144ms, % non-child: 0.24%) - ConvertRowBatchTime: 21s336ms - PeakMemoryUsage: 19.53 MB (20482115) - RowsReturned: 632.87M (632871540) - RowsReturnedRate: 29.78 K/sec Buffer pool: - AllocTime: 3h5m - CumulativeAllocationBytes: 287.41 GB (308600217600) - CumulativeAllocations: 15.07M (15068370) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.02 GB, 40.74 GB, 50.19 GB, 58.54 GB, 66.19 GB, 72.93 GB, 78.76 GB, 84.25 GB, 89.49 GB, 94.64 GB, 99.39 GB, 103.74 GB, 107.99 GB, 112.19 GB, 116.14 GB, 119.99 GB, 123.57 GB, 127.03 GB, 130.30 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.08 GB, 147.86 GB, 150.55 GB, 153.27 GB, 155.96 GB, 158.67 GB, 161.39 GB, 163.09 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 979.072ms - TotalBytesReceived: 163.14 GB (175169968899) - TotalGetBatchTime: 5h53m - DataArrivalTimer: 5h53m SenderSide: - DeserializeRowBatchTime: 3h15m - NumBatchesArrived: 7.53M (7534185) - NumBatchesDeferred: 79.60K (79604) - NumBatchesEnqueued: 7.53M (7534185) - NumBatchesReceived: 7.53M (7534185) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 51.343ms, non-child: 51.343ms, % non-child: 100.00%) - CodegenTime: 1.037ms - CompileTime: 5.678ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 26.732ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 18.462ms Instance 844a0200d7876345:20bb38b9000000ff (host=vd1130.foo.com:22000):(Total: 1s233ms, non-child: 208.526ms, % non-child: 16.90%) Fragment Instance Lifecycle Event Timeline: 1s234ms - Prepare Finished: 207.537ms (207.537ms) - Open Finished: 1s234ms (1s026ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.28 GB, 117.23 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157573792182) - PeakReservation: 146.71 GB (157533863936) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.44 GB (171200212402) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h56m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1s026ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 61.991ms - TotalThreadsVoluntaryContextSwitches: 8 (8) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s026ms - ExecTreeOpenTime: 962.966ms - PrepareTime: 207.090ms - ExecTreePrepareTime: 166.948us KuduTableSink:(Total: 12.142us, non-child: 12.142us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 963.130ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157533872128) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 27m48s - CumulativeAllocationBytes: 146.71 GB (157533863936) - CumulativeAllocations: 75.12K (75118) - PeakReservation: 146.71 GB (157533863936) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157533863936) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h57m, non-child: 47s467ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s755ms - PeakMemoryUsage: 19.53 MB (20483599) - RowsReturned: 575.36M (575355312) - RowsReturnedRate: 26.86 K/sec Buffer pool: - AllocTime: 2h30m - CumulativeAllocationBytes: 261.29 GB (280554209280) - CumulativeAllocations: 13.70M (13698936) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 962.903ms - TotalBytesReceived: 148.31 GB (159249952002) - TotalGetBatchTime: 5h56m - DataArrivalTimer: 5h56m SenderSide: - DeserializeRowBatchTime: 2h38m - NumBatchesArrived: 6.85M (6849468) - NumBatchesDeferred: 24.55K (24545) - NumBatchesEnqueued: 6.85M (6849468) - NumBatchesReceived: 6.85M (6849468) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.279ms, non-child: 62.279ms, % non-child: 100.00%) - CodegenTime: 1.136ms - CompileTime: 5.146ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.336ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.712ms Instance 844a0200d7876345:20bb38b900000089 (host=vd1110.foo.com:22000):(Total: 1s233ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h13m - Prepare Finished: 213.602ms (213.602ms) - Open Finished: 1s234ms (1s020ms) - First Batch Produced: 7h13m (7h13m) - First Batch Sent: 7h13m (9.253ms) MemoryUsage(8m32s): 22.81 GB, 40.35 GB, 49.70 GB, 57.96 GB, 65.53 GB, 72.20 GB, 77.96 GB, 83.40 GB, 88.58 GB, 93.67 GB, 98.37 GB, 102.68 GB, 106.88 GB, 111.03 GB, 114.95 GB, 118.76 GB, 122.30 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.09 GB, 137.97 GB, 140.72 GB, 143.57 GB, 146.32 GB, 148.98 GB, 151.67 GB, 154.33 GB, 157.01 GB, 159.71 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325926177) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.25 GB (188170087602) - RowsProduced: 253.92M (253918230) - TotalNetworkReceiveTime: 5h49m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s020ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 59.991ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h13m - OpenTime: 1s020ms - ExecTreeOpenTime: 956.328ms - PrepareTime: 212.914ms - ExecTreePrepareTime: 148.119us KuduTableSink:(Total: 47m43s, non-child: 47m43s, % non-child: 100.00%) - KuduApplyTimer: 39m10s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 88.66 K/sec - TotalNumRows: 253.92M (253917206) SORT_NODE (id=2):(Total: 7h13m, non-child: 1h23m, % non-child: 19.37%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 29m53s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 253.92M (253918230) - RowsReturnedRate: 9.76 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276632744) Buffer pool: - AllocTime: 34m40s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h49m, non-child: 51s590ms, % non-child: 0.25%) - ConvertRowBatchTime: 21s181ms - PeakMemoryUsage: 19.53 MB (20481380) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.18 K/sec Buffer pool: - AllocTime: 4h31m - CumulativeAllocationBytes: 287.43 GB (308628357120) - CumulativeAllocations: 15.07M (15069744) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.02 GB, 40.75 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.27 GB, 89.51 GB, 94.65 GB, 99.40 GB, 103.76 GB, 108.01 GB, 112.20 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.32 GB, 133.46 GB, 136.53 GB, 139.44 GB, 142.21 GB, 145.10 GB, 147.88 GB, 150.56 GB, 153.28 GB, 155.98 GB, 158.69 GB, 161.41 GB, 163.11 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 956.211ms - TotalBytesReceived: 163.15 GB (175185837152) - TotalGetBatchTime: 5h48m - DataArrivalTimer: 5h48m SenderSide: - DeserializeRowBatchTime: 4h42m - NumBatchesArrived: 7.53M (7534907) - NumBatchesDeferred: 122.81K (122815) - NumBatchesEnqueued: 7.53M (7534872) - NumBatchesReceived: 7.53M (7534872) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.780ms, non-child: 62.780ms, % non-child: 100.00%) - CodegenTime: 1.478ms - CompileTime: 5.559ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.392ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.124ms Instance 844a0200d7876345:20bb38b9000000ee (host=vd1135.foo.com:22000):(Total: 1s233ms, non-child: 193.401ms, % non-child: 15.68%) Fragment Instance Lifecycle Event Timeline: 1s233ms - Prepare Finished: 192.613ms (192.613ms) - Open Finished: 1s233ms (1s041ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.18 GB, 52.70 GB, 59.58 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.35 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.28 GB, 117.23 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157577986486) - PeakReservation: 146.72 GB (157538058240) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.46 GB (171223949275) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h57m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1s041ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 62.991ms - TotalThreadsVoluntaryContextSwitches: 7 (7) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s041ms - ExecTreeOpenTime: 977.019ms - PrepareTime: 192.188ms - ExecTreePrepareTime: 151.279us KuduTableSink:(Total: 16.020us, non-child: 16.020us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 977.167ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157538066432) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 27m9s - CumulativeAllocationBytes: 146.72 GB (157538058240) - CumulativeAllocations: 75.12K (75120) - PeakReservation: 146.72 GB (157538058240) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157538058240) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h58m, non-child: 48s807ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s166ms - PeakMemoryUsage: 19.53 MB (20481833) - RowsReturned: 575.37M (575370600) - RowsReturnedRate: 26.78 K/sec Buffer pool: - AllocTime: 2h34m - CumulativeAllocationBytes: 261.29 GB (280561664000) - CumulativeAllocations: 13.70M (13699300) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.63 GB, 53.23 GB, 60.19 GB, 66.32 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.37 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.28 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 976.945ms - TotalBytesReceived: 148.32 GB (159254425913) - TotalGetBatchTime: 5h57m - DataArrivalTimer: 5h57m SenderSide: - DeserializeRowBatchTime: 2h43m - NumBatchesArrived: 6.85M (6849650) - NumBatchesDeferred: 26.57K (26573) - NumBatchesEnqueued: 6.85M (6849650) - NumBatchesReceived: 6.85M (6849650) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.698ms, non-child: 62.698ms, % non-child: 100.00%) - CodegenTime: 1.416ms - CompileTime: 5.213ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.258ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 25.637ms Instance 844a0200d7876345:20bb38b9000000aa (host=vc1533.foo.com:22000):(Total: 1s233ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h14m - Prepare Finished: 222.018ms (222.018ms) - Open Finished: 1s233ms (1s011ms) - First Batch Produced: 7h14m (7h14m) - First Batch Sent: 7h14m (9.707ms) MemoryUsage(8m32s): 22.80 GB, 40.35 GB, 49.69 GB, 57.96 GB, 65.52 GB, 72.19 GB, 77.96 GB, 83.40 GB, 88.58 GB, 93.67 GB, 98.37 GB, 102.68 GB, 106.88 GB, 111.03 GB, 114.95 GB, 118.75 GB, 122.29 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.09 GB, 137.97 GB, 140.72 GB, 143.57 GB, 146.33 GB, 148.98 GB, 151.67 GB, 154.33 GB, 157.01 GB, 159.71 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173327037140) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.40 GB (188336111681) - RowsProduced: 260.46M (260460280) - TotalNetworkReceiveTime: 5h53m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 1s011ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 50.992ms - TotalThreadsVoluntaryContextSwitches: 11 (11) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h14m - OpenTime: 1s011ms - ExecTreeOpenTime: 960.343ms - PrepareTime: 221.726ms - ExecTreePrepareTime: 160.832us KuduTableSink:(Total: 47m13s, non-child: 47m13s, % non-child: 100.00%) - KuduApplyTimer: 38m23s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 91.92 K/sec - TotalNumRows: 260.46M (260459256) SORT_NODE (id=2):(Total: 7h14m, non-child: 1h20m, % non-child: 18.54%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 30m44s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 260.46M (260460280) - RowsReturnedRate: 9.99 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276051168) Buffer pool: - AllocTime: 30m36s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 47s842ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s214ms - PeakMemoryUsage: 19.53 MB (20482086) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 29.82 K/sec Buffer pool: - AllocTime: 3h3m - CumulativeAllocationBytes: 287.43 GB (308628398080) - CumulativeAllocations: 15.07M (15069746) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.01 GB, 40.75 GB, 50.19 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.27 GB, 89.51 GB, 94.65 GB, 99.40 GB, 103.76 GB, 108.01 GB, 112.20 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.32 GB, 133.46 GB, 136.53 GB, 139.44 GB, 142.22 GB, 145.10 GB, 147.88 GB, 150.56 GB, 153.29 GB, 155.98 GB, 158.69 GB, 161.41 GB, 163.11 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 960.263ms - TotalBytesReceived: 163.15 GB (175185277040) - TotalGetBatchTime: 5h53m - DataArrivalTimer: 5h52m SenderSide: - DeserializeRowBatchTime: 3h14m - NumBatchesArrived: 7.54M (7535001) - NumBatchesDeferred: 129.94K (129941) - NumBatchesEnqueued: 7.53M (7534873) - NumBatchesReceived: 7.53M (7534873) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 49.865ms, non-child: 49.865ms, % non-child: 100.00%) - CodegenTime: 1.015ms - CompileTime: 4.588ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 27.281ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 17.540ms Instance 844a0200d7876345:20bb38b9000000d4 (host=ve1139.foo.com:22000):(Total: 1s232ms, non-child: 213.539ms, % non-child: 17.32%) Fragment Instance Lifecycle Event Timeline: 1s232ms - Prepare Finished: 212.688ms (212.688ms) - Open Finished: 1s232ms (1s020ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.17 GB, 52.69 GB, 59.56 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.14 GB, 89.41 GB, 93.33 GB, 97.15 GB, 100.92 GB, 104.48 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.91 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.72 GB, 145.17 GB, 146.71 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157565403574) - PeakReservation: 146.71 GB (157525475328) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.40 GB (171149354521) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h54m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 1s020ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 63.990ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s020ms - ExecTreeOpenTime: 953.073ms - PrepareTime: 212.313ms - ExecTreePrepareTime: 153.340us KuduTableSink:(Total: 12.055us, non-child: 12.055us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 953.223ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157525483520) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 29m50s - CumulativeAllocationBytes: 146.71 GB (157525475328) - CumulativeAllocations: 75.11K (75114) - PeakReservation: 146.71 GB (157525475328) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157525475328) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h55m, non-child: 46s451ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s602ms - PeakMemoryUsage: 19.53 MB (20483314) - RowsReturned: 575.33M (575331876) - RowsReturnedRate: 26.98 K/sec Buffer pool: - AllocTime: 2h34m - CumulativeAllocationBytes: 261.28 GB (280542781440) - CumulativeAllocations: 13.70M (13698378) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.63 GB, 53.22 GB, 60.17 GB, 66.30 GB, 71.60 GB, 76.59 GB, 81.36 GB, 86.03 GB, 90.35 GB, 94.31 GB, 98.17 GB, 101.99 GB, 105.58 GB, 109.08 GB, 112.34 GB, 115.48 GB, 118.45 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.27 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 952.993ms - TotalBytesReceived: 148.31 GB (159243352345) - TotalGetBatchTime: 5h54m - DataArrivalTimer: 5h54m SenderSide: - DeserializeRowBatchTime: 2h43m - NumBatchesArrived: 6.85M (6849189) - NumBatchesDeferred: 74.01K (74009) - NumBatchesEnqueued: 6.85M (6849189) - NumBatchesReceived: 6.85M (6849189) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 65.810ms, non-child: 65.810ms, % non-child: 100.00%) - CodegenTime: 1.071ms - CompileTime: 6.604ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 34.045ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.566ms Instance 844a0200d7876345:20bb38b900000100 (host=vc1526.foo.com:22000):(Total: 1s231ms, non-child: 217.379ms, % non-child: 17.65%) Fragment Instance Lifecycle Event Timeline: 1s232ms - Prepare Finished: 216.279ms (216.279ms) - Open Finished: 1s232ms (1s015ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.19 GB, 52.70 GB, 59.57 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.94 GB, 104.49 GB, 107.96 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157577986486) - PeakReservation: 146.72 GB (157538058240) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.70 GB (172554349562) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h51m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s015ms - TotalThreadsSysTime: 999.000us - TotalThreadsUserTime: 53.992ms - TotalThreadsVoluntaryContextSwitches: 14 (14) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s015ms - ExecTreeOpenTime: 959.529ms - PrepareTime: 215.979ms - ExecTreePrepareTime: 159.772us KuduTableSink:(Total: 18.746us, non-child: 18.746us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 959.686ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157538066432) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 32m23s - CumulativeAllocationBytes: 146.72 GB (157538058240) - CumulativeAllocations: 75.12K (75120) - PeakReservation: 146.72 GB (157538058240) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157538058240) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h52m, non-child: 46s039ms, % non-child: 0.22%) - ConvertRowBatchTime: 18s846ms - PeakMemoryUsage: 19.54 MB (20484127) - RowsReturned: 575.37M (575366568) - RowsReturnedRate: 27.22 K/sec Buffer pool: - AllocTime: 3h37m - CumulativeAllocationBytes: 261.29 GB (280559697920) - CumulativeAllocations: 13.70M (13699204) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.18 GB, 102.00 GB, 105.59 GB, 109.09 GB, 112.35 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.27 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 959.430ms - TotalBytesReceived: 148.32 GB (159253163539) - TotalGetBatchTime: 5h51m - DataArrivalTimer: 5h51m SenderSide: - DeserializeRowBatchTime: 3h53m - NumBatchesArrived: 6.85M (6849602) - NumBatchesDeferred: 84.19K (84186) - NumBatchesEnqueued: 6.85M (6849602) - NumBatchesReceived: 6.85M (6849602) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 54.832ms, non-child: 54.832ms, % non-child: 100.00%) - CodegenTime: 1.009ms - CompileTime: 5.567ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 26.093ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.682ms Instance 844a0200d7876345:20bb38b9000000e3 (host=vd1121.foo.com:22000):(Total: 1s230ms, non-child: 177.946ms, % non-child: 14.46%) Fragment Instance Lifecycle Event Timeline: 1s230ms - Prepare Finished: 177.253ms (177.253ms) - Open Finished: 1s230ms (1s053ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.48 GB, 107.95 GB, 111.16 GB, 114.27 GB, 117.21 GB, 120.04 GB, 122.79 GB, 125.41 GB, 127.91 GB, 130.50 GB, 133.00 GB, 135.42 GB, 137.86 GB, 140.28 GB, 142.72 GB, 145.17 GB, 146.70 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157561209270) - PeakReservation: 146.70 GB (157521281024) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.69 GB (172535898899) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h46m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 1s053ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 52.992ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s053ms - ExecTreeOpenTime: 999.458ms - PrepareTime: 176.822ms - ExecTreePrepareTime: 203.459us KuduTableSink:(Total: 13.615us, non-child: 13.615us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 999.658ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.70 GB (157521289216) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 36m37s - CumulativeAllocationBytes: 146.70 GB (157521281024) - CumulativeAllocations: 75.11K (75112) - PeakReservation: 146.70 GB (157521281024) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.70 GB (157521281024) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h46m, non-child: 48s431ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s635ms - PeakMemoryUsage: 19.53 MB (20483733) - RowsReturned: 575.30M (575301636) - RowsReturnedRate: 27.65 K/sec Buffer pool: - AllocTime: 4h15m - CumulativeAllocationBytes: 261.26 GB (280528035840) - CumulativeAllocations: 13.70M (13697658) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.45 GB, 121.31 GB, 124.09 GB, 126.74 GB, 129.26 GB, 131.89 GB, 134.41 GB, 136.85 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.71 GB, 148.26 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB - FirstBatchArrivalWaitTime: 999.379ms - TotalBytesReceived: 148.30 GB (159235403047) - TotalGetBatchTime: 5h46m - DataArrivalTimer: 5h46m SenderSide: - DeserializeRowBatchTime: 4h26m - NumBatchesArrived: 6.85M (6848829) - NumBatchesDeferred: 152.62K (152622) - NumBatchesEnqueued: 6.85M (6848829) - NumBatchesReceived: 6.85M (6848829) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 52.768ms, non-child: 52.768ms, % non-child: 100.00%) - CodegenTime: 1.134ms - CompileTime: 4.515ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.670ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.024ms Instance 844a0200d7876345:20bb38b9000000f1 (host=vd1104.foo.com:22000):(Total: 1s227ms, non-child: 228.914ms, % non-child: 18.64%) Fragment Instance Lifecycle Event Timeline: 1s228ms - Prepare Finished: 228.153ms (228.153ms) - Open Finished: 1s228ms (1s000ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.18 GB, 52.70 GB, 59.58 GB, 65.64 GB, 70.88 GB, 75.83 GB, 80.54 GB, 85.16 GB, 89.44 GB, 93.35 GB, 97.18 GB, 100.95 GB, 104.51 GB, 107.97 GB, 111.19 GB, 114.29 GB, 117.24 GB, 120.07 GB, 122.82 GB, 125.44 GB, 127.93 GB, 130.53 GB, 133.03 GB, 135.44 GB, 137.89 GB, 140.31 GB, 142.75 GB, 145.20 GB, 146.73 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.77 GB (157594763702) - PeakReservation: 146.73 GB (157554835456) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.68 GB (172524821487) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h54m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s000ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 58.991ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s000ms - ExecTreeOpenTime: 937.996ms - PrepareTime: 227.705ms - ExecTreePrepareTime: 190.384us KuduTableSink:(Total: 12.425us, non-child: 12.425us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 938.184ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.73 GB (157554843648) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 29m24s - CumulativeAllocationBytes: 146.73 GB (157554835456) - CumulativeAllocations: 75.13K (75128) - PeakReservation: 146.73 GB (157554835456) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.73 GB (157554835456) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h55m, non-child: 47s325ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s587ms - PeakMemoryUsage: 19.54 MB (20484183) - RowsReturned: 575.43M (575428896) - RowsReturnedRate: 27.01 K/sec Buffer pool: - AllocTime: 2h56m - CumulativeAllocationBytes: 261.32 GB (280590090240) - CumulativeAllocations: 13.70M (13700688) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.63 GB, 53.23 GB, 60.19 GB, 66.32 GB, 71.62 GB, 76.61 GB, 81.38 GB, 86.05 GB, 90.37 GB, 94.33 GB, 98.20 GB, 102.01 GB, 105.61 GB, 109.11 GB, 112.36 GB, 115.50 GB, 118.48 GB, 121.34 GB, 124.12 GB, 126.77 GB, 129.29 GB, 131.91 GB, 134.44 GB, 136.88 GB, 139.36 GB, 141.81 GB, 144.27 GB, 146.74 GB, 148.29 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB - FirstBatchArrivalWaitTime: 937.935ms - TotalBytesReceived: 148.33 GB (159270502953) - TotalGetBatchTime: 5h54m - DataArrivalTimer: 5h54m SenderSide: - DeserializeRowBatchTime: 3h7m - NumBatchesArrived: 6.85M (6850344) - NumBatchesDeferred: 60.38K (60381) - NumBatchesEnqueued: 6.85M (6850344) - NumBatchesReceived: 6.85M (6850344) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 60.691ms, non-child: 60.691ms, % non-child: 100.00%) - CodegenTime: 1.142ms - CompileTime: 5.152ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.613ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.178ms Instance 844a0200d7876345:20bb38b9000000e5 (host=va1036.foo.com:22000):(Total: 1s227ms, non-child: 222.713ms, % non-child: 18.15%) Fragment Instance Lifecycle Event Timeline: 1s227ms - Prepare Finished: 221.686ms (221.686ms) - Open Finished: 1s227ms (1s005ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.49 GB, 107.96 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157580083638) - PeakReservation: 146.72 GB (157540155392) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.73 GB (172578984338) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h55m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s005ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 60.990ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s005ms - ExecTreeOpenTime: 943.386ms - PrepareTime: 221.318ms - ExecTreePrepareTime: 151.820us KuduTableSink:(Total: 11.741us, non-child: 11.741us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 943.535ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157540163584) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 28m34s - CumulativeAllocationBytes: 146.72 GB (157540155392) - CumulativeAllocations: 75.12K (75121) - PeakReservation: 146.72 GB (157540155392) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157540155392) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h55m, non-child: 47s648ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s796ms - PeakMemoryUsage: 19.53 MB (20482229) - RowsReturned: 575.38M (575382108) - RowsReturnedRate: 26.96 K/sec Buffer pool: - AllocTime: 2h39m - CumulativeAllocationBytes: 261.30 GB (280567275520) - CumulativeAllocations: 13.70M (13699574) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.76 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 943.319ms - TotalBytesReceived: 148.32 GB (159257632950) - TotalGetBatchTime: 5h55m - DataArrivalTimer: 5h54m SenderSide: - DeserializeRowBatchTime: 2h47m - NumBatchesArrived: 6.85M (6849787) - NumBatchesDeferred: 25.37K (25367) - NumBatchesEnqueued: 6.85M (6849787) - NumBatchesReceived: 6.85M (6849787) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 60.742ms, non-child: 60.742ms, % non-child: 100.00%) - CodegenTime: 1.213ms - CompileTime: 5.434ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 30.691ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.017ms Instance 844a0200d7876345:20bb38b9000000d9 (host=ve1134.foo.com:22000):(Total: 1s226ms, non-child: 201.759ms, % non-child: 16.45%) Fragment Instance Lifecycle Event Timeline: 1s227ms - Prepare Finished: 201.144ms (201.144ms) - Open Finished: 1s227ms (1s026ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.33 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157582180790) - PeakReservation: 146.72 GB (157542252544) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.46 GB (171213656099) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h43m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 1s026ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 65.990ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s026ms - ExecTreeOpenTime: 956.347ms - PrepareTime: 200.630ms - ExecTreePrepareTime: 162.502us KuduTableSink:(Total: 12.505us, non-child: 12.505us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 956.507ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157542260736) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 40m47s - CumulativeAllocationBytes: 146.72 GB (157542252544) - CumulativeAllocations: 75.12K (75122) - PeakReservation: 146.72 GB (157542252544) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157542252544) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h43m, non-child: 48s300ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s580ms - PeakMemoryUsage: 19.54 MB (20484610) - RowsReturned: 575.39M (575386644) - RowsReturnedRate: 27.90 K/sec Buffer pool: - AllocTime: 3h31m - CumulativeAllocationBytes: 261.30 GB (280569487360) - CumulativeAllocations: 13.70M (13699682) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.18 GB, 102.00 GB, 105.59 GB, 109.09 GB, 112.35 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.35 GB, 141.79 GB, 144.26 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 956.257ms - TotalBytesReceived: 148.32 GB (159258779958) - TotalGetBatchTime: 5h43m - DataArrivalTimer: 5h42m SenderSide: - DeserializeRowBatchTime: 3h41m - NumBatchesArrived: 6.85M (6849841) - NumBatchesDeferred: 153.38K (153383) - NumBatchesEnqueued: 6.85M (6849841) - NumBatchesReceived: 6.85M (6849841) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 68.494ms, non-child: 68.494ms, % non-child: 100.00%) - CodegenTime: 1.161ms - CompileTime: 5.621ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 34.729ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 27.622ms Instance 844a0200d7876345:20bb38b9000000a2 (host=vd1136.foo.com:22000):(Total: 1s226ms, non-child: 220.749ms, % non-child: 18.00%) Fragment Instance Lifecycle Event Timeline: 1s226ms - Prepare Finished: 219.948ms (219.948ms) - Open Finished: 1s226ms (1s006ms) MemoryUsage(8m32s): 22.80 GB, 40.35 GB, 49.69 GB, 57.95 GB, 65.52 GB, 72.19 GB, 77.95 GB, 83.38 GB, 88.57 GB, 93.66 GB, 98.35 GB, 102.66 GB, 106.87 GB, 111.01 GB, 114.93 GB, 118.74 GB, 122.27 GB, 125.69 GB, 128.93 GB, 132.04 GB, 135.07 GB, 137.95 GB, 140.69 GB, 143.55 GB, 146.30 GB, 148.95 GB, 151.64 GB, 154.31 GB, 156.98 GB, 159.68 GB, 161.36 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.40 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.41 GB (173308723638) - PeakReservation: 161.37 GB (173268795392) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.06 GB (186897707003) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h49m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 1s006ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 56.991ms - TotalThreadsVoluntaryContextSwitches: 5 (5) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s006ms - ExecTreeOpenTime: 949.889ms - PrepareTime: 219.472ms - ExecTreePrepareTime: 216.178us KuduTableSink:(Total: 12.920us, non-child: 12.920us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 950.103ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.37 GB (173268803584) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 34m8s - CumulativeAllocationBytes: 161.37 GB (173268795392) - CumulativeAllocations: 82.62K (82621) - PeakReservation: 161.37 GB (173268795392) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.37 GB (173268795392) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h49m, non-child: 50s746ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s989ms - PeakMemoryUsage: 19.53 MB (20481501) - RowsReturned: 632.82M (632824668) - RowsReturnedRate: 30.15 K/sec Buffer pool: - AllocTime: 4h4m - CumulativeAllocationBytes: 287.39 GB (308577361920) - CumulativeAllocations: 15.07M (15067254) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.01 GB, 40.75 GB, 50.19 GB, 58.55 GB, 66.19 GB, 72.93 GB, 78.76 GB, 84.25 GB, 89.49 GB, 94.64 GB, 99.39 GB, 103.74 GB, 107.99 GB, 112.19 GB, 116.14 GB, 119.99 GB, 123.57 GB, 127.02 GB, 130.30 GB, 133.44 GB, 136.50 GB, 139.41 GB, 142.19 GB, 145.08 GB, 147.86 GB, 150.54 GB, 153.26 GB, 155.95 GB, 158.66 GB, 161.38 GB, 163.08 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.12 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB - FirstBatchArrivalWaitTime: 949.799ms - TotalBytesReceived: 163.13 GB (175157007730) - TotalGetBatchTime: 5h49m - DataArrivalTimer: 5h48m SenderSide: - DeserializeRowBatchTime: 4h13m - NumBatchesArrived: 7.53M (7533627) - NumBatchesDeferred: 129.25K (129252) - NumBatchesEnqueued: 7.53M (7533627) - NumBatchesReceived: 7.53M (7533627) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 55.571ms, non-child: 55.571ms, % non-child: 100.00%) - CodegenTime: 1.154ms - CompileTime: 5.619ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.320ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 21.024ms Instance 844a0200d7876345:20bb38b9000000ca (host=vc1503.foo.com:22000):(Total: 1s226ms, non-child: 216.416ms, % non-child: 17.65%) Fragment Instance Lifecycle Event Timeline: 1s226ms - Prepare Finished: 215.897ms (215.897ms) - Open Finished: 1s226ms (1s011ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.17 GB, 52.68 GB, 59.56 GB, 65.62 GB, 70.86 GB, 75.80 GB, 80.51 GB, 85.14 GB, 89.41 GB, 93.33 GB, 97.15 GB, 100.92 GB, 104.48 GB, 107.95 GB, 111.16 GB, 114.27 GB, 117.21 GB, 120.04 GB, 122.80 GB, 125.41 GB, 127.91 GB, 130.50 GB, 133.01 GB, 135.42 GB, 137.86 GB, 140.29 GB, 142.72 GB, 145.17 GB, 146.70 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157563306422) - PeakReservation: 146.71 GB (157523378176) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.70 GB (172553938113) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h48m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s010ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 60.991ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s011ms - ExecTreeOpenTime: 949.334ms - PrepareTime: 215.231ms - ExecTreePrepareTime: 154.473us KuduTableSink:(Total: 12.368us, non-child: 12.368us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 949.486ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157523386368) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 36m - CumulativeAllocationBytes: 146.71 GB (157523378176) - CumulativeAllocations: 75.11K (75113) - PeakReservation: 146.71 GB (157523378176) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157523378176) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h49m, non-child: 47s781ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s732ms - PeakMemoryUsage: 19.53 MB (20483896) - RowsReturned: 575.32M (575316756) - RowsReturnedRate: 27.45 K/sec Buffer pool: - AllocTime: 3h56m - CumulativeAllocationBytes: 261.27 GB (280535408640) - CumulativeAllocations: 13.70M (13698018) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.63 GB, 53.22 GB, 60.17 GB, 66.30 GB, 71.60 GB, 76.59 GB, 81.36 GB, 86.03 GB, 90.35 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.26 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB - FirstBatchArrivalWaitTime: 949.242ms - TotalBytesReceived: 148.30 GB (159239019277) - TotalGetBatchTime: 5h48m - DataArrivalTimer: 5h48m SenderSide: - DeserializeRowBatchTime: 4h6m - NumBatchesArrived: 6.85M (6849009) - NumBatchesDeferred: 110.55K (110547) - NumBatchesEnqueued: 6.85M (6849009) - NumBatchesReceived: 6.85M (6849009) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 60.373ms, non-child: 60.373ms, % non-child: 100.00%) - CodegenTime: 1.312ms - CompileTime: 4.843ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.434ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.461ms Instance 844a0200d7876345:20bb38b9000000ad (host=vd1327.foo.com:22000):(Total: 1s226ms, non-child: 224.651ms, % non-child: 18.32%) Fragment Instance Lifecycle Event Timeline: 1s226ms - Prepare Finished: 224.295ms (224.295ms) - Open Finished: 1s226ms (1s002ms) MemoryUsage(8m32s): 22.80 GB, 40.35 GB, 49.69 GB, 57.96 GB, 65.52 GB, 72.19 GB, 77.95 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.35 GB, 102.66 GB, 106.87 GB, 111.01 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.69 GB, 128.93 GB, 132.04 GB, 135.07 GB, 137.95 GB, 140.70 GB, 143.55 GB, 146.30 GB, 148.95 GB, 151.64 GB, 154.31 GB, 156.99 GB, 159.68 GB, 161.37 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.41 GB (173315015094) - PeakReservation: 161.38 GB (173275086848) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.03 GB (186867349859) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h52m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s002ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 60.990ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s002ms - ExecTreeOpenTime: 939.238ms - PrepareTime: 223.584ms - ExecTreePrepareTime: 170.033us KuduTableSink:(Total: 11.954us, non-child: 11.954us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 939.404ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173275095040) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 30m26s - CumulativeAllocationBytes: 161.38 GB (173275086848) - CumulativeAllocations: 82.62K (82624) - PeakReservation: 161.38 GB (173275086848) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173275086848) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h52m, non-child: 51s552ms, % non-child: 0.24%) - ConvertRowBatchTime: 21s439ms - PeakMemoryUsage: 19.53 MB (20481700) - RowsReturned: 632.84M (632839788) - RowsReturnedRate: 29.90 K/sec Buffer pool: - AllocTime: 3h4m - CumulativeAllocationBytes: 287.39 GB (308584734720) - CumulativeAllocations: 15.07M (15067614) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.04 GB, 40.76 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.64 GB, 99.39 GB, 103.74 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.57 GB, 127.03 GB, 130.30 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.19 GB, 145.08 GB, 147.86 GB, 150.54 GB, 153.26 GB, 155.96 GB, 158.66 GB, 161.39 GB, 163.09 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB - FirstBatchArrivalWaitTime: 939.174ms - TotalBytesReceived: 163.13 GB (175161564938) - TotalGetBatchTime: 5h52m - DataArrivalTimer: 5h51m SenderSide: - DeserializeRowBatchTime: 3h14m - NumBatchesArrived: 7.53M (7533807) - NumBatchesDeferred: 81.45K (81451) - NumBatchesEnqueued: 7.53M (7533807) - NumBatchesReceived: 7.53M (7533807) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.170ms, non-child: 62.170ms, % non-child: 100.00%) - CodegenTime: 972.958us - CompileTime: 5.698ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.064ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.876ms Instance 844a0200d7876345:20bb38b9000000bc (host=ve1129.foo.com:22000):(Total: 1s225ms, non-child: 208.178ms, % non-child: 16.99%) Fragment Instance Lifecycle Event Timeline: 1s225ms - Prepare Finished: 207.231ms (207.231ms) - Open Finished: 1s225ms (1s018ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.96 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.43 GB, 137.87 GB, 140.29 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157569597878) - PeakReservation: 146.71 GB (157529669632) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.43 GB (171181419357) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h50m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s018ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 63.990ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s018ms - ExecTreeOpenTime: 952.536ms - PrepareTime: 206.810ms - ExecTreePrepareTime: 145.290us KuduTableSink:(Total: 15.505us, non-child: 15.505us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 952.678ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157529677824) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 32m33s - CumulativeAllocationBytes: 146.71 GB (157529669632) - CumulativeAllocations: 75.12K (75116) - PeakReservation: 146.71 GB (157529669632) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157529669632) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h51m, non-child: 47s700ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s074ms - PeakMemoryUsage: 19.53 MB (20482699) - RowsReturned: 575.34M (575339352) - RowsReturnedRate: 27.31 K/sec Buffer pool: - AllocTime: 2h16m - CumulativeAllocationBytes: 261.28 GB (280546426880) - CumulativeAllocations: 13.70M (13698556) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.92 GB, 37.04 GB, 45.63 GB, 53.22 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.86 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 952.457ms - TotalBytesReceived: 148.31 GB (159245746201) - TotalGetBatchTime: 5h50m - DataArrivalTimer: 5h50m SenderSide: - DeserializeRowBatchTime: 2h27m - NumBatchesArrived: 6.85M (6849278) - NumBatchesDeferred: 49.68K (49682) - NumBatchesEnqueued: 6.85M (6849278) - NumBatchesReceived: 6.85M (6849278) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 64.233ms, non-child: 64.233ms, % non-child: 100.00%) - CodegenTime: 1.614ms - CompileTime: 6.358ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 33.597ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.603ms Instance 844a0200d7876345:20bb38b9000000d6 (host=vd1341.foo.com:22000):(Total: 1s225ms, non-child: 192.352ms, % non-child: 15.70%) Fragment Instance Lifecycle Event Timeline: 1s225ms - Prepare Finished: 191.582ms (191.582ms) - Open Finished: 1s225ms (1s033ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.16 GB, 114.27 GB, 117.22 GB, 120.04 GB, 122.80 GB, 125.42 GB, 127.91 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157571695030) - PeakReservation: 146.71 GB (157531766784) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.47 GB (171226324656) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h55m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 1s033ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 60.991ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s033ms - ExecTreeOpenTime: 972.292ms - PrepareTime: 191.172ms - ExecTreePrepareTime: 157.325us KuduTableSink:(Total: 15.222us, non-child: 15.222us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 972.446ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157531774976) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 29m18s - CumulativeAllocationBytes: 146.71 GB (157531766784) - CumulativeAllocations: 75.12K (75117) - PeakReservation: 146.71 GB (157531766784) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157531766784) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h55m, non-child: 49s192ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s498ms - PeakMemoryUsage: 19.53 MB (20483867) - RowsReturned: 575.35M (575350272) - RowsReturnedRate: 26.96 K/sec Buffer pool: - AllocTime: 3h2m - CumulativeAllocationBytes: 261.28 GB (280551751680) - CumulativeAllocations: 13.70M (13698816) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.08 GB, 112.34 GB, 115.48 GB, 118.45 GB, 121.31 GB, 124.10 GB, 126.74 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 972.180ms - TotalBytesReceived: 148.31 GB (159248758146) - TotalGetBatchTime: 5h55m - DataArrivalTimer: 5h54m SenderSide: - DeserializeRowBatchTime: 3h10m - NumBatchesArrived: 6.85M (6849408) - NumBatchesDeferred: 52.25K (52247) - NumBatchesEnqueued: 6.85M (6849408) - NumBatchesReceived: 6.85M (6849408) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 60.231ms, non-child: 60.231ms, % non-child: 100.00%) - CodegenTime: 1.040ms - CompileTime: 5.740ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.850ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.086ms Instance 844a0200d7876345:20bb38b9000000ef (host=vd1122.foo.com:22000):(Total: 1s223ms, non-child: 239.349ms, % non-child: 19.56%) Fragment Instance Lifecycle Event Timeline: 1s224ms - Prepare Finished: 238.827ms (238.827ms) - Open Finished: 1s224ms (985.471ms) MemoryUsage(8m32s): 20.75 GB, 36.69 GB, 45.18 GB, 52.70 GB, 59.57 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.04 GB, 122.80 GB, 125.42 GB, 127.91 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.72 GB, 145.17 GB, 146.71 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157567500726) - PeakReservation: 146.71 GB (157527572480) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.65 GB (172495600241) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h45m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 985.462ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 61.990ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 985.465ms - ExecTreeOpenTime: 922.011ms - PrepareTime: 238.097ms - ExecTreePrepareTime: 149.535us KuduTableSink:(Total: 16.303us, non-child: 16.303us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 922.158ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157527580672) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 38m5s - CumulativeAllocationBytes: 146.71 GB (157527572480) - CumulativeAllocations: 75.11K (75115) - PeakReservation: 146.71 GB (157527572480) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157527572480) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h46m, non-child: 46s454ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s049ms - PeakMemoryUsage: 19.53 MB (20483654) - RowsReturned: 575.33M (575329776) - RowsReturnedRate: 27.69 K/sec Buffer pool: - AllocTime: 3h32m - CumulativeAllocationBytes: 261.27 GB (280541757440) - CumulativeAllocations: 13.70M (13698328) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.31 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.27 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 921.970ms - TotalBytesReceived: 148.31 GB (159243533710) - TotalGetBatchTime: 5h45m - DataArrivalTimer: 5h45m SenderSide: - DeserializeRowBatchTime: 3h46m - NumBatchesArrived: 6.85M (6849164) - NumBatchesDeferred: 140.09K (140087) - NumBatchesEnqueued: 6.85M (6849164) - NumBatchesReceived: 6.85M (6849164) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.097ms, non-child: 62.097ms, % non-child: 100.00%) - CodegenTime: 1.449ms - CompileTime: 6.145ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.544ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.810ms Instance 844a0200d7876345:20bb38b9000000c5 (host=vd1133.foo.com:22000):(Total: 1s223ms, non-child: 233.572ms, % non-child: 19.09%) Fragment Instance Lifecycle Event Timeline: 1s224ms - Prepare Finished: 233.049ms (233.049ms) - Open Finished: 1s224ms (991.230ms) MemoryUsage(8m32s): 20.71 GB, 36.67 GB, 45.16 GB, 52.68 GB, 59.56 GB, 65.62 GB, 70.86 GB, 75.81 GB, 80.52 GB, 85.14 GB, 89.41 GB, 93.33 GB, 97.15 GB, 100.93 GB, 104.48 GB, 107.95 GB, 111.16 GB, 114.27 GB, 117.21 GB, 120.04 GB, 122.79 GB, 125.41 GB, 127.91 GB, 130.50 GB, 133.00 GB, 135.41 GB, 137.86 GB, 140.28 GB, 142.71 GB, 145.16 GB, 146.70 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157559112118) - PeakReservation: 146.70 GB (157519183872) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.45 GB (171203678675) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h52m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 991.210ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 59.991ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 991.213ms - ExecTreeOpenTime: 930.479ms - PrepareTime: 232.289ms - ExecTreePrepareTime: 200.643us KuduTableSink:(Total: 12.269us, non-child: 12.269us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 930.677ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.70 GB (157519192064) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 30m1s - CumulativeAllocationBytes: 146.70 GB (157519183872) - CumulativeAllocations: 75.11K (75111) - PeakReservation: 146.70 GB (157519183872) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.70 GB (157519183872) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 48s533ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s104ms - PeakMemoryUsage: 19.53 MB (20483657) - RowsReturned: 575.31M (575305668) - RowsReturnedRate: 27.15 K/sec Buffer pool: - AllocTime: 3h14m - CumulativeAllocationBytes: 261.26 GB (280530001920) - CumulativeAllocations: 13.70M (13697754) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.04 GB, 45.62 GB, 53.22 GB, 60.17 GB, 66.30 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.35 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.45 GB, 121.31 GB, 124.10 GB, 126.74 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.85 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.71 GB, 148.26 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB - FirstBatchArrivalWaitTime: 930.403ms - TotalBytesReceived: 148.30 GB (159235912651) - TotalGetBatchTime: 5h52m - DataArrivalTimer: 5h52m SenderSide: - DeserializeRowBatchTime: 3h27m - NumBatchesArrived: 6.85M (6848877) - NumBatchesDeferred: 76.71K (76706) - NumBatchesEnqueued: 6.85M (6848877) - NumBatchesReceived: 6.85M (6848877) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 59.306ms, non-child: 59.306ms, % non-child: 100.00%) - CodegenTime: 1.402ms - CompileTime: 5.828ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 33.306ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.548ms Instance 844a0200d7876345:20bb38b9000000b8 (host=vd1127.foo.com:22000):(Total: 1s221ms, non-child: 220.073ms, % non-child: 18.02%) Fragment Instance Lifecycle Event Timeline: 1s221ms - Prepare Finished: 219.493ms (219.493ms) - Open Finished: 1s221ms (1s002ms) MemoryUsage(8m32s): 20.75 GB, 36.69 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.64 GB, 70.87 GB, 75.81 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.49 GB, 107.96 GB, 111.17 GB, 114.28 GB, 117.23 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157580083638) - PeakReservation: 146.72 GB (157540155392) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.44 GB (171199917754) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h49m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 1s002ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 60.991ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s002ms - ExecTreeOpenTime: 940.021ms - PrepareTime: 218.895ms - ExecTreePrepareTime: 159.811us KuduTableSink:(Total: 12.248us, non-child: 12.248us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 940.178ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157540163584) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 33m12s - CumulativeAllocationBytes: 146.72 GB (157540155392) - CumulativeAllocations: 75.12K (75121) - PeakReservation: 146.72 GB (157540155392) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157540155392) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h50m, non-child: 48s567ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s303ms - PeakMemoryUsage: 19.53 MB (20483103) - RowsReturned: 575.38M (575376816) - RowsReturnedRate: 27.37 K/sec Buffer pool: - AllocTime: 2h39m - CumulativeAllocationBytes: 261.30 GB (280564695040) - CumulativeAllocations: 13.70M (13699448) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.59 GB, 109.09 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.26 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 939.922ms - TotalBytesReceived: 148.32 GB (159256042714) - TotalGetBatchTime: 5h49m - DataArrivalTimer: 5h49m SenderSide: - DeserializeRowBatchTime: 2h50m - NumBatchesArrived: 6.85M (6849724) - NumBatchesDeferred: 82.28K (82276) - NumBatchesEnqueued: 6.85M (6849724) - NumBatchesReceived: 6.85M (6849724) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 61.043ms, non-child: 61.043ms, % non-child: 100.00%) - CodegenTime: 1.242ms - CompileTime: 5.425ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.828ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.155ms Instance 844a0200d7876345:20bb38b9000000d3 (host=vd1328.foo.com:22000):(Total: 1s219ms, non-child: 237.361ms, % non-child: 19.46%) Fragment Instance Lifecycle Event Timeline: 1s220ms - Prepare Finished: 236.537ms (236.537ms) - Open Finished: 1s220ms (983.717ms) MemoryUsage(8m32s): 20.73 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.14 GB, 89.41 GB, 93.33 GB, 97.15 GB, 100.92 GB, 104.48 GB, 107.94 GB, 111.16 GB, 114.27 GB, 117.21 GB, 120.04 GB, 122.79 GB, 125.41 GB, 127.91 GB, 130.50 GB, 133.00 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.72 GB, 145.17 GB, 146.71 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157569597878) - PeakReservation: 146.71 GB (157529669632) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.42 GB (171175729835) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h54m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 983.704ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 59.991ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 983.706ms - ExecTreeOpenTime: 920.142ms - PrepareTime: 236.136ms - ExecTreePrepareTime: 166.283us KuduTableSink:(Total: 11.589us, non-child: 11.589us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 920.305ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157529677824) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 29m29s - CumulativeAllocationBytes: 146.71 GB (157529669632) - CumulativeAllocations: 75.12K (75116) - PeakReservation: 146.71 GB (157529669632) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157529669632) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h55m, non-child: 47s692ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s936ms - PeakMemoryUsage: 19.53 MB (20482654) - RowsReturned: 575.34M (575336580) - RowsReturnedRate: 27.00 K/sec Buffer pool: - AllocTime: 3h10m - CumulativeAllocationBytes: 261.28 GB (280545075200) - CumulativeAllocations: 13.70M (13698490) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.95 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.35 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.58 GB, 109.08 GB, 112.34 GB, 115.48 GB, 118.45 GB, 121.31 GB, 124.10 GB, 126.74 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 920.068ms - TotalBytesReceived: 148.31 GB (159245048268) - TotalGetBatchTime: 5h54m - DataArrivalTimer: 5h54m SenderSide: - DeserializeRowBatchTime: 3h19m - NumBatchesArrived: 6.85M (6849245) - NumBatchesDeferred: 60.28K (60279) - NumBatchesEnqueued: 6.85M (6849245) - NumBatchesReceived: 6.85M (6849245) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.220ms, non-child: 62.220ms, % non-child: 100.00%) - CodegenTime: 1.172ms - CompileTime: 5.031ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.358ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.333ms Instance 844a0200d7876345:20bb38b9000000e6 (host=ve1126.foo.com:22000):(Total: 1s219ms, non-child: 224.713ms, % non-child: 18.43%) Fragment Instance Lifecycle Event Timeline: 1s219ms - Prepare Finished: 223.890ms (223.890ms) - Open Finished: 1s219ms (995.670ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.17 GB, 52.68 GB, 59.56 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.94 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157584277942) - PeakReservation: 146.72 GB (157544349696) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.41 GB (171160891997) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h56m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 995.655ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 59.991ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 995.658ms - ExecTreeOpenTime: 934.512ms - PrepareTime: 223.462ms - ExecTreePrepareTime: 171.335us KuduTableSink:(Total: 13.651us, non-child: 13.651us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 934.681ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157544357888) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 28m12s - CumulativeAllocationBytes: 146.72 GB (157544349696) - CumulativeAllocations: 75.12K (75123) - PeakReservation: 146.72 GB (157544349696) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157544349696) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h56m, non-child: 48s882ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s229ms - PeakMemoryUsage: 19.53 MB (20482370) - RowsReturned: 575.40M (575395044) - RowsReturnedRate: 26.90 K/sec Buffer pool: - AllocTime: 2h7m - CumulativeAllocationBytes: 261.30 GB (280573583360) - CumulativeAllocations: 13.70M (13699882) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 934.444ms - TotalBytesReceived: 148.32 GB (159261099887) - TotalGetBatchTime: 5h55m - DataArrivalTimer: 5h55m SenderSide: - DeserializeRowBatchTime: 2h16m - NumBatchesArrived: 6.85M (6849941) - NumBatchesDeferred: 26.81K (26813) - NumBatchesEnqueued: 6.85M (6849941) - NumBatchesReceived: 6.85M (6849941) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 59.776ms, non-child: 59.776ms, % non-child: 100.00%) - CodegenTime: 1.132ms - CompileTime: 5.957ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.607ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 21.316ms Instance 844a0200d7876345:20bb38b90000008e (host=vd1107.foo.com:22000):(Total: 1s218ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 4h48m - Prepare Finished: 233.783ms (233.783ms) - Open Finished: 1s219ms (985.559ms) - First Batch Produced: 4h48m (4h48m) - First Batch Sent: 4h48m (10.848ms) MemoryUsage(8m32s): 22.80 GB, 40.35 GB, 49.70 GB, 57.96 GB, 65.53 GB, 72.20 GB, 77.96 GB, 83.40 GB, 88.58 GB, 93.67 GB, 98.37 GB, 102.68 GB, 106.88 GB, 111.03 GB, 114.95 GB, 118.75 GB, 122.29 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.09 GB, 137.97 GB, 140.72 GB, 143.57 GB, 146.33 GB, 148.98 GB, 151.67 GB, 154.34 GB, 157.01 GB, 159.71 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.40 GB (188339399324) - RowsProduced: 632.89M (632891204) - TotalNetworkReceiveTime: 3h29m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 985.528ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 52.992ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 4h49m - OpenTime: 985.532ms - ExecTreeOpenTime: 928.739ms - PrepareTime: 233.331ms - ExecTreePrepareTime: 156.815us KuduTableSink:(Total: 1h48m, non-child: 1h48m, % non-child: 100.00%) - KuduApplyTimer: 1h28m - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 97.57 K/sec - TotalNumRows: 632.89M (632891204) SORT_NODE (id=2):(Total: 4h49m, non-child: 1h19m, % non-child: 27.45%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 31m34s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 632.89M (632891204) - RowsReturnedRate: 36.49 K/sec - RunsCreated: 2 (2) - SortDataSize: 161.38 GB (173276515440) Buffer pool: - AllocTime: 34m17s - CumulativeAllocationBytes: 161.40 GB (173306544128) - CumulativeAllocations: 82.64K (82639) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 3h29m, non-child: 51s420ms, % non-child: 0.41%) - ConvertRowBatchTime: 21s166ms - PeakMemoryUsage: 19.53 MB (20481898) - RowsReturned: 632.95M (632953356) - RowsReturnedRate: 50.30 K/sec Buffer pool: - AllocTime: 2h45m - CumulativeAllocationBytes: 287.44 GB (308640112640) - CumulativeAllocations: 15.07M (15070318) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.05 GB, 40.76 GB, 50.21 GB, 58.56 GB, 66.21 GB, 72.95 GB, 78.78 GB, 84.27 GB, 89.51 GB, 94.66 GB, 99.41 GB, 103.76 GB, 108.01 GB, 112.21 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.32 GB, 133.46 GB, 136.53 GB, 139.44 GB, 142.22 GB, 145.10 GB, 147.88 GB, 150.57 GB, 153.29 GB, 155.98 GB, 158.69 GB, 161.42 GB, 163.12 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB - FirstBatchArrivalWaitTime: 928.640ms - TotalBytesReceived: 163.16 GB (175192390528) - TotalGetBatchTime: 3h29m - DataArrivalTimer: 3h28m SenderSide: - DeserializeRowBatchTime: 2h56m - NumBatchesArrived: 7.54M (7535159) - NumBatchesDeferred: 218.72K (218715) - NumBatchesEnqueued: 7.54M (7535159) - NumBatchesReceived: 7.54M (7535159) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 55.414ms, non-child: 55.414ms, % non-child: 100.00%) - CodegenTime: 1.234ms - CompileTime: 5.395ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 30.342ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.061ms Instance 844a0200d7876345:20bb38b9000000f9 (host=va1041.foo.com:22000):(Total: 1s218ms, non-child: 213.986ms, % non-child: 17.56%) Fragment Instance Lifecycle Event Timeline: 1s218ms - Prepare Finished: 213.150ms (213.150ms) - Open Finished: 1s218ms (1s005ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.91 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157573792182) - PeakReservation: 146.71 GB (157533863936) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.70 GB (172551290778) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h56m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 1s005ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 59.991ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 1s005ms - ExecTreeOpenTime: 945.705ms - PrepareTime: 212.730ms - ExecTreePrepareTime: 147.106us KuduTableSink:(Total: 15.398us, non-child: 15.398us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 945.849ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157533872128) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 27m35s - CumulativeAllocationBytes: 146.71 GB (157533863936) - CumulativeAllocations: 75.12K (75118) - PeakReservation: 146.71 GB (157533863936) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157533863936) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h56m, non-child: 47s916ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s944ms - PeakMemoryUsage: 19.53 MB (20483045) - RowsReturned: 575.35M (575354724) - RowsReturnedRate: 26.89 K/sec Buffer pool: - AllocTime: 2h35m - CumulativeAllocationBytes: 261.29 GB (280553922560) - CumulativeAllocations: 13.70M (13698922) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.25 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 945.641ms - TotalBytesReceived: 148.31 GB (159249728330) - TotalGetBatchTime: 5h56m - DataArrivalTimer: 5h55m SenderSide: - DeserializeRowBatchTime: 2h43m - NumBatchesArrived: 6.85M (6849461) - NumBatchesDeferred: 30.30K (30299) - NumBatchesEnqueued: 6.85M (6849461) - NumBatchesReceived: 6.85M (6849461) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 58.631ms, non-child: 58.631ms, % non-child: 100.00%) - CodegenTime: 1.471ms - CompileTime: 5.642ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.667ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.547ms Instance 844a0200d7876345:20bb38b9000000f3 (host=ve1131.foo.com:22000):(Total: 1s218ms, non-child: 246.000ms, % non-child: 20.19%) Fragment Instance Lifecycle Event Timeline: 1s218ms - Prepare Finished: 245.338ms (245.338ms) - Open Finished: 1s218ms (973.488ms) MemoryUsage(8m32s): 20.73 GB, 36.68 GB, 45.17 GB, 52.69 GB, 59.56 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.41 GB, 93.33 GB, 97.15 GB, 100.92 GB, 104.48 GB, 107.94 GB, 111.16 GB, 114.27 GB, 117.21 GB, 120.04 GB, 122.80 GB, 125.42 GB, 127.91 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157565403574) - PeakReservation: 146.71 GB (157525475328) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.41 GB (171161855986) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 6h - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 973.471ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 49.992ms - TotalThreadsVoluntaryContextSwitches: 5 (5) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 973.475ms - ExecTreeOpenTime: 923.428ms - PrepareTime: 244.886ms - ExecTreePrepareTime: 173.836us KuduTableSink:(Total: 14.079us, non-child: 14.079us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 923.598ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157525483520) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 24m59s - CumulativeAllocationBytes: 146.71 GB (157525475328) - CumulativeAllocations: 75.11K (75114) - PeakReservation: 146.71 GB (157525475328) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157525475328) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 6h, non-child: 49s379ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s873ms - PeakMemoryUsage: 19.53 MB (20481035) - RowsReturned: 575.33M (575332968) - RowsReturnedRate: 26.59 K/sec Buffer pool: - AllocTime: 1h43m - CumulativeAllocationBytes: 261.28 GB (280543313920) - CumulativeAllocations: 13.70M (13698404) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.92 GB, 37.04 GB, 45.62 GB, 53.22 GB, 60.17 GB, 66.30 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.03 GB, 90.35 GB, 94.31 GB, 98.17 GB, 101.98 GB, 105.58 GB, 109.08 GB, 112.33 GB, 115.47 GB, 118.45 GB, 121.31 GB, 124.10 GB, 126.74 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.34 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.27 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 923.339ms - TotalBytesReceived: 148.31 GB (159243333600) - TotalGetBatchTime: 6h - DataArrivalTimer: 5h59m SenderSide: - DeserializeRowBatchTime: 1h52m - NumBatchesArrived: 6.85M (6849202) - NumBatchesDeferred: 9.98K (9978) - NumBatchesEnqueued: 6.85M (6849202) - NumBatchesReceived: 6.85M (6849202) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 48.814ms, non-child: 48.814ms, % non-child: 100.00%) - CodegenTime: 1.084ms - CompileTime: 5.359ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 26.981ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 15.981ms Instance 844a0200d7876345:20bb38b900000099 (host=vd1101.foo.com:22000):(Total: 1s216ms, non-child: 234.721ms, % non-child: 19.29%) Fragment Instance Lifecycle Event Timeline: 1s216ms - Prepare Finished: 233.815ms (233.815ms) - Open Finished: 1s216ms (983.052ms) MemoryUsage(8m32s): 22.79 GB, 40.34 GB, 49.69 GB, 57.95 GB, 65.52 GB, 72.19 GB, 77.95 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.36 GB, 102.66 GB, 106.87 GB, 111.01 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.69 GB, 128.93 GB, 132.04 GB, 135.07 GB, 137.95 GB, 140.70 GB, 143.55 GB, 146.31 GB, 148.96 GB, 151.65 GB, 154.32 GB, 156.99 GB, 159.69 GB, 161.37 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173321306550) - PeakReservation: 161.38 GB (173281378304) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.27 GB (188192130616) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h50m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 9 (9) - TotalThreadsTotalWallClockTime: 983.035ms - TotalThreadsSysTime: 3.999ms - TotalThreadsUserTime: 58.991ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 983.038ms - ExecTreeOpenTime: 919.418ms - PrepareTime: 233.481ms - ExecTreePrepareTime: 141.339us KuduTableSink:(Total: 12.028us, non-child: 12.028us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 919.557ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173281386496) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 31m17s - CumulativeAllocationBytes: 161.38 GB (173281378304) - CumulativeAllocations: 82.63K (82627) - PeakReservation: 161.38 GB (173281378304) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173281378304) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h51m, non-child: 50s512ms, % non-child: 0.24%) - ConvertRowBatchTime: 21s162ms - PeakMemoryUsage: 19.53 MB (20481798) - RowsReturned: 632.88M (632876076) - RowsReturnedRate: 30.05 K/sec Buffer pool: - AllocTime: 2h33m - CumulativeAllocationBytes: 287.41 GB (308602429440) - CumulativeAllocations: 15.07M (15068478) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.04 GB, 40.75 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.65 GB, 99.39 GB, 103.74 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.57 GB, 127.03 GB, 130.30 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.08 GB, 147.87 GB, 150.55 GB, 153.27 GB, 155.96 GB, 158.67 GB, 161.39 GB, 163.09 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 919.306ms - TotalBytesReceived: 163.14 GB (175170429745) - TotalGetBatchTime: 5h50m - DataArrivalTimer: 5h50m SenderSide: - DeserializeRowBatchTime: 2h45m - NumBatchesArrived: 7.53M (7534239) - NumBatchesDeferred: 60.60K (60602) - NumBatchesEnqueued: 7.53M (7534239) - NumBatchesReceived: 7.53M (7534239) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.296ms, non-child: 62.296ms, % non-child: 100.00%) - CodegenTime: 1.062ms - CompileTime: 5.406ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 29.805ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 26.480ms Instance 844a0200d7876345:20bb38b90000008d (host=vd1116.foo.com:22000):(Total: 1s215ms, non-child: 225.826ms, % non-child: 18.57%) Fragment Instance Lifecycle Event Timeline: 1s216ms - Prepare Finished: 224.981ms (224.981ms) - Open Finished: 1s216ms (991.236ms) MemoryUsage(8m32s): 22.79 GB, 40.33 GB, 49.68 GB, 57.94 GB, 65.51 GB, 72.18 GB, 77.94 GB, 83.37 GB, 88.55 GB, 93.64 GB, 98.34 GB, 102.64 GB, 106.85 GB, 111.00 GB, 114.91 GB, 118.72 GB, 122.26 GB, 125.67 GB, 128.91 GB, 132.03 GB, 135.06 GB, 137.94 GB, 140.68 GB, 143.54 GB, 146.29 GB, 148.94 GB, 151.63 GB, 154.29 GB, 156.97 GB, 159.67 GB, 161.35 GB, 161.38 GB, 161.38 GB, 161.38 GB, 161.38 GB, 161.38 GB, 161.38 GB, 161.38 GB, 161.38 GB, 161.38 GB, 161.38 GB, 161.38 GB, 161.38 GB, 161.38 GB, 161.38 GB, 161.38 GB, 161.38 GB, 161.39 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.40 GB (173298237878) - PeakReservation: 161.36 GB (173258309632) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.39 GB (188326743535) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h39m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 991.209ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 62.990ms - TotalThreadsVoluntaryContextSwitches: 10 (10) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 991.224ms - ExecTreeOpenTime: 927.736ms - PrepareTime: 224.582ms - ExecTreePrepareTime: 157.907us KuduTableSink:(Total: 11.437us, non-child: 11.437us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 927.891ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.36 GB (173258317824) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 43m25s - CumulativeAllocationBytes: 161.36 GB (173258309632) - CumulativeAllocations: 82.62K (82616) - PeakReservation: 161.36 GB (173258309632) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.36 GB (173258309632) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h40m, non-child: 49s297ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s350ms - PeakMemoryUsage: 19.53 MB (20482152) - RowsReturned: 632.79M (632786784) - RowsReturnedRate: 31.00 K/sec Buffer pool: - AllocTime: 5h4m - CumulativeAllocationBytes: 287.37 GB (308558888960) - CumulativeAllocations: 15.07M (15066352) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.74 GB, 50.19 GB, 58.54 GB, 66.19 GB, 72.93 GB, 78.76 GB, 84.25 GB, 89.49 GB, 94.63 GB, 99.38 GB, 103.73 GB, 107.98 GB, 112.17 GB, 116.13 GB, 119.98 GB, 123.56 GB, 127.01 GB, 130.28 GB, 133.43 GB, 136.49 GB, 139.40 GB, 142.18 GB, 145.07 GB, 147.85 GB, 150.53 GB, 153.25 GB, 155.94 GB, 158.65 GB, 161.37 GB, 163.07 GB, 163.10 GB, 163.10 GB, 163.10 GB, 163.10 GB, 163.10 GB, 163.10 GB, 163.10 GB, 163.10 GB, 163.10 GB, 163.10 GB, 163.10 GB, 163.10 GB, 163.10 GB, 163.10 GB, 163.10 GB, 163.10 GB, 163.11 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB - FirstBatchArrivalWaitTime: 927.645ms - TotalBytesReceived: 163.12 GB (175145862439) - TotalGetBatchTime: 5h39m - DataArrivalTimer: 5h39m SenderSide: - DeserializeRowBatchTime: 5h17m - NumBatchesArrived: 7.53M (7533176) - NumBatchesDeferred: 259.12K (259123) - NumBatchesEnqueued: 7.53M (7533176) - NumBatchesReceived: 7.53M (7533176) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.159ms, non-child: 62.159ms, % non-child: 100.00%) - CodegenTime: 1.032ms - CompileTime: 5.763ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 34.630ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 21.112ms Instance 844a0200d7876345:20bb38b9000000d8 (host=vc1540.foo.com:22000):(Total: 1s215ms, non-child: 222.983ms, % non-child: 18.35%) Fragment Instance Lifecycle Event Timeline: 1s215ms - Prepare Finished: 222.156ms (222.156ms) - Open Finished: 1s215ms (993.508ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.17 GB, 52.69 GB, 59.56 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157571695030) - PeakReservation: 146.71 GB (157531766784) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.69 GB (172535195133) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 6h - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 993.491ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 53.992ms - TotalThreadsVoluntaryContextSwitches: 26 (26) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 993.495ms - ExecTreeOpenTime: 938.361ms - PrepareTime: 221.746ms - ExecTreePrepareTime: 169.535us KuduTableSink:(Total: 12.004us, non-child: 12.004us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 938.528ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157531774976) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 23m51s - CumulativeAllocationBytes: 146.71 GB (157531766784) - CumulativeAllocations: 75.12K (75117) - PeakReservation: 146.71 GB (157531766784) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157531766784) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 6h1m, non-child: 45s591ms, % non-child: 0.21%) - ConvertRowBatchTime: 19s307ms - PeakMemoryUsage: 19.53 MB (20482591) - RowsReturned: 575.35M (575349180) - RowsReturnedRate: 26.54 K/sec Buffer pool: - AllocTime: 2h32m - CumulativeAllocationBytes: 261.28 GB (280551219200) - CumulativeAllocations: 13.70M (13698790) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.03 GB, 45.62 GB, 53.22 GB, 60.17 GB, 66.30 GB, 71.61 GB, 76.60 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.59 GB, 109.09 GB, 112.35 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 938.283ms - TotalBytesReceived: 148.31 GB (159248255058) - TotalGetBatchTime: 6h - DataArrivalTimer: 6h SenderSide: - DeserializeRowBatchTime: 2h44m - NumBatchesArrived: 6.85M (6849395) - NumBatchesDeferred: 20.77K (20769) - NumBatchesEnqueued: 6.85M (6849395) - NumBatchesReceived: 6.85M (6849395) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 53.773ms, non-child: 53.773ms, % non-child: 100.00%) - CodegenTime: 969.541us - CompileTime: 5.607ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 25.364ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.332ms Instance 844a0200d7876345:20bb38b9000000b5 (host=vc1528.foo.com:22000):(Total: 1s212ms, non-child: 235.879ms, % non-child: 19.45%) Fragment Instance Lifecycle Event Timeline: 1s213ms - Prepare Finished: 235.188ms (235.188ms) - Open Finished: 1s213ms (978.010ms) MemoryUsage(8m32s): 20.75 GB, 36.69 GB, 45.18 GB, 52.70 GB, 59.57 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.35 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.28 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.43 GB, 127.93 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157577986486) - PeakReservation: 146.72 GB (157538058240) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.63 GB (172475100443) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h57m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 977.989ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 51.992ms - TotalThreadsVoluntaryContextSwitches: 15 (15) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 977.993ms - ExecTreeOpenTime: 925.724ms - PrepareTime: 234.749ms - ExecTreePrepareTime: 160.724us KuduTableSink:(Total: 12.669us, non-child: 12.669us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 925.882ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157538066432) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 26m47s - CumulativeAllocationBytes: 146.72 GB (157538058240) - CumulativeAllocations: 75.12K (75120) - PeakReservation: 146.72 GB (157538058240) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157538058240) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h58m, non-child: 49s221ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s495ms - PeakMemoryUsage: 19.53 MB (20483126) - RowsReturned: 575.37M (575369004) - RowsReturnedRate: 26.76 K/sec Buffer pool: - AllocTime: 1h55m - CumulativeAllocationBytes: 261.29 GB (280560885760) - CumulativeAllocations: 13.70M (13699262) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.76 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.28 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 925.654ms - TotalBytesReceived: 148.32 GB (159254202131) - TotalGetBatchTime: 5h57m - DataArrivalTimer: 5h57m SenderSide: - DeserializeRowBatchTime: 2h5m - NumBatchesArrived: 6.85M (6849631) - NumBatchesDeferred: 11.11K (11106) - NumBatchesEnqueued: 6.85M (6849631) - NumBatchesReceived: 6.85M (6849631) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 51.023ms, non-child: 51.023ms, % non-child: 100.00%) - CodegenTime: 992.653us - CompileTime: 5.178ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 26.147ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.141ms Instance 844a0200d7876345:20bb38b900000093 (host=vd1335.foo.com:22000):(Total: 1s212ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h15m - Prepare Finished: 220.712ms (220.712ms) - Open Finished: 1s213ms (992.623ms) - First Batch Produced: 7h15m (7h15m) - First Batch Sent: 7h15m (11.625ms) MemoryUsage(8m32s): 22.82 GB, 40.35 GB, 49.69 GB, 57.96 GB, 65.53 GB, 72.19 GB, 77.96 GB, 83.39 GB, 88.58 GB, 93.67 GB, 98.36 GB, 102.67 GB, 106.88 GB, 111.03 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.09 GB, 137.97 GB, 140.71 GB, 143.57 GB, 146.32 GB, 148.97 GB, 151.66 GB, 154.33 GB, 157.01 GB, 159.70 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325747015) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.06 GB (186898093888) - RowsProduced: 226.95M (226945046) - TotalNetworkReceiveTime: 5h48m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 992.608ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 53.991ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h15m - OpenTime: 992.611ms - ExecTreeOpenTime: 934.242ms - PrepareTime: 219.965ms - ExecTreePrepareTime: 157.904us KuduTableSink:(Total: 45m37s, non-child: 45m37s, % non-child: 100.00%) - KuduApplyTimer: 37m43s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 82.90 K/sec - TotalNumRows: 226.94M (226944022) SORT_NODE (id=2):(Total: 7h15m, non-child: 1h26m, % non-child: 19.82%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 32m16s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 226.95M (226945046) - RowsReturnedRate: 8.68 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173275601615) Buffer pool: - AllocTime: 34m7s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h49m, non-child: 52s678ms, % non-child: 0.25%) - ConvertRowBatchTime: 21s717ms - PeakMemoryUsage: 19.53 MB (20481823) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.19 K/sec Buffer pool: - AllocTime: 3h58m - CumulativeAllocationBytes: 287.43 GB (308628398080) - CumulativeAllocations: 15.07M (15069746) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.75 GB, 50.19 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.65 GB, 99.40 GB, 103.75 GB, 108.01 GB, 112.20 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.31 GB, 133.46 GB, 136.52 GB, 139.43 GB, 142.21 GB, 145.09 GB, 147.88 GB, 150.56 GB, 153.28 GB, 155.97 GB, 158.68 GB, 161.41 GB, 163.11 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 934.157ms - TotalBytesReceived: 163.15 GB (175184825579) - TotalGetBatchTime: 5h48m - DataArrivalTimer: 5h48m SenderSide: - DeserializeRowBatchTime: 4h10m - NumBatchesArrived: 7.53M (7534893) - NumBatchesDeferred: 108.50K (108504) - NumBatchesEnqueued: 7.53M (7534873) - NumBatchesReceived: 7.53M (7534873) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.957ms, non-child: 56.957ms, % non-child: 100.00%) - CodegenTime: 1.346ms - CompileTime: 5.489ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.979ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 18.940ms Instance 844a0200d7876345:20bb38b9000000b9 (host=vd1108.foo.com:22000):(Total: 1s212ms, non-child: 225.516ms, % non-child: 18.60%) Fragment Instance Lifecycle Event Timeline: 1s212ms - Prepare Finished: 224.856ms (224.856ms) - Open Finished: 1s212ms (988.043ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.17 GB, 52.68 GB, 59.56 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157569597878) - PeakReservation: 146.71 GB (157529669632) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.68 GB (172532104702) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h53m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 988.033ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 57.991ms - TotalThreadsVoluntaryContextSwitches: 8 (8) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 988.035ms - ExecTreeOpenTime: 927.989ms - PrepareTime: 224.469ms - ExecTreePrepareTime: 177.790us KuduTableSink:(Total: 13.010us, non-child: 13.010us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 928.164ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157529677824) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 31m20s - CumulativeAllocationBytes: 146.71 GB (157529669632) - CumulativeAllocations: 75.12K (75116) - PeakReservation: 146.71 GB (157529669632) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157529669632) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 48s231ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s845ms - PeakMemoryUsage: 19.54 MB (20484258) - RowsReturned: 575.34M (575339940) - RowsReturnedRate: 27.10 K/sec Buffer pool: - AllocTime: 3h32m - CumulativeAllocationBytes: 261.28 GB (280546713600) - CumulativeAllocations: 13.70M (13698570) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.62 GB, 53.22 GB, 60.17 GB, 66.30 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.35 GB, 94.31 GB, 98.18 GB, 102.00 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.34 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 927.930ms - TotalBytesReceived: 148.31 GB (159245893110) - TotalGetBatchTime: 5h53m - DataArrivalTimer: 5h53m SenderSide: - DeserializeRowBatchTime: 3h40m - NumBatchesArrived: 6.85M (6849285) - NumBatchesDeferred: 78.99K (78994) - NumBatchesEnqueued: 6.85M (6849285) - NumBatchesReceived: 6.85M (6849285) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 58.880ms, non-child: 58.880ms, % non-child: 100.00%) - CodegenTime: 1.219ms - CompileTime: 5.293ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.301ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 20.766ms Instance 844a0200d7876345:20bb38b9000000f0 (host=vc1520.foo.com:22000):(Total: 1s211ms, non-child: 230.157ms, % non-child: 18.99%) Fragment Instance Lifecycle Event Timeline: 1s212ms - Prepare Finished: 229.327ms (229.327ms) - Open Finished: 1s212ms (982.911ms) MemoryUsage(8m32s): 20.72 GB, 36.67 GB, 45.17 GB, 52.68 GB, 59.56 GB, 65.62 GB, 70.86 GB, 75.81 GB, 80.52 GB, 85.14 GB, 89.41 GB, 93.33 GB, 97.15 GB, 100.92 GB, 104.48 GB, 107.94 GB, 111.16 GB, 114.27 GB, 117.21 GB, 120.04 GB, 122.80 GB, 125.41 GB, 127.91 GB, 130.50 GB, 133.00 GB, 135.42 GB, 137.86 GB, 140.28 GB, 142.72 GB, 145.17 GB, 146.70 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157563306422) - PeakReservation: 146.71 GB (157523378176) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.59 GB (172429530616) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h58m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 9 (9) - TotalThreadsTotalWallClockTime: 982.891ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 57.991ms - TotalThreadsVoluntaryContextSwitches: 9 (9) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 982.894ms - ExecTreeOpenTime: 923.203ms - PrepareTime: 228.926ms - ExecTreePrepareTime: 203.824us KuduTableSink:(Total: 16.434us, non-child: 16.434us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 923.404ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157523386368) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 27m3s - CumulativeAllocationBytes: 146.71 GB (157523378176) - CumulativeAllocations: 75.11K (75113) - PeakReservation: 146.71 GB (157523378176) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157523378176) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h58m, non-child: 48s266ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s720ms - PeakMemoryUsage: 19.54 MB (20484322) - RowsReturned: 575.32M (575315664) - RowsReturnedRate: 26.73 K/sec Buffer pool: - AllocTime: 2h26m - CumulativeAllocationBytes: 261.27 GB (280534876160) - CumulativeAllocations: 13.70M (13697992) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.63 GB, 53.22 GB, 60.17 GB, 66.30 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.03 GB, 90.35 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.58 GB, 109.08 GB, 112.34 GB, 115.48 GB, 118.45 GB, 121.31 GB, 124.10 GB, 126.74 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.71 GB, 148.26 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB - FirstBatchArrivalWaitTime: 923.123ms - TotalBytesReceived: 148.30 GB (159238991257) - TotalGetBatchTime: 5h58m - DataArrivalTimer: 5h57m SenderSide: - DeserializeRowBatchTime: 2h35m - NumBatchesArrived: 6.85M (6848996) - NumBatchesDeferred: 61.24K (61237) - NumBatchesEnqueued: 6.85M (6848996) - NumBatchesReceived: 6.85M (6848996) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 58.310ms, non-child: 58.310ms, % non-child: 100.00%) - CodegenTime: 1.112ms - CompileTime: 4.947ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 29.830ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.002ms Instance 844a0200d7876345:20bb38b9000000f8 (host=vd1114.foo.com:22000):(Total: 1s211ms, non-child: 213.455ms, % non-child: 17.61%) Fragment Instance Lifecycle Event Timeline: 1s212ms - Prepare Finished: 212.691ms (212.691ms) - Open Finished: 1s212ms (999.470ms) MemoryUsage(8m32s): 20.73 GB, 36.69 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.49 GB, 107.96 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157584277942) - PeakReservation: 146.72 GB (157544349696) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.72 GB (172570493482) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h51m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 999.443ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 54.992ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 999.445ms - ExecTreeOpenTime: 944.509ms - PrepareTime: 212.290ms - ExecTreePrepareTime: 171.589us KuduTableSink:(Total: 8.038us, non-child: 8.038us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 944.679ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157544357888) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 31m14s - CumulativeAllocationBytes: 146.72 GB (157544349696) - CumulativeAllocations: 75.12K (75123) - PeakReservation: 146.72 GB (157544349696) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157544349696) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h52m, non-child: 47s890ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s106ms - PeakMemoryUsage: 19.53 MB (20483607) - RowsReturned: 575.40M (575396472) - RowsReturnedRate: 27.22 K/sec Buffer pool: - AllocTime: 3h53m - CumulativeAllocationBytes: 261.31 GB (280574279680) - CumulativeAllocations: 13.70M (13699916) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.59 GB, 109.09 GB, 112.35 GB, 115.49 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 944.426ms - TotalBytesReceived: 148.32 GB (159261665948) - TotalGetBatchTime: 5h51m - DataArrivalTimer: 5h51m SenderSide: - DeserializeRowBatchTime: 4h7m - NumBatchesArrived: 6.85M (6849958) - NumBatchesDeferred: 101.03K (101033) - NumBatchesEnqueued: 6.85M (6849958) - NumBatchesReceived: 6.85M (6849958) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 53.655ms, non-child: 53.655ms, % non-child: 100.00%) - CodegenTime: 972.295us - CompileTime: 5.174ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.073ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.902ms Instance 844a0200d7876345:20bb38b90000009b (host=vc1534.foo.com:22000):(Total: 1s210ms, non-child: 234.561ms, % non-child: 19.38%) Fragment Instance Lifecycle Event Timeline: 1s210ms - Prepare Finished: 233.775ms (233.775ms) - Open Finished: 1s210ms (976.679ms) MemoryUsage(8m32s): 22.79 GB, 40.33 GB, 49.68 GB, 57.94 GB, 65.51 GB, 72.18 GB, 77.95 GB, 83.38 GB, 88.57 GB, 93.66 GB, 98.36 GB, 102.66 GB, 106.87 GB, 111.02 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.69 GB, 128.93 GB, 132.04 GB, 135.07 GB, 137.95 GB, 140.70 GB, 143.56 GB, 146.31 GB, 148.96 GB, 151.65 GB, 154.32 GB, 157.00 GB, 159.69 GB, 161.37 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173323403702) - PeakReservation: 161.38 GB (173283475456) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.18 GB (188095084451) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h49m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 976.672ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 49.992ms - TotalThreadsVoluntaryContextSwitches: 12 (12) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 976.675ms - ExecTreeOpenTime: 925.665ms - PrepareTime: 233.441ms - ExecTreePrepareTime: 222.571us KuduTableSink:(Total: 12.599us, non-child: 12.599us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 925.885ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173283483648) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 31m53s - CumulativeAllocationBytes: 161.38 GB (173283475456) - CumulativeAllocations: 82.63K (82628) - PeakReservation: 161.38 GB (173283475456) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173283475456) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h49m, non-child: 51s465ms, % non-child: 0.25%) - ConvertRowBatchTime: 21s090ms - PeakMemoryUsage: 19.53 MB (20481375) - RowsReturned: 632.88M (632879100) - RowsReturnedRate: 30.17 K/sec Buffer pool: - AllocTime: 3h51m - CumulativeAllocationBytes: 287.41 GB (308603904000) - CumulativeAllocations: 15.07M (15068550) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.74 GB, 50.19 GB, 58.54 GB, 66.19 GB, 72.94 GB, 78.76 GB, 84.26 GB, 89.50 GB, 94.64 GB, 99.39 GB, 103.75 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.03 GB, 130.30 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.08 GB, 147.87 GB, 150.55 GB, 153.27 GB, 155.97 GB, 158.67 GB, 161.40 GB, 163.10 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 925.600ms - TotalBytesReceived: 163.14 GB (175171664528) - TotalGetBatchTime: 5h49m - DataArrivalTimer: 5h48m SenderSide: - DeserializeRowBatchTime: 4h1m - NumBatchesArrived: 7.53M (7534275) - NumBatchesDeferred: 123.42K (123417) - NumBatchesEnqueued: 7.53M (7534275) - NumBatchesReceived: 7.53M (7534275) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 49.718ms, non-child: 49.718ms, % non-child: 100.00%) - CodegenTime: 995.992us - CompileTime: 5.187ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 26.962ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 17.028ms Instance 844a0200d7876345:20bb38b9000000bd (host=vc1504.foo.com:22000):(Total: 1s209ms, non-child: 222.282ms, % non-child: 18.37%) Fragment Instance Lifecycle Event Timeline: 1s210ms - Prepare Finished: 221.505ms (221.505ms) - Open Finished: 1s210ms (988.718ms) MemoryUsage(8m32s): 20.75 GB, 36.69 GB, 45.19 GB, 52.70 GB, 59.58 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.43 GB, 137.87 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157575889334) - PeakReservation: 146.72 GB (157535961088) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.71 GB (172563104240) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h51m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 988.700ms - TotalThreadsSysTime: 3.999ms - TotalThreadsUserTime: 49.993ms - TotalThreadsVoluntaryContextSwitches: 15 (15) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 988.705ms - ExecTreeOpenTime: 920.200ms - PrepareTime: 221.128ms - ExecTreePrepareTime: 201.751us KuduTableSink:(Total: 15.315us, non-child: 15.315us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 920.398ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157535969280) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 31m55s - CumulativeAllocationBytes: 146.72 GB (157535961088) - CumulativeAllocations: 75.12K (75119) - PeakReservation: 146.72 GB (157535961088) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157535961088) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h51m, non-child: 47s724ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s932ms - PeakMemoryUsage: 19.53 MB (20483705) - RowsReturned: 575.36M (575364804) - RowsReturnedRate: 27.25 K/sec Buffer pool: - AllocTime: 4h4m - CumulativeAllocationBytes: 261.29 GB (280558837760) - CumulativeAllocations: 13.70M (13699162) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.19 GB, 66.32 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.27 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 920.100ms - TotalBytesReceived: 148.32 GB (159252780783) - TotalGetBatchTime: 5h51m - DataArrivalTimer: 5h51m SenderSide: - DeserializeRowBatchTime: 4h12m - NumBatchesArrived: 6.85M (6849581) - NumBatchesDeferred: 88.64K (88644) - NumBatchesEnqueued: 6.85M (6849581) - NumBatchesReceived: 6.85M (6849581) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 67.199ms, non-child: 67.199ms, % non-child: 100.00%) - CodegenTime: 848.001us - CompileTime: 4.733ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 23.782ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 38.239ms Instance 844a0200d7876345:20bb38b9000000a1 (host=vc1505.foo.com:22000):(Total: 1s209ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h15m - Prepare Finished: 226.474ms (226.474ms) - Open Finished: 1s209ms (982.923ms) - First Batch Produced: 7h15m (7h15m) - First Batch Sent: 7h15m (7.897ms) MemoryUsage(8m32s): 22.79 GB, 40.35 GB, 49.70 GB, 57.96 GB, 65.53 GB, 72.20 GB, 77.97 GB, 83.40 GB, 88.58 GB, 93.67 GB, 98.37 GB, 102.67 GB, 106.88 GB, 111.03 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.70 GB, 128.94 GB, 132.06 GB, 135.09 GB, 137.97 GB, 140.71 GB, 143.57 GB, 146.32 GB, 148.97 GB, 151.66 GB, 154.33 GB, 157.01 GB, 159.70 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.41 GB (188345462065) - RowsProduced: 237.49M (237490512) - TotalNetworkReceiveTime: 5h52m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 982.907ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 60.991ms - TotalThreadsVoluntaryContextSwitches: 16 (16) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h15m - OpenTime: 982.910ms - ExecTreeOpenTime: 920.960ms - PrepareTime: 226.041ms - ExecTreePrepareTime: 165.799us KuduTableSink:(Total: 45m17s, non-child: 45m17s, % non-child: 100.00%) - KuduApplyTimer: 36m52s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 87.38 K/sec - TotalNumRows: 237.49M (237489488) SORT_NODE (id=2):(Total: 7h16m, non-child: 1h23m, % non-child: 19.15%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 32m26s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 237.49M (237490512) - RowsReturnedRate: 9.08 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276328865) Buffer pool: - AllocTime: 30m48s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h52m, non-child: 50s244ms, % non-child: 0.24%) - ConvertRowBatchTime: 21s079ms - PeakMemoryUsage: 19.53 MB (20481701) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 29.92 K/sec Buffer pool: - AllocTime: 2h38m - CumulativeAllocationBytes: 287.43 GB (308626636800) - CumulativeAllocations: 15.07M (15069660) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.76 GB, 50.21 GB, 58.56 GB, 66.21 GB, 72.96 GB, 78.78 GB, 84.28 GB, 89.52 GB, 94.66 GB, 99.40 GB, 103.76 GB, 108.01 GB, 112.20 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.31 GB, 133.46 GB, 136.52 GB, 139.43 GB, 142.21 GB, 145.10 GB, 147.88 GB, 150.56 GB, 153.28 GB, 155.98 GB, 158.68 GB, 161.41 GB, 163.11 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 920.868ms - TotalBytesReceived: 163.15 GB (175184557102) - TotalGetBatchTime: 5h52m - DataArrivalTimer: 5h51m SenderSide: - DeserializeRowBatchTime: 2h50m - NumBatchesArrived: 7.53M (7534830) - NumBatchesDeferred: 83.32K (83322) - NumBatchesEnqueued: 7.53M (7534830) - NumBatchesReceived: 7.53M (7534830) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 60.429ms, non-child: 60.429ms, % non-child: 100.00%) - CodegenTime: 1.628ms - CompileTime: 4.838ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.549ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.402ms Instance 844a0200d7876345:20bb38b90000008a (host=va1028.foo.com:22000):(Total: 1s207ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h25m - Prepare Finished: 230.111ms (230.111ms) - Open Finished: 1s208ms (977.944ms) - First Batch Produced: 7h25m (7h25m) - First Batch Sent: 7h25m (9.232ms) MemoryUsage(8m32s): 22.82 GB, 40.35 GB, 49.69 GB, 57.96 GB, 65.52 GB, 72.20 GB, 77.96 GB, 83.40 GB, 88.58 GB, 93.67 GB, 98.37 GB, 102.68 GB, 106.89 GB, 111.03 GB, 114.95 GB, 118.76 GB, 122.30 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.09 GB, 137.97 GB, 140.72 GB, 143.57 GB, 146.32 GB, 148.98 GB, 151.67 GB, 154.33 GB, 157.01 GB, 159.70 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173326141821) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.25 GB (188172965432) - RowsProduced: 195.49M (195486044) - TotalNetworkReceiveTime: 5h45m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 977.929ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 60.991ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h25m - OpenTime: 977.932ms - ExecTreeOpenTime: 914.039ms - PrepareTime: 229.739ms - ExecTreePrepareTime: 167.730us KuduTableSink:(Total: 36m9s, non-child: 36m9s, % non-child: 100.00%) - KuduApplyTimer: 29m15s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 90.13 K/sec - TotalNumRows: 195.49M (195485020) SORT_NODE (id=2):(Total: 7h25m, non-child: 1h39m, % non-child: 22.35%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 41m45s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 195.49M (195486044) - RowsReturnedRate: 7.32 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276709387) Buffer pool: - AllocTime: 37m1s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h45m, non-child: 50s460ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s725ms - PeakMemoryUsage: 19.53 MB (20482691) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.50 K/sec Buffer pool: - AllocTime: 3h11m - CumulativeAllocationBytes: 287.43 GB (308628398080) - CumulativeAllocations: 15.07M (15069746) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.75 GB, 50.19 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.27 GB, 89.51 GB, 94.65 GB, 99.40 GB, 103.76 GB, 108.01 GB, 112.21 GB, 116.16 GB, 120.02 GB, 123.59 GB, 127.05 GB, 130.32 GB, 133.46 GB, 136.53 GB, 139.44 GB, 142.21 GB, 145.10 GB, 147.88 GB, 150.56 GB, 153.28 GB, 155.98 GB, 158.68 GB, 161.41 GB, 163.11 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 913.973ms - TotalBytesReceived: 163.15 GB (175185934283) - TotalGetBatchTime: 5h45m - DataArrivalTimer: 5h44m SenderSide: - DeserializeRowBatchTime: 3h24m - NumBatchesArrived: 7.53M (7534926) - NumBatchesDeferred: 160.15K (160146) - NumBatchesEnqueued: 7.53M (7534873) - NumBatchesReceived: 7.53M (7534873) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.303ms, non-child: 62.303ms, % non-child: 100.00%) - CodegenTime: 1.424ms - CompileTime: 4.975ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 33.221ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.347ms Instance 844a0200d7876345:20bb38b9000000a8 (host=vc1501.foo.com:22000):(Total: 1s207ms, non-child: 224.839ms, % non-child: 18.62%) Fragment Instance Lifecycle Event Timeline: 1s208ms - Prepare Finished: 224.335ms (224.335ms) - Open Finished: 1s208ms (983.800ms) MemoryUsage(8m32s): 22.81 GB, 40.35 GB, 49.69 GB, 57.95 GB, 65.52 GB, 72.19 GB, 77.96 GB, 83.39 GB, 88.58 GB, 93.67 GB, 98.36 GB, 102.67 GB, 106.88 GB, 111.02 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.70 GB, 128.94 GB, 132.05 GB, 135.08 GB, 137.96 GB, 140.70 GB, 143.56 GB, 146.31 GB, 148.96 GB, 151.66 GB, 154.32 GB, 157.00 GB, 159.69 GB, 161.37 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173321306550) - PeakReservation: 161.38 GB (173281378304) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.35 GB (188284393114) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h44m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 983.787ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 53.991ms - TotalThreadsVoluntaryContextSwitches: 13 (13) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 983.790ms - ExecTreeOpenTime: 926.145ms - PrepareTime: 223.820ms - ExecTreePrepareTime: 153.916us KuduTableSink:(Total: 11.587us, non-child: 11.587us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 926.295ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173281386496) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 38m37s - CumulativeAllocationBytes: 161.38 GB (173281378304) - CumulativeAllocations: 82.63K (82627) - PeakReservation: 161.38 GB (173281378304) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173281378304) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h44m, non-child: 51s004ms, % non-child: 0.25%) - ConvertRowBatchTime: 20s977ms - PeakMemoryUsage: 19.53 MB (20481760) - RowsReturned: 632.87M (632874228) - RowsReturnedRate: 30.61 K/sec Buffer pool: - AllocTime: 3h3m - CumulativeAllocationBytes: 287.41 GB (308601528320) - CumulativeAllocations: 15.07M (15068434) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.02 GB, 40.75 GB, 50.19 GB, 58.54 GB, 66.19 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.65 GB, 99.40 GB, 103.75 GB, 108.01 GB, 112.20 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.03 GB, 130.31 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.09 GB, 147.87 GB, 150.55 GB, 153.27 GB, 155.96 GB, 158.67 GB, 161.39 GB, 163.09 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 926.058ms - TotalBytesReceived: 163.14 GB (175170086452) - TotalGetBatchTime: 5h44m - DataArrivalTimer: 5h43m SenderSide: - DeserializeRowBatchTime: 3h14m - NumBatchesArrived: 7.53M (7534217) - NumBatchesDeferred: 163.90K (163902) - NumBatchesEnqueued: 7.53M (7534217) - NumBatchesReceived: 7.53M (7534217) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.521ms, non-child: 56.521ms, % non-child: 100.00%) - CodegenTime: 1.098ms - CompileTime: 6.061ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 27.567ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.442ms Instance 844a0200d7876345:20bb38b9000000a7 (host=vd1334.foo.com:22000):(Total: 1s207ms, non-child: 227.273ms, % non-child: 18.82%) Fragment Instance Lifecycle Event Timeline: 1s207ms - Prepare Finished: 226.446ms (226.446ms) - Open Finished: 1s207ms (981.452ms) MemoryUsage(8m32s): 22.79 GB, 40.34 GB, 49.69 GB, 57.95 GB, 65.51 GB, 72.18 GB, 77.95 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.36 GB, 102.66 GB, 106.87 GB, 111.01 GB, 114.93 GB, 118.74 GB, 122.27 GB, 125.69 GB, 128.93 GB, 132.04 GB, 135.07 GB, 137.95 GB, 140.69 GB, 143.55 GB, 146.30 GB, 148.95 GB, 151.64 GB, 154.31 GB, 156.99 GB, 159.68 GB, 161.36 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.41 GB (173312917942) - PeakReservation: 161.37 GB (173272989696) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.11 GB (186950953992) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h44m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 981.435ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 58.991ms - TotalThreadsVoluntaryContextSwitches: 5 (5) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 981.438ms - ExecTreeOpenTime: 922.141ms - PrepareTime: 226.075ms - ExecTreePrepareTime: 176.526us KuduTableSink:(Total: 13.170us, non-child: 13.170us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 922.315ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.37 GB (173272997888) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 38m9s - CumulativeAllocationBytes: 161.37 GB (173272989696) - CumulativeAllocations: 82.62K (82623) - PeakReservation: 161.37 GB (173272989696) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.37 GB (173272989696) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h44m, non-child: 51s540ms, % non-child: 0.25%) - ConvertRowBatchTime: 21s468ms - PeakMemoryUsage: 19.53 MB (20482331) - RowsReturned: 632.84M (632839620) - RowsReturnedRate: 30.58 K/sec Buffer pool: - AllocTime: 3h20m - CumulativeAllocationBytes: 287.39 GB (308584652800) - CumulativeAllocations: 15.07M (15067610) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.04 GB, 40.75 GB, 50.20 GB, 58.55 GB, 66.19 GB, 72.94 GB, 78.76 GB, 84.26 GB, 89.50 GB, 94.64 GB, 99.39 GB, 103.75 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.57 GB, 127.02 GB, 130.30 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.19 GB, 145.08 GB, 147.86 GB, 150.54 GB, 153.26 GB, 155.96 GB, 158.66 GB, 161.39 GB, 163.09 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB - FirstBatchArrivalWaitTime: 922.083ms - TotalBytesReceived: 163.13 GB (175161066874) - TotalGetBatchTime: 5h44m - DataArrivalTimer: 5h44m SenderSide: - DeserializeRowBatchTime: 3h32m - NumBatchesArrived: 7.53M (7533805) - NumBatchesDeferred: 158.69K (158685) - NumBatchesEnqueued: 7.53M (7533805) - NumBatchesReceived: 7.53M (7533805) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 57.973ms, non-child: 57.973ms, % non-child: 100.00%) - CodegenTime: 1.189ms - CompileTime: 4.877ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.975ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.501ms Instance 844a0200d7876345:20bb38b9000000df (host=vd1329.foo.com:22000):(Total: 1s207ms, non-child: 215.249ms, % non-child: 17.83%) Fragment Instance Lifecycle Event Timeline: 1s207ms - Prepare Finished: 214.329ms (214.329ms) - Open Finished: 1s207ms (993.248ms) MemoryUsage(8m32s): 20.73 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157586375094) - PeakReservation: 146.73 GB (157546446848) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.46 GB (171221639999) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h55m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 993.226ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 50.992ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 993.236ms - ExecTreeOpenTime: 939.369ms - PrepareTime: 213.885ms - ExecTreePrepareTime: 156.708us KuduTableSink:(Total: 13.512us, non-child: 13.512us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 939.522ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.73 GB (157546455040) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 28m44s - CumulativeAllocationBytes: 146.73 GB (157546446848) - CumulativeAllocations: 75.12K (75124) - PeakReservation: 146.73 GB (157546446848) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.73 GB (157546446848) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h55m, non-child: 48s890ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s558ms - PeakMemoryUsage: 19.53 MB (20481445) - RowsReturned: 575.40M (575402520) - RowsReturnedRate: 26.95 K/sec Buffer pool: - AllocTime: 2h20m - CumulativeAllocationBytes: 261.31 GB (280577228800) - CumulativeAllocations: 13.70M (13700060) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.95 GB, 37.05 GB, 45.64 GB, 53.24 GB, 60.19 GB, 66.32 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.59 GB, 109.09 GB, 112.35 GB, 115.49 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.76 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.74 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB - FirstBatchArrivalWaitTime: 939.273ms - TotalBytesReceived: 148.33 GB (159263123041) - TotalGetBatchTime: 5h55m - DataArrivalTimer: 5h55m SenderSide: - DeserializeRowBatchTime: 2h29m - NumBatchesArrived: 6.85M (6850030) - NumBatchesDeferred: 23.18K (23184) - NumBatchesEnqueued: 6.85M (6850030) - NumBatchesReceived: 6.85M (6850030) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 52.419ms, non-child: 52.419ms, % non-child: 100.00%) - CodegenTime: 1.259ms - CompileTime: 4.789ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 30.029ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 17.026ms Instance 844a0200d7876345:20bb38b9000000cf (host=va1035.foo.com:22000):(Total: 1s207ms, non-child: 231.690ms, % non-child: 19.19%) Fragment Instance Lifecycle Event Timeline: 1s207ms - Prepare Finished: 231.185ms (231.185ms) - Open Finished: 1s207ms (976.623ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.18 GB, 52.70 GB, 59.58 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.35 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.28 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.43 GB, 127.93 GB, 130.52 GB, 133.02 GB, 135.44 GB, 137.89 GB, 140.31 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157586375094) - PeakReservation: 146.73 GB (157546446848) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.64 GB (172484812903) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 6h1m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 976.606ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 50.992ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 976.609ms - ExecTreeOpenTime: 925.174ms - PrepareTime: 230.539ms - ExecTreePrepareTime: 195.562us KuduTableSink:(Total: 11.854us, non-child: 11.854us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 925.366ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.73 GB (157546455040) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 23m15s - CumulativeAllocationBytes: 146.73 GB (157546446848) - CumulativeAllocations: 75.12K (75124) - PeakReservation: 146.73 GB (157546446848) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.73 GB (157546446848) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 6h1m, non-child: 48s897ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s473ms - PeakMemoryUsage: 19.53 MB (20483096) - RowsReturned: 575.40M (575400504) - RowsReturnedRate: 26.51 K/sec Buffer pool: - AllocTime: 2h21m - CumulativeAllocationBytes: 261.31 GB (280576245760) - CumulativeAllocations: 13.70M (13700012) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.19 GB, 66.32 GB, 71.62 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.37 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.76 GB, 129.28 GB, 131.91 GB, 134.44 GB, 136.88 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.74 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 925.105ms - TotalBytesReceived: 148.32 GB (159262316439) - TotalGetBatchTime: 6h1m - DataArrivalTimer: 6h SenderSide: - DeserializeRowBatchTime: 2h32m - NumBatchesArrived: 6.85M (6850006) - NumBatchesDeferred: 24.02K (24023) - NumBatchesEnqueued: 6.85M (6850006) - NumBatchesReceived: 6.85M (6850006) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 50.134ms, non-child: 50.134ms, % non-child: 100.00%) - CodegenTime: 1.042ms - CompileTime: 5.505ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 27.095ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 16.971ms Instance 844a0200d7876345:20bb38b90000009d (host=va1037.foo.com:22000):(Total: 1s206ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h13m - Prepare Finished: 211.085ms (211.085ms) - Open Finished: 1s206ms (995.716ms) - First Batch Produced: 7h13m (7h13m) - First Batch Sent: 7h13m (10.905ms) MemoryUsage(8m32s): 22.82 GB, 40.35 GB, 49.70 GB, 57.96 GB, 65.52 GB, 72.19 GB, 77.96 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.36 GB, 102.67 GB, 106.87 GB, 111.02 GB, 114.94 GB, 118.74 GB, 122.28 GB, 125.70 GB, 128.94 GB, 132.05 GB, 135.09 GB, 137.97 GB, 140.71 GB, 143.57 GB, 146.32 GB, 148.97 GB, 151.67 GB, 154.33 GB, 157.01 GB, 159.70 GB, 161.38 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.24 GB (188165586922) - RowsProduced: 238.63M (238634034) - TotalNetworkReceiveTime: 5h49m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 995.697ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 50.992ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h13m - OpenTime: 995.701ms - ExecTreeOpenTime: 940.801ms - PrepareTime: 210.704ms - ExecTreePrepareTime: 157.022us KuduTableSink:(Total: 47m25s, non-child: 47m25s, % non-child: 100.00%) - KuduApplyTimer: 38m33s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 83.87 K/sec - TotalNumRows: 238.63M (238633010) SORT_NODE (id=2):(Total: 7h13m, non-child: 1h24m, % non-child: 19.38%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 30m8s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 238.63M (238634034) - RowsReturnedRate: 9.17 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276292377) Buffer pool: - AllocTime: 33m9s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h49m, non-child: 50s665ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s973ms - PeakMemoryUsage: 19.53 MB (20481975) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.17 K/sec Buffer pool: - AllocTime: 4h3m - CumulativeAllocationBytes: 287.43 GB (308624588800) - CumulativeAllocations: 15.07M (15069560) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.75 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.64 GB, 99.39 GB, 103.75 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.03 GB, 130.31 GB, 133.46 GB, 136.52 GB, 139.43 GB, 142.21 GB, 145.09 GB, 147.88 GB, 150.56 GB, 153.28 GB, 155.97 GB, 158.68 GB, 161.40 GB, 163.11 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 940.713ms - TotalBytesReceived: 163.15 GB (175183357247) - TotalGetBatchTime: 5h49m - DataArrivalTimer: 5h48m SenderSide: - DeserializeRowBatchTime: 4h13m - NumBatchesArrived: 7.53M (7534780) - NumBatchesDeferred: 155.51K (155515) - NumBatchesEnqueued: 7.53M (7534780) - NumBatchesReceived: 7.53M (7534780) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 53.451ms, non-child: 53.451ms, % non-child: 100.00%) - CodegenTime: 1.155ms - CompileTime: 5.020ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.776ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.060ms Instance 844a0200d7876345:20bb38b9000000d2 (host=vc1538.foo.com:22000):(Total: 1s206ms, non-child: 241.319ms, % non-child: 20.01%) Fragment Instance Lifecycle Event Timeline: 1s206ms - Prepare Finished: 240.497ms (240.497ms) - Open Finished: 1s206ms (965.994ms) MemoryUsage(8m32s): 20.72 GB, 36.67 GB, 45.17 GB, 52.68 GB, 59.56 GB, 65.62 GB, 70.86 GB, 75.81 GB, 80.52 GB, 85.14 GB, 89.41 GB, 93.33 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.02 GB, 135.43 GB, 137.87 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157571695030) - PeakReservation: 146.71 GB (157531766784) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.70 GB (172549140051) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h53m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 965.963ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 54.992ms - TotalThreadsVoluntaryContextSwitches: 18 (18) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 965.967ms - ExecTreeOpenTime: 907.770ms - PrepareTime: 240.233ms - ExecTreePrepareTime: 194.336us KuduTableSink:(Total: 12.657us, non-child: 12.657us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 907.961ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157531774976) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 30m40s - CumulativeAllocationBytes: 146.71 GB (157531766784) - CumulativeAllocations: 75.12K (75117) - PeakReservation: 146.71 GB (157531766784) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157531766784) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 47s801ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s607ms - PeakMemoryUsage: 19.53 MB (20482184) - RowsReturned: 575.35M (575345064) - RowsReturnedRate: 27.13 K/sec Buffer pool: - AllocTime: 2h25m - CumulativeAllocationBytes: 261.28 GB (280549212160) - CumulativeAllocations: 13.70M (13698692) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.04 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.30 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.35 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 907.680ms - TotalBytesReceived: 148.31 GB (159247415754) - TotalGetBatchTime: 5h53m - DataArrivalTimer: 5h52m SenderSide: - DeserializeRowBatchTime: 2h35m - NumBatchesArrived: 6.85M (6849346) - NumBatchesDeferred: 53.43K (53428) - NumBatchesEnqueued: 6.85M (6849346) - NumBatchesReceived: 6.85M (6849346) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.967ms, non-child: 56.967ms, % non-child: 100.00%) - CodegenTime: 957.235us - CompileTime: 5.082ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 27.097ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.235ms Instance 844a0200d7876345:20bb38b9000000ab (host=vc1541.foo.com:22000):(Total: 1s204ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h16m - Prepare Finished: 231.849ms (231.849ms) - Open Finished: 1s204ms (973.034ms) - First Batch Produced: 7h16m (7h16m) - First Batch Sent: 7h16m (8.172ms) MemoryUsage(8m32s): 22.80 GB, 40.35 GB, 49.70 GB, 57.96 GB, 65.53 GB, 72.20 GB, 77.97 GB, 83.40 GB, 88.58 GB, 93.67 GB, 98.36 GB, 102.67 GB, 106.88 GB, 111.03 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.09 GB, 137.97 GB, 140.71 GB, 143.57 GB, 146.32 GB, 148.97 GB, 151.66 GB, 154.33 GB, 157.00 GB, 159.70 GB, 161.38 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.38 GB (188314335618) - RowsProduced: 260.99M (260985132) - TotalNetworkReceiveTime: 5h46m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 973.019ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 57.991ms - TotalThreadsVoluntaryContextSwitches: 19 (19) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h16m - OpenTime: 973.022ms - ExecTreeOpenTime: 913.847ms - PrepareTime: 231.434ms - ExecTreePrepareTime: 187.198us KuduTableSink:(Total: 45m15s, non-child: 45m15s, % non-child: 100.00%) - KuduApplyTimer: 37m49s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 96.09 K/sec - TotalNumRows: 260.98M (260984108) SORT_NODE (id=2):(Total: 7h16m, non-child: 1h29m, % non-child: 20.40%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 32m44s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 260.99M (260985132) - RowsReturnedRate: 9.97 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276409761) Buffer pool: - AllocTime: 37m4s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h47m, non-child: 51s909ms, % non-child: 0.25%) - ConvertRowBatchTime: 21s393ms - PeakMemoryUsage: 19.53 MB (20482278) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.37 K/sec Buffer pool: - AllocTime: 4h33m - CumulativeAllocationBytes: 287.42 GB (308617379840) - CumulativeAllocations: 15.07M (15069208) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.01 GB, 40.75 GB, 50.20 GB, 58.56 GB, 66.20 GB, 72.95 GB, 78.78 GB, 84.27 GB, 89.51 GB, 94.65 GB, 99.40 GB, 103.75 GB, 108.01 GB, 112.20 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.32 GB, 133.46 GB, 136.52 GB, 139.43 GB, 142.21 GB, 145.10 GB, 147.88 GB, 150.56 GB, 153.28 GB, 155.97 GB, 158.68 GB, 161.40 GB, 163.10 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 913.754ms - TotalBytesReceived: 163.15 GB (175179383733) - TotalGetBatchTime: 5h46m - DataArrivalTimer: 5h46m SenderSide: - DeserializeRowBatchTime: 4h43m - NumBatchesArrived: 7.53M (7534604) - NumBatchesDeferred: 163.43K (163429) - NumBatchesEnqueued: 7.53M (7534604) - NumBatchesReceived: 7.53M (7534604) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 57.776ms, non-child: 57.776ms, % non-child: 100.00%) - CodegenTime: 1.773ms - CompileTime: 5.603ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 30.823ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 20.827ms Instance 844a0200d7876345:20bb38b9000000be (host=vd1109.foo.com:22000):(Total: 1s204ms, non-child: 230.700ms, % non-child: 19.16%) Fragment Instance Lifecycle Event Timeline: 1s204ms - Prepare Finished: 230.028ms (230.028ms) - Open Finished: 1s204ms (974.595ms) MemoryUsage(8m32s): 20.73 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.43 GB, 137.87 GB, 140.29 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157571695030) - PeakReservation: 146.71 GB (157531766784) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.70 GB (172550786545) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h47m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 974.576ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 50.992ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 974.580ms - ExecTreeOpenTime: 921.709ms - PrepareTime: 229.607ms - ExecTreePrepareTime: 168.593us KuduTableSink:(Total: 12.983us, non-child: 12.983us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 921.874ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157531774976) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 33m29s - CumulativeAllocationBytes: 146.71 GB (157531766784) - CumulativeAllocations: 75.12K (75117) - PeakReservation: 146.71 GB (157531766784) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157531766784) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h47m, non-child: 47s875ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s716ms - PeakMemoryUsage: 19.54 MB (20484347) - RowsReturned: 575.35M (575349012) - RowsReturnedRate: 27.59 K/sec Buffer pool: - AllocTime: 4h50m - CumulativeAllocationBytes: 261.28 GB (280551137280) - CumulativeAllocations: 13.70M (13698786) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.92 GB, 37.04 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.36 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 921.615ms - TotalBytesReceived: 148.31 GB (159247997824) - TotalGetBatchTime: 5h47m - DataArrivalTimer: 5h46m SenderSide: - DeserializeRowBatchTime: 5h12m - NumBatchesArrived: 6.85M (6849393) - NumBatchesDeferred: 151.78K (151784) - NumBatchesEnqueued: 6.85M (6849393) - NumBatchesReceived: 6.85M (6849393) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 51.668ms, non-child: 51.668ms, % non-child: 100.00%) - CodegenTime: 848.154us - CompileTime: 4.082ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 25.897ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 21.208ms Instance 844a0200d7876345:20bb38b900000081 (host=vd1138.foo.com:22000):(Total: 1s203ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h17m - Prepare Finished: 225.406ms (225.406ms) - Open Finished: 1s204ms (978.662ms) - First Batch Produced: 7h17m (7h17m) - First Batch Sent: 7h17m (12.554ms) MemoryUsage(8m32s): 22.81 GB, 40.35 GB, 49.69 GB, 57.96 GB, 65.52 GB, 72.19 GB, 77.96 GB, 83.39 GB, 88.58 GB, 93.67 GB, 98.36 GB, 102.67 GB, 106.88 GB, 111.02 GB, 114.94 GB, 118.74 GB, 122.28 GB, 125.70 GB, 128.94 GB, 132.05 GB, 135.08 GB, 137.96 GB, 140.71 GB, 143.56 GB, 146.31 GB, 148.97 GB, 151.66 GB, 154.32 GB, 157.00 GB, 159.69 GB, 161.38 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.09 GB (186929579116) - RowsProduced: 235.83M (235829604) - TotalNetworkReceiveTime: 5h46m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 978.643ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 60.991ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h17m - OpenTime: 978.646ms - ExecTreeOpenTime: 916.804ms - PrepareTime: 224.886ms - ExecTreePrepareTime: 168.669us KuduTableSink:(Total: 44m, non-child: 44m, % non-child: 100.00%) - KuduApplyTimer: 35m21s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 89.32 K/sec - TotalNumRows: 235.83M (235828580) SORT_NODE (id=2):(Total: 7h17m, non-child: 1h29m, % non-child: 20.55%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 33m40s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 235.83M (235829604) - RowsReturnedRate: 8.99 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276526542) Buffer pool: - AllocTime: 35m32s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h47m, non-child: 50s603ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s817ms - PeakMemoryUsage: 19.53 MB (20481895) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.36 K/sec Buffer pool: - AllocTime: 2h45m - CumulativeAllocationBytes: 287.42 GB (308612014080) - CumulativeAllocations: 15.07M (15068946) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.02 GB, 40.75 GB, 50.19 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.65 GB, 99.40 GB, 103.75 GB, 108.00 GB, 112.20 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.03 GB, 130.31 GB, 133.45 GB, 136.52 GB, 139.43 GB, 142.20 GB, 145.09 GB, 147.87 GB, 150.55 GB, 153.27 GB, 155.97 GB, 158.67 GB, 161.40 GB, 163.10 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 916.744ms - TotalBytesReceived: 163.15 GB (175176453464) - TotalGetBatchTime: 5h46m - DataArrivalTimer: 5h46m SenderSide: - DeserializeRowBatchTime: 2h55m - NumBatchesArrived: 7.53M (7534473) - NumBatchesDeferred: 105.66K (105661) - NumBatchesEnqueued: 7.53M (7534473) - NumBatchesReceived: 7.53M (7534473) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 60.614ms, non-child: 60.614ms, % non-child: 100.00%) - CodegenTime: 1.108ms - CompileTime: 5.680ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 34.703ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.575ms Instance 844a0200d7876345:20bb38b9000000cc (host=ve1141.foo.com:22000):(Total: 1s201ms, non-child: 218.938ms, % non-child: 18.22%) Fragment Instance Lifecycle Event Timeline: 1s202ms - Prepare Finished: 218.360ms (218.360ms) - Open Finished: 1s202ms (984.151ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.19 GB, 52.70 GB, 59.58 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.35 GB, 97.17 GB, 100.95 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.29 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.43 GB, 127.93 GB, 130.52 GB, 133.02 GB, 135.44 GB, 137.89 GB, 140.31 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157584277942) - PeakReservation: 146.72 GB (157544349696) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.46 GB (171222448026) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h53m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 10 (10) - TotalThreadsTotalWallClockTime: 984.127ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 65.990ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 984.131ms - ExecTreeOpenTime: 916.438ms - PrepareTime: 217.747ms - ExecTreePrepareTime: 145.462us KuduTableSink:(Total: 13.344us, non-child: 13.344us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 916.581ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157544357888) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 30m50s - CumulativeAllocationBytes: 146.72 GB (157544349696) - CumulativeAllocations: 75.12K (75123) - PeakReservation: 146.72 GB (157544349696) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157544349696) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 48s667ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s241ms - PeakMemoryUsage: 19.53 MB (20483930) - RowsReturned: 575.40M (575396556) - RowsReturnedRate: 27.11 K/sec Buffer pool: - AllocTime: 3h26m - CumulativeAllocationBytes: 261.31 GB (280574320640) - CumulativeAllocations: 13.70M (13699918) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.64 GB, 53.24 GB, 60.19 GB, 66.32 GB, 71.62 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.37 GB, 94.33 GB, 98.19 GB, 102.01 GB, 105.60 GB, 109.10 GB, 112.36 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.76 GB, 129.28 GB, 131.91 GB, 134.44 GB, 136.88 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 916.355ms - TotalBytesReceived: 148.32 GB (159261214083) - TotalGetBatchTime: 5h53m - DataArrivalTimer: 5h52m SenderSide: - DeserializeRowBatchTime: 3h35m - NumBatchesArrived: 6.85M (6849959) - NumBatchesDeferred: 87.46K (87459) - NumBatchesEnqueued: 6.85M (6849959) - NumBatchesReceived: 6.85M (6849959) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 66.395ms, non-child: 66.395ms, % non-child: 100.00%) - CodegenTime: 1.084ms - CompileTime: 5.963ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 35.955ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.929ms Instance 844a0200d7876345:20bb38b900000095 (host=va1033.foo.com:22000):(Total: 1s201ms, non-child: 223.250ms, % non-child: 18.59%) Fragment Instance Lifecycle Event Timeline: 1s201ms - Prepare Finished: 222.538ms (222.538ms) - Open Finished: 1s201ms (979.007ms) MemoryUsage(8m32s): 22.81 GB, 40.34 GB, 49.69 GB, 57.95 GB, 65.52 GB, 72.19 GB, 77.96 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.36 GB, 102.67 GB, 106.87 GB, 111.02 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.70 GB, 128.94 GB, 132.05 GB, 135.08 GB, 137.96 GB, 140.71 GB, 143.56 GB, 146.31 GB, 148.96 GB, 151.66 GB, 154.32 GB, 157.00 GB, 159.69 GB, 161.37 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173323403702) - PeakReservation: 161.38 GB (173283475456) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.26 GB (188180250498) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h44m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 978.978ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 62.991ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 978.981ms - ExecTreeOpenTime: 914.628ms - PrepareTime: 222.109ms - ExecTreePrepareTime: 175.221us KuduTableSink:(Total: 16.157us, non-child: 16.157us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 914.800ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173283483648) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 37m27s - CumulativeAllocationBytes: 161.38 GB (173283475456) - CumulativeAllocations: 82.63K (82628) - PeakReservation: 161.38 GB (173283475456) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173283475456) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h45m, non-child: 50s426ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s783ms - PeakMemoryUsage: 19.53 MB (20482082) - RowsReturned: 632.88M (632878764) - RowsReturnedRate: 30.54 K/sec Buffer pool: - AllocTime: 4h38m - CumulativeAllocationBytes: 287.41 GB (308603740160) - CumulativeAllocations: 15.07M (15068542) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.02 GB, 40.74 GB, 50.19 GB, 58.55 GB, 66.19 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.64 GB, 99.39 GB, 103.75 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.03 GB, 130.31 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.09 GB, 147.87 GB, 150.55 GB, 153.27 GB, 155.96 GB, 158.67 GB, 161.39 GB, 163.10 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 914.550ms - TotalBytesReceived: 163.14 GB (175172149611) - TotalGetBatchTime: 5h44m - DataArrivalTimer: 5h44m SenderSide: - DeserializeRowBatchTime: 4h49m - NumBatchesArrived: 7.53M (7534271) - NumBatchesDeferred: 177.41K (177410) - NumBatchesEnqueued: 7.53M (7534271) - NumBatchesReceived: 7.53M (7534271) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 63.080ms, non-child: 63.080ms, % non-child: 100.00%) - CodegenTime: 977.651us - CompileTime: 5.410ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.197ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.875ms Instance 844a0200d7876345:20bb38b9000000b3 (host=ve1135.foo.com:22000):(Total: 1s200ms, non-child: 216.960ms, % non-child: 18.07%) Fragment Instance Lifecycle Event Timeline: 1s201ms - Prepare Finished: 216.307ms (216.307ms) - Open Finished: 1s201ms (985.126ms) MemoryUsage(8m32s): 20.75 GB, 36.69 GB, 45.19 GB, 52.70 GB, 59.57 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.91 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.72 GB, 145.17 GB, 146.70 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157565403574) - PeakReservation: 146.71 GB (157525475328) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.39 GB (171139269720) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h54m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 985.109ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 53.992ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 985.113ms - ExecTreeOpenTime: 929.144ms - PrepareTime: 215.693ms - ExecTreePrepareTime: 160.055us KuduTableSink:(Total: 11.236us, non-child: 11.236us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 929.301ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157525483520) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 30m11s - CumulativeAllocationBytes: 146.71 GB (157525475328) - CumulativeAllocations: 75.11K (75114) - PeakReservation: 146.71 GB (157525475328) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157525475328) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h54m, non-child: 48s987ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s276ms - PeakMemoryUsage: 19.53 MB (20482933) - RowsReturned: 575.32M (575322972) - RowsReturnedRate: 27.02 K/sec Buffer pool: - AllocTime: 3h4m - CumulativeAllocationBytes: 261.27 GB (280538439680) - CumulativeAllocations: 13.70M (13698166) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.26 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB - FirstBatchArrivalWaitTime: 929.029ms - TotalBytesReceived: 148.30 GB (159241098947) - TotalGetBatchTime: 5h54m - DataArrivalTimer: 5h54m SenderSide: - DeserializeRowBatchTime: 3h14m - NumBatchesArrived: 6.85M (6849083) - NumBatchesDeferred: 55.14K (55141) - NumBatchesEnqueued: 6.85M (6849083) - NumBatchesReceived: 6.85M (6849083) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 54.618ms, non-child: 54.618ms, % non-child: 100.00%) - CodegenTime: 1.161ms - CompileTime: 5.098ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.386ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 17.569ms Instance 844a0200d7876345:20bb38b9000000e0 (host=vc1509.foo.com:22000):(Total: 1s200ms, non-child: 218.340ms, % non-child: 18.19%) Fragment Instance Lifecycle Event Timeline: 1s201ms - Prepare Finished: 217.857ms (217.857ms) - Open Finished: 1s201ms (983.300ms) MemoryUsage(8m32s): 20.73 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.73 GB, 145.17 GB, 146.71 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157569597878) - PeakReservation: 146.71 GB (157529669632) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.73 GB (172584902379) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h47m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 983.276ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 54.992ms - TotalThreadsVoluntaryContextSwitches: 21 (21) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 983.279ms - ExecTreeOpenTime: 925.269ms - PrepareTime: 217.206ms - ExecTreePrepareTime: 157.935us KuduTableSink:(Total: 12.555us, non-child: 12.555us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 925.424ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157529677824) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 35m35s - CumulativeAllocationBytes: 146.71 GB (157529669632) - CumulativeAllocations: 75.12K (75116) - PeakReservation: 146.71 GB (157529669632) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157529669632) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h48m, non-child: 47s922ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s934ms - PeakMemoryUsage: 19.53 MB (20483732) - RowsReturned: 575.34M (575336832) - RowsReturnedRate: 27.53 K/sec Buffer pool: - AllocTime: 3h19m - CumulativeAllocationBytes: 261.28 GB (280545198080) - CumulativeAllocations: 13.70M (13698496) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.95 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.27 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 925.207ms - TotalBytesReceived: 148.31 GB (159245146442) - TotalGetBatchTime: 5h47m - DataArrivalTimer: 5h47m SenderSide: - DeserializeRowBatchTime: 3h35m - NumBatchesArrived: 6.85M (6849248) - NumBatchesDeferred: 99.85K (99853) - NumBatchesEnqueued: 6.85M (6849248) - NumBatchesReceived: 6.85M (6849248) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.759ms, non-child: 56.759ms, % non-child: 100.00%) - CodegenTime: 1.097ms - CompileTime: 5.425ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 30.350ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 20.371ms Instance 844a0200d7876345:20bb38b90000009e (host=vc1524.foo.com:22000):(Total: 1s199ms, non-child: 230.270ms, % non-child: 19.19%) Fragment Instance Lifecycle Event Timeline: 1s200ms - Prepare Finished: 229.578ms (229.578ms) - Open Finished: 1s200ms (970.702ms) MemoryUsage(8m32s): 22.82 GB, 40.34 GB, 49.69 GB, 57.96 GB, 65.52 GB, 72.19 GB, 77.95 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.35 GB, 102.66 GB, 106.87 GB, 111.02 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.69 GB, 128.93 GB, 132.04 GB, 135.07 GB, 137.95 GB, 140.70 GB, 143.55 GB, 146.31 GB, 148.96 GB, 151.65 GB, 154.31 GB, 156.99 GB, 159.68 GB, 161.37 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173319209398) - PeakReservation: 161.38 GB (173279281152) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.31 GB (188237586452) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h51m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 970.672ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 59.991ms - TotalThreadsVoluntaryContextSwitches: 8 (8) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 970.689ms - ExecTreeOpenTime: 907.150ms - PrepareTime: 229.150ms - ExecTreePrepareTime: 270.220us KuduTableSink:(Total: 12.660us, non-child: 12.660us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 907.417ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173279289344) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 31m51s - CumulativeAllocationBytes: 161.38 GB (173279281152) - CumulativeAllocations: 82.63K (82626) - PeakReservation: 161.38 GB (173279281152) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173279281152) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h51m, non-child: 51s668ms, % non-child: 0.24%) - ConvertRowBatchTime: 21s393ms - PeakMemoryUsage: 19.53 MB (20481153) - RowsReturned: 632.86M (632862552) - RowsReturnedRate: 29.98 K/sec Buffer pool: - AllocTime: 3h38m - CumulativeAllocationBytes: 287.40 GB (308595834880) - CumulativeAllocations: 15.07M (15068156) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.74 GB, 50.19 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.76 GB, 84.26 GB, 89.50 GB, 94.64 GB, 99.39 GB, 103.74 GB, 108.00 GB, 112.19 GB, 116.14 GB, 119.99 GB, 123.57 GB, 127.02 GB, 130.30 GB, 133.44 GB, 136.51 GB, 139.42 GB, 142.19 GB, 145.08 GB, 147.86 GB, 150.55 GB, 153.27 GB, 155.96 GB, 158.66 GB, 161.39 GB, 163.09 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 907.097ms - TotalBytesReceived: 163.14 GB (175167050496) - TotalGetBatchTime: 5h51m - DataArrivalTimer: 5h51m SenderSide: - DeserializeRowBatchTime: 3h47m - NumBatchesArrived: 7.53M (7534078) - NumBatchesDeferred: 88.38K (88375) - NumBatchesEnqueued: 7.53M (7534078) - NumBatchesReceived: 7.53M (7534078) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.195ms, non-child: 62.195ms, % non-child: 100.00%) - CodegenTime: 1.014ms - CompileTime: 5.424ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 30.857ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 25.317ms Instance 844a0200d7876345:20bb38b9000000e9 (host=vc1508.foo.com:22000):(Total: 1s198ms, non-child: 227.281ms, % non-child: 18.96%) Fragment Instance Lifecycle Event Timeline: 1s199ms - Prepare Finished: 226.399ms (226.399ms) - Open Finished: 1s199ms (972.618ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.18 GB, 52.70 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.53 GB, 85.15 GB, 89.43 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.49 GB, 107.96 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157569597878) - PeakReservation: 146.71 GB (157529669632) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.57 GB (172411812995) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h56m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 972.603ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 58.991ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 972.606ms - ExecTreeOpenTime: 910.497ms - PrepareTime: 226.006ms - ExecTreePrepareTime: 166.277us KuduTableSink:(Total: 12.890us, non-child: 12.890us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 910.660ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157529677824) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 28m35s - CumulativeAllocationBytes: 146.71 GB (157529669632) - CumulativeAllocations: 75.12K (75116) - PeakReservation: 146.71 GB (157529669632) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157529669632) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h56m, non-child: 48s021ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s832ms - PeakMemoryUsage: 19.53 MB (20483554) - RowsReturned: 575.34M (575337840) - RowsReturnedRate: 26.89 K/sec Buffer pool: - AllocTime: 3h5m - CumulativeAllocationBytes: 261.28 GB (280545689600) - CumulativeAllocations: 13.70M (13698520) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.92 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.59 GB, 109.09 GB, 112.35 GB, 115.49 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 910.419ms - TotalBytesReceived: 148.31 GB (159245453420) - TotalGetBatchTime: 5h56m - DataArrivalTimer: 5h55m SenderSide: - DeserializeRowBatchTime: 3h13m - NumBatchesArrived: 6.85M (6849260) - NumBatchesDeferred: 52.75K (52747) - NumBatchesEnqueued: 6.85M (6849260) - NumBatchesReceived: 6.85M (6849260) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 60.714ms, non-child: 60.714ms, % non-child: 100.00%) - CodegenTime: 1.239ms - CompileTime: 5.429ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.379ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.295ms Instance 844a0200d7876345:20bb38b9000000f2 (host=ve1132.foo.com:22000):(Total: 1s198ms, non-child: 207.931ms, % non-child: 17.35%) Fragment Instance Lifecycle Event Timeline: 1s199ms - Prepare Finished: 207.314ms (207.314ms) - Open Finished: 1s199ms (991.831ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.17 GB, 52.69 GB, 59.56 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.14 GB, 89.41 GB, 93.32 GB, 97.15 GB, 100.92 GB, 104.47 GB, 107.94 GB, 111.16 GB, 114.26 GB, 117.21 GB, 120.04 GB, 122.79 GB, 125.40 GB, 127.90 GB, 130.49 GB, 133.00 GB, 135.41 GB, 137.85 GB, 140.28 GB, 142.71 GB, 145.16 GB, 146.69 GB, 146.72 GB, 146.72 GB, 146.72 GB, 146.72 GB, 146.72 GB, 146.72 GB, 146.72 GB, 146.72 GB, 146.72 GB, 146.72 GB, 146.72 GB, 146.72 GB, 146.72 GB, 146.72 GB, 146.72 GB, 146.72 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.73 GB (157554917814) - PeakReservation: 146.70 GB (157514989568) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.44 GB (171200310156) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h51m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 991.817ms - TotalThreadsSysTime: 5.000ms - TotalThreadsUserTime: 57.991ms - TotalThreadsVoluntaryContextSwitches: 5 (5) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 991.820ms - ExecTreeOpenTime: 928.149ms - PrepareTime: 206.781ms - ExecTreePrepareTime: 168.418us KuduTableSink:(Total: 15.122us, non-child: 15.122us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 928.315ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.70 GB (157514997760) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 32m23s - CumulativeAllocationBytes: 146.70 GB (157514989568) - CumulativeAllocations: 75.11K (75109) - PeakReservation: 146.70 GB (157514989568) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.70 GB (157514989568) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h51m, non-child: 47s929ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s895ms - PeakMemoryUsage: 19.54 MB (20483951) - RowsReturned: 575.29M (575286600) - RowsReturnedRate: 27.25 K/sec Buffer pool: - AllocTime: 3h22m - CumulativeAllocationBytes: 261.26 GB (280520704000) - CumulativeAllocations: 13.70M (13697300) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.95 GB, 37.05 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.03 GB, 90.35 GB, 94.31 GB, 98.17 GB, 101.98 GB, 105.58 GB, 109.08 GB, 112.33 GB, 115.47 GB, 118.45 GB, 121.31 GB, 124.09 GB, 126.74 GB, 129.26 GB, 131.88 GB, 134.41 GB, 136.85 GB, 139.32 GB, 141.77 GB, 144.23 GB, 146.71 GB, 148.25 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.28 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB - FirstBatchArrivalWaitTime: 928.075ms - TotalBytesReceived: 148.29 GB (159230443855) - TotalGetBatchTime: 5h51m - DataArrivalTimer: 5h51m SenderSide: - DeserializeRowBatchTime: 3h33m - NumBatchesArrived: 6.85M (6848650) - NumBatchesDeferred: 83.90K (83900) - NumBatchesEnqueued: 6.85M (6848650) - NumBatchesReceived: 6.85M (6848650) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.406ms, non-child: 62.406ms, % non-child: 100.00%) - CodegenTime: 1.102ms - CompileTime: 5.761ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.674ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.420ms Instance 844a0200d7876345:20bb38b9000000a4 (host=vd1326.foo.com:22000):(Total: 1s197ms, non-child: 226.975ms, % non-child: 18.95%) Fragment Instance Lifecycle Event Timeline: 1s198ms - Prepare Finished: 226.237ms (226.237ms) - Open Finished: 1s198ms (972.069ms) MemoryUsage(8m32s): 22.79 GB, 40.34 GB, 49.68 GB, 57.95 GB, 65.51 GB, 72.19 GB, 77.95 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.35 GB, 102.66 GB, 106.87 GB, 111.02 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.69 GB, 128.93 GB, 132.05 GB, 135.08 GB, 137.96 GB, 140.70 GB, 143.56 GB, 146.31 GB, 148.96 GB, 151.65 GB, 154.32 GB, 157.00 GB, 159.69 GB, 161.37 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173323403702) - PeakReservation: 161.38 GB (173283475456) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.11 GB (186952073115) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h42m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 972.055ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 57.991ms - TotalThreadsVoluntaryContextSwitches: 16 (16) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 972.058ms - ExecTreeOpenTime: 913.091ms - PrepareTime: 225.846ms - ExecTreePrepareTime: 149.327us KuduTableSink:(Total: 17.547us, non-child: 17.547us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 913.237ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173283483648) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 40m11s - CumulativeAllocationBytes: 161.38 GB (173283475456) - CumulativeAllocations: 82.63K (82628) - PeakReservation: 161.38 GB (173283475456) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173283475456) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h43m, non-child: 50s993ms, % non-child: 0.25%) - ConvertRowBatchTime: 21s333ms - PeakMemoryUsage: 19.53 MB (20482605) - RowsReturned: 632.88M (632877168) - RowsReturnedRate: 30.73 K/sec Buffer pool: - AllocTime: 4h8m - CumulativeAllocationBytes: 287.41 GB (308602961920) - CumulativeAllocations: 15.07M (15068504) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.75 GB, 50.19 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.64 GB, 99.39 GB, 103.75 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.57 GB, 127.03 GB, 130.30 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.09 GB, 147.87 GB, 150.55 GB, 153.27 GB, 155.97 GB, 158.67 GB, 161.39 GB, 163.10 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 912.990ms - TotalBytesReceived: 163.14 GB (175171397448) - TotalGetBatchTime: 5h42m - DataArrivalTimer: 5h42m SenderSide: - DeserializeRowBatchTime: 4h21m - NumBatchesArrived: 7.53M (7534252) - NumBatchesDeferred: 235.61K (235609) - NumBatchesEnqueued: 7.53M (7534252) - NumBatchesReceived: 7.53M (7534252) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 57.736ms, non-child: 57.736ms, % non-child: 100.00%) - CodegenTime: 971.777us - CompileTime: 5.036ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 29.799ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.389ms Instance 844a0200d7876345:20bb38b9000000d5 (host=vd1102.foo.com:22000):(Total: 1s195ms, non-child: 216.758ms, % non-child: 18.13%) Fragment Instance Lifecycle Event Timeline: 1s196ms - Prepare Finished: 215.844ms (215.844ms) - Open Finished: 1s196ms (980.158ms) MemoryUsage(8m32s): 20.73 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.56 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.33 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.04 GB, 122.80 GB, 125.42 GB, 127.91 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.72 GB, 145.17 GB, 146.70 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157563306422) - PeakReservation: 146.71 GB (157523378176) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.55 GB (172384177811) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h56m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 980.140ms - TotalThreadsSysTime: 999.000us - TotalThreadsUserTime: 58.991ms - TotalThreadsVoluntaryContextSwitches: 12 (12) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 980.144ms - ExecTreeOpenTime: 919.482ms - PrepareTime: 215.558ms - ExecTreePrepareTime: 193.774us KuduTableSink:(Total: 14.079us, non-child: 14.079us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 919.673ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157523386368) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 27m57s - CumulativeAllocationBytes: 146.71 GB (157523378176) - CumulativeAllocations: 75.11K (75113) - PeakReservation: 146.71 GB (157523378176) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157523378176) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h56m, non-child: 48s927ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s546ms - PeakMemoryUsage: 19.53 MB (20483170) - RowsReturned: 575.32M (575318352) - RowsReturnedRate: 26.86 K/sec Buffer pool: - AllocTime: 2h18m - CumulativeAllocationBytes: 261.27 GB (280536186880) - CumulativeAllocations: 13.70M (13698056) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.92 GB, 37.04 GB, 45.63 GB, 53.22 GB, 60.17 GB, 66.30 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.35 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.31 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.71 GB, 148.26 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB - FirstBatchArrivalWaitTime: 919.367ms - TotalBytesReceived: 148.30 GB (159239617987) - TotalGetBatchTime: 5h56m - DataArrivalTimer: 5h56m SenderSide: - DeserializeRowBatchTime: 2h28m - NumBatchesArrived: 6.85M (6849028) - NumBatchesDeferred: 34.38K (34384) - NumBatchesEnqueued: 6.85M (6849028) - NumBatchesReceived: 6.85M (6849028) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 59.319ms, non-child: 59.319ms, % non-child: 100.00%) - CodegenTime: 1.069ms - CompileTime: 4.796ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.757ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 25.209ms Instance 844a0200d7876345:20bb38b9000000db (host=vd1339.foo.com:22000):(Total: 1s195ms, non-child: 215.573ms, % non-child: 18.04%) Fragment Instance Lifecycle Event Timeline: 1s195ms - Prepare Finished: 214.449ms (214.449ms) - Open Finished: 1s195ms (980.935ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.17 GB, 52.68 GB, 59.56 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.33 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.18 GB, 146.72 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157577986486) - PeakReservation: 146.72 GB (157538058240) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.48 GB (171236853582) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h55m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 980.918ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 58.991ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 980.926ms - ExecTreeOpenTime: 920.723ms - PrepareTime: 214.175ms - ExecTreePrepareTime: 176.413us KuduTableSink:(Total: 19.314us, non-child: 19.314us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 920.897ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157538066432) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 29m4s - CumulativeAllocationBytes: 146.72 GB (157538058240) - CumulativeAllocations: 75.12K (75120) - PeakReservation: 146.72 GB (157538058240) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157538058240) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h55m, non-child: 48s401ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s048ms - PeakMemoryUsage: 19.53 MB (20482754) - RowsReturned: 575.38M (575375052) - RowsReturnedRate: 26.96 K/sec Buffer pool: - AllocTime: 2h54m - CumulativeAllocationBytes: 261.30 GB (280563834880) - CumulativeAllocations: 13.70M (13699406) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.63 GB, 53.22 GB, 60.17 GB, 66.30 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.35 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 920.633ms - TotalBytesReceived: 148.32 GB (159255336545) - TotalGetBatchTime: 5h55m - DataArrivalTimer: 5h54m SenderSide: - DeserializeRowBatchTime: 3h2m - NumBatchesArrived: 6.85M (6849703) - NumBatchesDeferred: 45.12K (45122) - NumBatchesEnqueued: 6.85M (6849703) - NumBatchesReceived: 6.85M (6849703) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 58.677ms, non-child: 58.677ms, % non-child: 100.00%) - CodegenTime: 1.985ms - CompileTime: 4.452ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 29.121ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.578ms Instance 844a0200d7876345:20bb38b9000000f4 (host=vd1140.foo.com:22000):(Total: 1s194ms, non-child: 205.508ms, % non-child: 17.20%) Fragment Instance Lifecycle Event Timeline: 1s195ms - Prepare Finished: 205.086ms (205.086ms) - Open Finished: 1s195ms (990.602ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.18 GB, 52.70 GB, 59.57 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.35 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.97 GB, 111.18 GB, 114.29 GB, 117.23 GB, 120.06 GB, 122.82 GB, 125.43 GB, 127.93 GB, 130.53 GB, 133.03 GB, 135.44 GB, 137.89 GB, 140.31 GB, 142.75 GB, 145.19 GB, 146.73 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.77 GB (157592666550) - PeakReservation: 146.73 GB (157552738304) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.48 GB (171236344158) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h55m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 990.587ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 54.991ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 990.590ms - ExecTreeOpenTime: 932.857ms - PrepareTime: 204.304ms - ExecTreePrepareTime: 185.966us KuduTableSink:(Total: 12.141us, non-child: 12.141us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 933.039ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.73 GB (157552746496) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 29m6s - CumulativeAllocationBytes: 146.73 GB (157552738304) - CumulativeAllocations: 75.13K (75127) - PeakReservation: 146.73 GB (157552738304) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.73 GB (157552738304) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h56m, non-child: 48s961ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s259ms - PeakMemoryUsage: 19.53 MB (20483513) - RowsReturned: 575.42M (575421252) - RowsReturnedRate: 26.91 K/sec Buffer pool: - AllocTime: 2h13m - CumulativeAllocationBytes: 261.32 GB (280586362880) - CumulativeAllocations: 13.70M (13700506) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.37 GB, 94.32 GB, 98.19 GB, 102.01 GB, 105.60 GB, 109.10 GB, 112.36 GB, 115.50 GB, 118.47 GB, 121.33 GB, 124.12 GB, 126.76 GB, 129.29 GB, 131.91 GB, 134.44 GB, 136.88 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.74 GB, 148.29 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB - FirstBatchArrivalWaitTime: 932.763ms - TotalBytesReceived: 148.33 GB (159268101051) - TotalGetBatchTime: 5h55m - DataArrivalTimer: 5h55m SenderSide: - DeserializeRowBatchTime: 2h21m - NumBatchesArrived: 6.85M (6850253) - NumBatchesDeferred: 56.78K (56781) - NumBatchesEnqueued: 6.85M (6850253) - NumBatchesReceived: 6.85M (6850253) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.410ms, non-child: 56.410ms, % non-child: 100.00%) - CodegenTime: 1.215ms - CompileTime: 5.076ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.914ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 18.882ms Instance 844a0200d7876345:20bb38b900000094 (host=vc1502.foo.com:22000):(Total: 1s194ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h18m - Prepare Finished: 211.654ms (211.654ms) - Open Finished: 1s194ms (982.892ms) - First Batch Produced: 7h18m (7h18m) - First Batch Sent: 7h18m (16.218ms) MemoryUsage(8m32s): 22.81 GB, 40.35 GB, 49.69 GB, 57.96 GB, 65.53 GB, 72.19 GB, 77.96 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.35 GB, 102.66 GB, 106.87 GB, 111.02 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.70 GB, 128.93 GB, 132.05 GB, 135.08 GB, 137.96 GB, 140.70 GB, 143.56 GB, 146.31 GB, 148.97 GB, 151.66 GB, 154.32 GB, 157.00 GB, 159.69 GB, 161.38 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.37 GB (188304511172) - RowsProduced: 248.91M (248913704) - TotalNetworkReceiveTime: 5h49m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 982.866ms - TotalThreadsSysTime: 2.999ms - TotalThreadsUserTime: 59.991ms - TotalThreadsVoluntaryContextSwitches: 10 (10) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h18m - OpenTime: 982.868ms - ExecTreeOpenTime: 919.730ms - PrepareTime: 211.305ms - ExecTreePrepareTime: 166.689us KuduTableSink:(Total: 42m13s, non-child: 42m13s, % non-child: 100.00%) - KuduApplyTimer: 34m54s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 98.25 K/sec - TotalNumRows: 248.91M (248912680) SORT_NODE (id=2):(Total: 7h18m, non-child: 1h29m, % non-child: 20.39%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 35m22s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 248.91M (248913704) - RowsReturnedRate: 9.45 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276760543) Buffer pool: - AllocTime: 33m38s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h49m, non-child: 51s352ms, % non-child: 0.24%) - ConvertRowBatchTime: 21s194ms - PeakMemoryUsage: 19.53 MB (20481204) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.18 K/sec Buffer pool: - AllocTime: 2h59m - CumulativeAllocationBytes: 287.42 GB (308611522560) - CumulativeAllocations: 15.07M (15068922) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.05 GB, 40.76 GB, 50.20 GB, 58.56 GB, 66.21 GB, 72.95 GB, 78.77 GB, 84.27 GB, 89.50 GB, 94.64 GB, 99.39 GB, 103.75 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.03 GB, 130.31 GB, 133.45 GB, 136.51 GB, 139.43 GB, 142.20 GB, 145.09 GB, 147.87 GB, 150.55 GB, 153.28 GB, 155.97 GB, 158.68 GB, 161.40 GB, 163.10 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 919.674ms - TotalBytesReceived: 163.15 GB (175176409345) - TotalGetBatchTime: 5h48m - DataArrivalTimer: 5h48m SenderSide: - DeserializeRowBatchTime: 3h11m - NumBatchesArrived: 7.53M (7534461) - NumBatchesDeferred: 98.12K (98121) - NumBatchesEnqueued: 7.53M (7534461) - NumBatchesReceived: 7.53M (7534461) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 61.982ms, non-child: 61.982ms, % non-child: 100.00%) - CodegenTime: 978.687us - CompileTime: 6.193ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 29.938ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 25.291ms Instance 844a0200d7876345:20bb38b9000000fe (host=vc1513.foo.com:22000):(Total: 1s192ms, non-child: 217.965ms, % non-child: 18.28%) Fragment Instance Lifecycle Event Timeline: 1s192ms - Prepare Finished: 217.258ms (217.258ms) - Open Finished: 1s192ms (975.544ms) MemoryUsage(8m32s): 20.72 GB, 36.67 GB, 45.17 GB, 52.69 GB, 59.56 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.96 GB, 111.18 GB, 114.28 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.43 GB, 127.93 GB, 130.52 GB, 133.02 GB, 135.44 GB, 137.88 GB, 140.31 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB, 146.77 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.77 GB (157588472246) - PeakReservation: 146.73 GB (157548544000) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.65 GB (172491275414) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h49m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 975.529ms - TotalThreadsSysTime: 4.000ms - TotalThreadsUserTime: 48.992ms - TotalThreadsVoluntaryContextSwitches: 25 (25) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 975.531ms - ExecTreeOpenTime: 921.311ms - PrepareTime: 216.873ms - ExecTreePrepareTime: 216.162us KuduTableSink:(Total: 12.065us, non-child: 12.065us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 921.525ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.73 GB (157548552192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 34m28s - CumulativeAllocationBytes: 146.73 GB (157548544000) - CumulativeAllocations: 75.12K (75125) - PeakReservation: 146.73 GB (157548544000) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.73 GB (157548544000) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h50m, non-child: 47s998ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s694ms - PeakMemoryUsage: 19.53 MB (20483786) - RowsReturned: 575.41M (575409744) - RowsReturnedRate: 27.39 K/sec Buffer pool: - AllocTime: 3h32m - CumulativeAllocationBytes: 261.31 GB (280580751360) - CumulativeAllocations: 13.70M (13700232) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.04 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.60 GB, 76.60 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.12 GB, 126.76 GB, 129.29 GB, 131.91 GB, 134.44 GB, 136.88 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.74 GB, 148.29 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB - FirstBatchArrivalWaitTime: 921.223ms - TotalBytesReceived: 148.33 GB (159264991162) - TotalGetBatchTime: 5h49m - DataArrivalTimer: 5h49m SenderSide: - DeserializeRowBatchTime: 3h48m - NumBatchesArrived: 6.85M (6850116) - NumBatchesDeferred: 102.04K (102040) - NumBatchesEnqueued: 6.85M (6850116) - NumBatchesReceived: 6.85M (6850116) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 52.956ms, non-child: 52.956ms, % non-child: 100.00%) - CodegenTime: 1.022ms - CompileTime: 4.289ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.067ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 20.091ms Instance 844a0200d7876345:20bb38b9000000c7 (host=va1034.foo.com:22000):(Total: 1s192ms, non-child: 212.211ms, % non-child: 17.80%) Fragment Instance Lifecycle Event Timeline: 1s192ms - Prepare Finished: 211.300ms (211.300ms) - Open Finished: 1s192ms (980.951ms) MemoryUsage(8m32s): 20.73 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.28 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.43 GB, 127.93 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.18 GB, 146.72 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157577986486) - PeakReservation: 146.72 GB (157538058240) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.71 GB (172565125624) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h58m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 980.916ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 51.992ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 980.930ms - ExecTreeOpenTime: 926.393ms - PrepareTime: 211.018ms - ExecTreePrepareTime: 183.538us KuduTableSink:(Total: 20.795us, non-child: 20.795us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 926.574ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157538066432) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 26m14s - CumulativeAllocationBytes: 146.72 GB (157538058240) - CumulativeAllocations: 75.12K (75120) - PeakReservation: 146.72 GB (157538058240) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157538058240) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h58m, non-child: 48s062ms, % non-child: 0.22%) - ConvertRowBatchTime: 20s038ms - PeakMemoryUsage: 19.53 MB (20482304) - RowsReturned: 575.37M (575373960) - RowsReturnedRate: 26.72 K/sec Buffer pool: - AllocTime: 1h58m - CumulativeAllocationBytes: 261.29 GB (280563302400) - CumulativeAllocations: 13.70M (13699380) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.95 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.19 GB, 66.32 GB, 71.62 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.37 GB, 94.33 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.76 GB, 129.28 GB, 131.91 GB, 134.44 GB, 136.87 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 926.278ms - TotalBytesReceived: 148.32 GB (159255211264) - TotalGetBatchTime: 5h58m - DataArrivalTimer: 5h58m SenderSide: - DeserializeRowBatchTime: 2h7m - NumBatchesArrived: 6.85M (6849690) - NumBatchesDeferred: 26.95K (26955) - NumBatchesEnqueued: 6.85M (6849690) - NumBatchesReceived: 6.85M (6849690) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 53.202ms, non-child: 53.202ms, % non-child: 100.00%) - CodegenTime: 1.145ms - CompileTime: 5.734ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.376ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 18.601ms Instance 844a0200d7876345:20bb38b9000000dd (host=vd1338.foo.com:22000):(Total: 1s191ms, non-child: 218.322ms, % non-child: 18.32%) Fragment Instance Lifecycle Event Timeline: 1s191ms - Prepare Finished: 217.525ms (217.525ms) - Open Finished: 1s191ms (974.229ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.19 GB, 52.70 GB, 59.58 GB, 65.64 GB, 70.88 GB, 75.83 GB, 80.54 GB, 85.16 GB, 89.43 GB, 93.35 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.28 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.43 GB, 127.93 GB, 130.52 GB, 133.03 GB, 135.44 GB, 137.89 GB, 140.31 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157586375094) - PeakReservation: 146.73 GB (157546446848) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.35 GB (171106113336) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h54m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 974.215ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 61.990ms - TotalThreadsVoluntaryContextSwitches: 8 (8) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 974.218ms - ExecTreeOpenTime: 908.846ms - PrepareTime: 217.111ms - ExecTreePrepareTime: 168.524us KuduTableSink:(Total: 17.032us, non-child: 17.032us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 909.012ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.73 GB (157546455040) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 30m51s - CumulativeAllocationBytes: 146.73 GB (157546446848) - CumulativeAllocations: 75.12K (75124) - PeakReservation: 146.73 GB (157546446848) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.73 GB (157546446848) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h54m, non-child: 48s183ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s856ms - PeakMemoryUsage: 19.53 MB (20483081) - RowsReturned: 575.40M (575402436) - RowsReturnedRate: 27.04 K/sec Buffer pool: - AllocTime: 3h23m - CumulativeAllocationBytes: 261.31 GB (280577187840) - CumulativeAllocations: 13.70M (13700058) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.19 GB, 66.32 GB, 71.62 GB, 76.61 GB, 81.38 GB, 86.05 GB, 90.37 GB, 94.33 GB, 98.19 GB, 102.01 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.76 GB, 129.28 GB, 131.91 GB, 134.44 GB, 136.88 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.74 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB, 148.33 GB - FirstBatchArrivalWaitTime: 908.772ms - TotalBytesReceived: 148.33 GB (159263094250) - TotalGetBatchTime: 5h54m - DataArrivalTimer: 5h53m SenderSide: - DeserializeRowBatchTime: 3h31m - NumBatchesArrived: 6.85M (6850029) - NumBatchesDeferred: 58.23K (58229) - NumBatchesEnqueued: 6.85M (6850029) - NumBatchesReceived: 6.85M (6850029) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 64.040ms, non-child: 64.040ms, % non-child: 100.00%) - CodegenTime: 1.492ms - CompileTime: 6.012ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 33.264ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.122ms Instance 844a0200d7876345:20bb38b9000000c9 (host=vc1516.foo.com:22000):(Total: 1s190ms, non-child: 219.779ms, % non-child: 18.46%) Fragment Instance Lifecycle Event Timeline: 1s190ms - Prepare Finished: 218.943ms (218.943ms) - Open Finished: 1s190ms (971.898ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.19 GB, 52.70 GB, 59.58 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.49 GB, 107.96 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.43 GB, 137.87 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157575889334) - PeakReservation: 146.72 GB (157535961088) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.69 GB (172540772510) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h50m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 971.882ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 54.991ms - TotalThreadsVoluntaryContextSwitches: 18 (18) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 971.885ms - ExecTreeOpenTime: 914.968ms - PrepareTime: 218.540ms - ExecTreePrepareTime: 190.469us KuduTableSink:(Total: 13.675us, non-child: 13.675us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 915.156ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157535969280) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 32m27s - CumulativeAllocationBytes: 146.72 GB (157535961088) - CumulativeAllocations: 75.12K (75119) - PeakReservation: 146.72 GB (157535961088) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157535961088) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h51m, non-child: 47s799ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s886ms - PeakMemoryUsage: 19.53 MB (20482979) - RowsReturned: 575.36M (575360856) - RowsReturnedRate: 27.29 K/sec Buffer pool: - AllocTime: 2h45m - CumulativeAllocationBytes: 261.29 GB (280556912640) - CumulativeAllocations: 13.70M (13699068) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.64 GB, 53.24 GB, 60.19 GB, 66.32 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.59 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 914.886ms - TotalBytesReceived: 148.31 GB (159251651019) - TotalGetBatchTime: 5h50m - DataArrivalTimer: 5h50m SenderSide: - DeserializeRowBatchTime: 2h55m - NumBatchesArrived: 6.85M (6849534) - NumBatchesDeferred: 68.47K (68469) - NumBatchesEnqueued: 6.85M (6849534) - NumBatchesReceived: 6.85M (6849534) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 55.534ms, non-child: 55.534ms, % non-child: 100.00%) - CodegenTime: 1.000ms - CompileTime: 5.999ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 29.223ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.716ms Instance 844a0200d7876345:20bb38b900000101 (host=vc1529.foo.com:22000):(Total: 1s189ms, non-child: 217.092ms, % non-child: 18.25%) Fragment Instance Lifecycle Event Timeline: 1s190ms - Prepare Finished: 216.364ms (216.364ms) - Open Finished: 1s190ms (973.777ms) MemoryUsage(8m32s): 20.75 GB, 36.69 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157573792182) - PeakReservation: 146.71 GB (157533863936) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.71 GB (172557724564) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h54m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 973.762ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 49.992ms - TotalThreadsVoluntaryContextSwitches: 14 (14) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 973.765ms - ExecTreeOpenTime: 921.944ms - PrepareTime: 215.983ms - ExecTreePrepareTime: 165.996us KuduTableSink:(Total: 11.914us, non-child: 11.914us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 922.107ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157533872128) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 29m35s - CumulativeAllocationBytes: 146.71 GB (157533863936) - CumulativeAllocations: 75.12K (75118) - PeakReservation: 146.71 GB (157533863936) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157533863936) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h55m, non-child: 47s576ms, % non-child: 0.22%) - ConvertRowBatchTime: 20s054ms - PeakMemoryUsage: 19.53 MB (20483286) - RowsReturned: 575.35M (575354892) - RowsReturnedRate: 27.00 K/sec Buffer pool: - AllocTime: 2h41m - CumulativeAllocationBytes: 261.29 GB (280554004480) - CumulativeAllocations: 13.70M (13698926) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.36 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 921.883ms - TotalBytesReceived: 148.31 GB (159250138230) - TotalGetBatchTime: 5h54m - DataArrivalTimer: 5h54m SenderSide: - DeserializeRowBatchTime: 2h49m - NumBatchesArrived: 6.85M (6849463) - NumBatchesDeferred: 55.75K (55748) - NumBatchesEnqueued: 6.85M (6849463) - NumBatchesReceived: 6.85M (6849463) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 50.588ms, non-child: 50.588ms, % non-child: 100.00%) - CodegenTime: 974.848us - CompileTime: 5.406ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.117ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 16.550ms Instance 844a0200d7876345:20bb38b9000000e8 (host=vc1527.foo.com:22000):(Total: 1s189ms, non-child: 204.232ms, % non-child: 17.17%) Fragment Instance Lifecycle Event Timeline: 1s189ms - Prepare Finished: 203.273ms (203.273ms) - Open Finished: 1s189ms (986.397ms) MemoryUsage(8m32s): 20.73 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.96 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.18 GB, 146.72 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157577986486) - PeakReservation: 146.72 GB (157538058240) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.68 GB (172525369267) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h58m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 986.380ms - TotalThreadsSysTime: 999.000us - TotalThreadsUserTime: 58.991ms - TotalThreadsVoluntaryContextSwitches: 8 (8) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 986.384ms - ExecTreeOpenTime: 926.169ms - PrepareTime: 202.876ms - ExecTreePrepareTime: 161.279us KuduTableSink:(Total: 17.805us, non-child: 17.805us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 926.328ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157538066432) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 24m50s - CumulativeAllocationBytes: 146.72 GB (157538058240) - CumulativeAllocations: 75.12K (75120) - PeakReservation: 146.72 GB (157538058240) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157538058240) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h58m, non-child: 47s782ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s903ms - PeakMemoryUsage: 19.53 MB (20483351) - RowsReturned: 575.38M (575375052) - RowsReturnedRate: 26.73 K/sec Buffer pool: - AllocTime: 2h41m - CumulativeAllocationBytes: 261.30 GB (280563834880) - CumulativeAllocations: 13.70M (13699406) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.92 GB, 37.04 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.18 GB, 102.00 GB, 105.59 GB, 109.09 GB, 112.35 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 926.113ms - TotalBytesReceived: 148.32 GB (159255155383) - TotalGetBatchTime: 5h58m - DataArrivalTimer: 5h57m SenderSide: - DeserializeRowBatchTime: 2h51m - NumBatchesArrived: 6.85M (6849703) - NumBatchesDeferred: 51.21K (51206) - NumBatchesEnqueued: 6.85M (6849703) - NumBatchesReceived: 6.85M (6849703) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 58.737ms, non-child: 58.737ms, % non-child: 100.00%) - CodegenTime: 1.282ms - CompileTime: 5.420ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.972ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.738ms Instance 844a0200d7876345:20bb38b900000083 (host=vd1118.foo.com:22000):(Total: 1s188ms, non-child: 208.134ms, % non-child: 17.51%) Fragment Instance Lifecycle Event Timeline: 1s189ms - Prepare Finished: 207.642ms (207.642ms) - Open Finished: 1s189ms (981.568ms) MemoryUsage(8m32s): 22.79 GB, 40.34 GB, 49.69 GB, 57.95 GB, 65.52 GB, 72.19 GB, 77.95 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.35 GB, 102.66 GB, 106.87 GB, 111.02 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.69 GB, 128.93 GB, 132.05 GB, 135.07 GB, 137.96 GB, 140.70 GB, 143.56 GB, 146.31 GB, 148.96 GB, 151.66 GB, 154.32 GB, 156.99 GB, 159.69 GB, 161.37 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173323403702) - PeakReservation: 161.38 GB (173283475456) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.36 GB (188294284108) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h52m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 981.551ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 59.991ms - TotalThreadsVoluntaryContextSwitches: 5 (5) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 981.553ms - ExecTreeOpenTime: 919.414ms - PrepareTime: 207.055ms - ExecTreePrepareTime: 169.788us KuduTableSink:(Total: 15.008us, non-child: 15.008us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 919.581ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173283483648) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 30m11s - CumulativeAllocationBytes: 161.38 GB (173283475456) - CumulativeAllocations: 82.63K (82628) - PeakReservation: 161.38 GB (173283475456) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173283475456) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h52m, non-child: 49s871ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s584ms - PeakMemoryUsage: 19.53 MB (20481819) - RowsReturned: 632.88M (632876496) - RowsReturnedRate: 29.88 K/sec Buffer pool: - AllocTime: 2h23m - CumulativeAllocationBytes: 287.41 GB (308602634240) - CumulativeAllocations: 15.07M (15068488) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.75 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.64 GB, 99.39 GB, 103.75 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.03 GB, 130.30 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.09 GB, 147.87 GB, 150.55 GB, 153.27 GB, 155.97 GB, 158.67 GB, 161.39 GB, 163.09 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 919.343ms - TotalBytesReceived: 163.14 GB (175170914831) - TotalGetBatchTime: 5h52m - DataArrivalTimer: 5h52m SenderSide: - DeserializeRowBatchTime: 2h33m - NumBatchesArrived: 7.53M (7534244) - NumBatchesDeferred: 59.13K (59130) - NumBatchesEnqueued: 7.53M (7534244) - NumBatchesReceived: 7.53M (7534244) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 60.940ms, non-child: 60.940ms, % non-child: 100.00%) - CodegenTime: 1.246ms - CompileTime: 5.040ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 30.892ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.331ms Instance 844a0200d7876345:20bb38b9000000c2 (host=vc1518.foo.com:22000):(Total: 1s188ms, non-child: 217.705ms, % non-child: 18.32%) Fragment Instance Lifecycle Event Timeline: 1s188ms - Prepare Finished: 216.999ms (216.999ms) - Open Finished: 1s188ms (971.756ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.64 GB, 70.87 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.04 GB, 122.80 GB, 125.42 GB, 127.91 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.72 GB, 145.17 GB, 146.71 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157565403574) - PeakReservation: 146.71 GB (157525475328) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.62 GB (172468846769) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h50m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 971.741ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 54.992ms - TotalThreadsVoluntaryContextSwitches: 11 (11) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 971.744ms - ExecTreeOpenTime: 913.703ms - PrepareTime: 216.546ms - ExecTreePrepareTime: 158.827us KuduTableSink:(Total: 12.118us, non-child: 12.118us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 913.859ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157525483520) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 31m49s - CumulativeAllocationBytes: 146.71 GB (157525475328) - CumulativeAllocations: 75.11K (75114) - PeakReservation: 146.71 GB (157525475328) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157525475328) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h51m, non-child: 47s728ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s759ms - PeakMemoryUsage: 19.54 MB (20484098) - RowsReturned: 575.33M (575329272) - RowsReturnedRate: 27.31 K/sec Buffer pool: - AllocTime: 2h59m - CumulativeAllocationBytes: 261.27 GB (280541511680) - CumulativeAllocations: 13.70M (13698316) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.04 GB, 45.64 GB, 53.23 GB, 60.19 GB, 66.32 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.36 GB, 94.32 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.34 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.26 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 913.640ms - TotalBytesReceived: 148.31 GB (159242565940) - TotalGetBatchTime: 5h50m - DataArrivalTimer: 5h50m SenderSide: - DeserializeRowBatchTime: 3h9m - NumBatchesArrived: 6.85M (6849158) - NumBatchesDeferred: 63.15K (63147) - NumBatchesEnqueued: 6.85M (6849158) - NumBatchesReceived: 6.85M (6849158) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.812ms, non-child: 56.812ms, % non-child: 100.00%) - CodegenTime: 1.018ms - CompileTime: 4.923ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.447ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.873ms Instance 844a0200d7876345:20bb38b9000000fb (host=vd1131.foo.com:22000):(Total: 1s188ms, non-child: 214.321ms, % non-child: 18.03%) Fragment Instance Lifecycle Event Timeline: 1s188ms - Prepare Finished: 213.926ms (213.926ms) - Open Finished: 1s188ms (974.904ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.17 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.53 GB, 85.15 GB, 89.43 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.28 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.43 GB, 127.92 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157580083638) - PeakReservation: 146.72 GB (157540155392) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.40 GB (171150354649) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h55m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 974.893ms - TotalThreadsSysTime: 999.000us - TotalThreadsUserTime: 55.991ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 974.896ms - ExecTreeOpenTime: 917.240ms - PrepareTime: 213.424ms - ExecTreePrepareTime: 200.753us KuduTableSink:(Total: 15.389us, non-child: 15.389us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 917.437ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157540163584) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 27m26s - CumulativeAllocationBytes: 146.72 GB (157540155392) - CumulativeAllocations: 75.12K (75121) - PeakReservation: 146.72 GB (157540155392) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157540155392) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h56m, non-child: 48s548ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s076ms - PeakMemoryUsage: 19.54 MB (20484067) - RowsReturned: 575.38M (575381856) - RowsReturnedRate: 26.91 K/sec Buffer pool: - AllocTime: 3h9m - CumulativeAllocationBytes: 261.30 GB (280567152640) - CumulativeAllocations: 13.70M (13699568) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.62 GB, 53.22 GB, 60.18 GB, 66.31 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 917.177ms - TotalBytesReceived: 148.32 GB (159257457297) - TotalGetBatchTime: 5h55m - DataArrivalTimer: 5h55m SenderSide: - DeserializeRowBatchTime: 3h18m - NumBatchesArrived: 6.85M (6849784) - NumBatchesDeferred: 39.84K (39836) - NumBatchesEnqueued: 6.85M (6849784) - NumBatchesReceived: 6.85M (6849784) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.607ms, non-child: 56.607ms, % non-child: 100.00%) - CodegenTime: 773.452us - CompileTime: 4.759ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 27.652ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.726ms Instance 844a0200d7876345:20bb38b9000000ba (host=ve1137.foo.com:22000):(Total: 1s187ms, non-child: 217.902ms, % non-child: 18.35%) Fragment Instance Lifecycle Event Timeline: 1s188ms - Prepare Finished: 216.907ms (216.907ms) - Open Finished: 1s188ms (971.094ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.43 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.49 GB, 107.96 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.02 GB, 135.43 GB, 137.87 GB, 140.29 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157573792182) - PeakReservation: 146.71 GB (157533863936) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.44 GB (171194539085) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h54m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 971.072ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 53.991ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 971.075ms - ExecTreeOpenTime: 916.239ms - PrepareTime: 216.574ms - ExecTreePrepareTime: 177.385us KuduTableSink:(Total: 13.304us, non-child: 13.304us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 916.413ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157533872128) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 29m29s - CumulativeAllocationBytes: 146.71 GB (157533863936) - CumulativeAllocations: 75.12K (75118) - PeakReservation: 146.71 GB (157533863936) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157533863936) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h54m, non-child: 48s746ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s409ms - PeakMemoryUsage: 19.53 MB (20483622) - RowsReturned: 575.35M (575354136) - RowsReturnedRate: 27.05 K/sec Buffer pool: - AllocTime: 3h19m - CumulativeAllocationBytes: 261.29 GB (280553635840) - CumulativeAllocations: 13.70M (13698908) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.96 GB, 37.06 GB, 45.64 GB, 53.23 GB, 60.19 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.37 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 916.144ms - TotalBytesReceived: 148.31 GB (159249671708) - TotalGetBatchTime: 5h54m - DataArrivalTimer: 5h53m SenderSide: - DeserializeRowBatchTime: 3h30m - NumBatchesArrived: 6.85M (6849454) - NumBatchesDeferred: 66.98K (66977) - NumBatchesEnqueued: 6.85M (6849454) - NumBatchesReceived: 6.85M (6849454) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 53.400ms, non-child: 53.400ms, % non-child: 100.00%) - CodegenTime: 1.188ms - CompileTime: 4.605ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 27.259ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 20.941ms Instance 844a0200d7876345:20bb38b900000087 (host=va1044.foo.com:22000):(Total: 1s187ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 4h52m - Prepare Finished: 212.029ms (212.029ms) - Open Finished: 1s187ms (975.821ms) - First Batch Produced: 4h52m (4h52m) - First Batch Sent: 4h52m (10.817ms) MemoryUsage(8m32s): 22.79 GB, 40.35 GB, 49.69 GB, 57.96 GB, 65.52 GB, 72.19 GB, 77.96 GB, 83.39 GB, 88.58 GB, 93.67 GB, 98.36 GB, 102.67 GB, 106.88 GB, 111.03 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.09 GB, 137.97 GB, 140.72 GB, 143.57 GB, 146.33 GB, 148.98 GB, 151.67 GB, 154.34 GB, 157.01 GB, 159.71 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.37 GB (188304308648) - RowsProduced: 632.89M (632891204) - TotalNetworkReceiveTime: 3h21m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 975.802ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 60.991ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 4h52m - OpenTime: 975.805ms - ExecTreeOpenTime: 914.813ms - PrepareTime: 211.617ms - ExecTreePrepareTime: 155.127us KuduTableSink:(Total: 1h49m, non-child: 1h49m, % non-child: 100.00%) - KuduApplyTimer: 1h28m - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 96.50 K/sec - TotalNumRows: 632.89M (632891204) SORT_NODE (id=2):(Total: 4h52m, non-child: 1h30m, % non-child: 31.02%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 35m8s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 632.89M (632891204) - RowsReturnedRate: 36.05 K/sec - RunsCreated: 2 (2) - SortDataSize: 161.38 GB (173276756591) Buffer pool: - AllocTime: 35m6s - CumulativeAllocationBytes: 161.40 GB (173306544128) - CumulativeAllocations: 82.64K (82639) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 3h21m, non-child: 50s879ms, % non-child: 0.42%) - ConvertRowBatchTime: 21s131ms - PeakMemoryUsage: 19.53 MB (20482244) - RowsReturned: 632.95M (632952684) - RowsReturnedRate: 52.27 K/sec Buffer pool: - AllocTime: 4h4m - CumulativeAllocationBytes: 287.44 GB (308639784960) - CumulativeAllocations: 15.07M (15070302) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.04 GB, 40.76 GB, 50.20 GB, 58.56 GB, 66.21 GB, 72.95 GB, 78.77 GB, 84.27 GB, 89.51 GB, 94.65 GB, 99.40 GB, 103.76 GB, 108.01 GB, 112.21 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.05 GB, 130.32 GB, 133.47 GB, 136.53 GB, 139.44 GB, 142.22 GB, 145.10 GB, 147.89 GB, 150.57 GB, 153.29 GB, 155.98 GB, 158.69 GB, 161.41 GB, 163.12 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB, 163.16 GB - FirstBatchArrivalWaitTime: 914.741ms - TotalBytesReceived: 163.16 GB (175192443035) - TotalGetBatchTime: 3h21m - DataArrivalTimer: 3h21m SenderSide: - DeserializeRowBatchTime: 4h19m - NumBatchesArrived: 7.54M (7535151) - NumBatchesDeferred: 193.48K (193477) - NumBatchesEnqueued: 7.54M (7535151) - NumBatchesReceived: 7.54M (7535151) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 59.521ms, non-child: 59.521ms, % non-child: 100.00%) - CodegenTime: 1.173ms - CompileTime: 5.552ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.565ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 20.802ms Instance 844a0200d7876345:20bb38b9000000b4 (host=vd1125.foo.com:22000):(Total: 1s187ms, non-child: 217.230ms, % non-child: 18.30%) Fragment Instance Lifecycle Event Timeline: 1s187ms - Prepare Finished: 216.315ms (216.315ms) - Open Finished: 1s187ms (971.175ms) MemoryUsage(8m32s): 20.75 GB, 36.69 GB, 45.18 GB, 52.70 GB, 59.58 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157575889334) - PeakReservation: 146.72 GB (157535961088) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.67 GB (172519742292) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h55m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 971.155ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 59.991ms - TotalThreadsVoluntaryContextSwitches: 9 (9) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 971.158ms - ExecTreeOpenTime: 908.706ms - PrepareTime: 215.913ms - ExecTreePrepareTime: 162.231us KuduTableSink:(Total: 16.081us, non-child: 16.081us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 908.865ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157535969280) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 29m14s - CumulativeAllocationBytes: 146.72 GB (157535961088) - CumulativeAllocations: 75.12K (75119) - PeakReservation: 146.72 GB (157535961088) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157535961088) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h56m, non-child: 49s359ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s567ms - PeakMemoryUsage: 19.54 MB (20484166) - RowsReturned: 575.36M (575364384) - RowsReturnedRate: 26.92 K/sec Buffer pool: - AllocTime: 2h35m - CumulativeAllocationBytes: 261.29 GB (280558632960) - CumulativeAllocations: 13.70M (13699152) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.63 GB, 53.23 GB, 60.19 GB, 66.32 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 908.632ms - TotalBytesReceived: 148.32 GB (159252254847) - TotalGetBatchTime: 5h55m - DataArrivalTimer: 5h55m SenderSide: - DeserializeRowBatchTime: 2h44m - NumBatchesArrived: 6.85M (6849576) - NumBatchesDeferred: 39.90K (39902) - NumBatchesEnqueued: 6.85M (6849576) - NumBatchesReceived: 6.85M (6849576) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 61.022ms, non-child: 61.022ms, % non-child: 100.00%) - CodegenTime: 1.123ms - CompileTime: 4.658ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.853ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.847ms Instance 844a0200d7876345:20bb38b9000000fc (host=vd1126.foo.com:22000):(Total: 1s186ms, non-child: 211.258ms, % non-child: 17.80%) Fragment Instance Lifecycle Event Timeline: 1s187ms - Prepare Finished: 210.286ms (210.286ms) - Open Finished: 1s187ms (976.975ms) MemoryUsage(8m32s): 20.73 GB, 36.68 GB, 45.17 GB, 52.68 GB, 59.56 GB, 65.62 GB, 70.86 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.33 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.04 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.73 GB, 145.17 GB, 146.71 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157567500726) - PeakReservation: 146.71 GB (157527572480) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.70 GB (172551802940) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h55m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 976.956ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 64.989ms - TotalThreadsVoluntaryContextSwitches: 11 (11) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 976.958ms - ExecTreeOpenTime: 910.433ms - PrepareTime: 209.941ms - ExecTreePrepareTime: 155.032us KuduTableSink:(Total: 19.458us, non-child: 19.458us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 910.585ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157527580672) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 29m58s - CumulativeAllocationBytes: 146.71 GB (157527572480) - CumulativeAllocations: 75.11K (75115) - PeakReservation: 146.71 GB (157527572480) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157527572480) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h55m, non-child: 48s221ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s235ms - PeakMemoryUsage: 19.53 MB (20482872) - RowsReturned: 575.33M (575333976) - RowsReturnedRate: 26.96 K/sec Buffer pool: - AllocTime: 3h4m - CumulativeAllocationBytes: 261.28 GB (280543805440) - CumulativeAllocations: 13.70M (13698428) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.95 GB, 37.05 GB, 45.63 GB, 53.22 GB, 60.17 GB, 66.30 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.90 GB, 134.42 GB, 136.86 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.72 GB, 148.27 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 910.336ms - TotalBytesReceived: 148.31 GB (159243727751) - TotalGetBatchTime: 5h55m - DataArrivalTimer: 5h54m SenderSide: - DeserializeRowBatchTime: 3h12m - NumBatchesArrived: 6.85M (6849214) - NumBatchesDeferred: 55.99K (55989) - NumBatchesEnqueued: 6.85M (6849214) - NumBatchesReceived: 6.85M (6849214) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 65.080ms, non-child: 65.080ms, % non-child: 100.00%) - CodegenTime: 1.429ms - CompileTime: 5.389ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 33.561ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 25.430ms Instance 844a0200d7876345:20bb38b9000000f7 (host=vd1134.foo.com:22000):(Total: 1s186ms, non-child: 212.490ms, % non-child: 17.91%) Fragment Instance Lifecycle Event Timeline: 1s187ms - Prepare Finished: 211.766ms (211.766ms) - Open Finished: 1s187ms (975.255ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.33 GB, 97.16 GB, 100.93 GB, 104.48 GB, 107.95 GB, 111.16 GB, 114.27 GB, 117.21 GB, 120.04 GB, 122.80 GB, 125.41 GB, 127.91 GB, 130.50 GB, 133.00 GB, 135.42 GB, 137.86 GB, 140.29 GB, 142.72 GB, 145.17 GB, 146.70 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157563306422) - PeakReservation: 146.71 GB (157523378176) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.44 GB (171193939379) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h57m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 975.233ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 54.991ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 975.236ms - ExecTreeOpenTime: 916.103ms - PrepareTime: 211.271ms - ExecTreePrepareTime: 186.093us KuduTableSink:(Total: 24.927us, non-child: 24.927us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 916.286ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157523386368) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 25m51s - CumulativeAllocationBytes: 146.71 GB (157523378176) - CumulativeAllocations: 75.11K (75113) - PeakReservation: 146.71 GB (157523378176) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157523378176) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h58m, non-child: 49s148ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s238ms - PeakMemoryUsage: 19.53 MB (20482873) - RowsReturned: 575.32M (575316336) - RowsReturnedRate: 26.75 K/sec Buffer pool: - AllocTime: 2h1m - CumulativeAllocationBytes: 261.27 GB (280535203840) - CumulativeAllocations: 13.70M (13698008) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.32 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.74 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.26 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB - FirstBatchArrivalWaitTime: 916.033ms - TotalBytesReceived: 148.30 GB (159239104660) - TotalGetBatchTime: 5h57m - DataArrivalTimer: 5h57m SenderSide: - DeserializeRowBatchTime: 2h11m - NumBatchesArrived: 6.85M (6849004) - NumBatchesDeferred: 21.55K (21548) - NumBatchesEnqueued: 6.85M (6849004) - NumBatchesReceived: 6.85M (6849004) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 57.763ms, non-child: 57.763ms, % non-child: 100.00%) - CodegenTime: 1.044ms - CompileTime: 4.543ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.540ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.119ms Instance 844a0200d7876345:20bb38b900000086 (host=vd1333.foo.com:22000):(Total: 1s186ms, non-child: 201.493ms, % non-child: 16.98%) Fragment Instance Lifecycle Event Timeline: 1s186ms - Prepare Finished: 200.700ms (200.700ms) - Open Finished: 1s186ms (986.107ms) MemoryUsage(8m32s): 22.81 GB, 40.34 GB, 49.68 GB, 57.95 GB, 65.51 GB, 72.18 GB, 77.95 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.36 GB, 102.67 GB, 106.87 GB, 111.02 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.70 GB, 128.94 GB, 132.05 GB, 135.08 GB, 137.95 GB, 140.70 GB, 143.55 GB, 146.31 GB, 148.96 GB, 151.65 GB, 154.32 GB, 156.99 GB, 159.69 GB, 161.37 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173319209398) - PeakReservation: 161.38 GB (173279281152) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.12 GB (186955879181) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h48m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 986.085ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 54.992ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 986.088ms - ExecTreeOpenTime: 927.852ms - PrepareTime: 200.286ms - ExecTreePrepareTime: 177.762us KuduTableSink:(Total: 13.568us, non-child: 13.568us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 928.027ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173279289344) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 34m46s - CumulativeAllocationBytes: 161.38 GB (173279281152) - CumulativeAllocations: 82.63K (82626) - PeakReservation: 161.38 GB (173279281152) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173279281152) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h48m, non-child: 50s928ms, % non-child: 0.24%) - ConvertRowBatchTime: 21s032ms - PeakMemoryUsage: 19.53 MB (20481910) - RowsReturned: 632.86M (632863224) - RowsReturnedRate: 30.26 K/sec Buffer pool: - AllocTime: 3h42m - CumulativeAllocationBytes: 287.40 GB (308596162560) - CumulativeAllocations: 15.07M (15068172) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.02 GB, 40.74 GB, 50.19 GB, 58.54 GB, 66.19 GB, 72.93 GB, 78.76 GB, 84.26 GB, 89.50 GB, 94.64 GB, 99.39 GB, 103.75 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.03 GB, 130.30 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.08 GB, 147.86 GB, 150.55 GB, 153.27 GB, 155.96 GB, 158.67 GB, 161.39 GB, 163.09 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 927.781ms - TotalBytesReceived: 163.14 GB (175166954381) - TotalGetBatchTime: 5h48m - DataArrivalTimer: 5h47m SenderSide: - DeserializeRowBatchTime: 3h53m - NumBatchesArrived: 7.53M (7534086) - NumBatchesDeferred: 172.40K (172400) - NumBatchesEnqueued: 7.53M (7534086) - NumBatchesReceived: 7.53M (7534086) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.896ms, non-child: 56.896ms, % non-child: 100.00%) - CodegenTime: 964.700us - CompileTime: 4.580ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.129ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.624ms Instance 844a0200d7876345:20bb38b9000000c1 (host=ve1142.foo.com:22000):(Total: 1s186ms, non-child: 212.815ms, % non-child: 17.94%) Fragment Instance Lifecycle Event Timeline: 1s186ms - Prepare Finished: 212.009ms (212.009ms) - Open Finished: 1s186ms (974.370ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.28 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.43 GB, 127.93 GB, 130.52 GB, 133.03 GB, 135.44 GB, 137.89 GB, 140.31 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157586375094) - PeakReservation: 146.73 GB (157546446848) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.43 GB (171185773237) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h54m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 974.351ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 50.992ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 974.355ms - ExecTreeOpenTime: 921.784ms - PrepareTime: 211.667ms - ExecTreePrepareTime: 157.005us KuduTableSink:(Total: 22.402us, non-child: 22.402us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 921.938ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.73 GB (157546455040) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 30m1s - CumulativeAllocationBytes: 146.73 GB (157546446848) - CumulativeAllocations: 75.12K (75124) - PeakReservation: 146.73 GB (157546446848) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.73 GB (157546446848) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h55m, non-child: 49s104ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s377ms - PeakMemoryUsage: 19.53 MB (20483478) - RowsReturned: 575.40M (575400756) - RowsReturnedRate: 27.01 K/sec Buffer pool: - AllocTime: 3h8m - CumulativeAllocationBytes: 261.31 GB (280576368640) - CumulativeAllocations: 13.70M (13700018) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.76 GB, 129.29 GB, 131.91 GB, 134.44 GB, 136.88 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 921.706ms - TotalBytesReceived: 148.32 GB (159262306046) - TotalGetBatchTime: 5h54m - DataArrivalTimer: 5h54m SenderSide: - DeserializeRowBatchTime: 3h17m - NumBatchesArrived: 6.85M (6850009) - NumBatchesDeferred: 77.05K (77046) - NumBatchesEnqueued: 6.85M (6850009) - NumBatchesReceived: 6.85M (6850009) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 51.311ms, non-child: 51.311ms, % non-child: 100.00%) - CodegenTime: 1.009ms - CompileTime: 4.394ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 26.736ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.691ms Instance 844a0200d7876345:20bb38b90000009c (host=vd1139.foo.com:22000):(Total: 1s185ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h14m - Prepare Finished: 221.121ms (221.121ms) - Open Finished: 1s185ms (964.553ms) - First Batch Produced: 7h14m (7h14m) - First Batch Sent: 7h14m (11.896ms) MemoryUsage(8m32s): 22.79 GB, 40.35 GB, 49.69 GB, 57.96 GB, 65.52 GB, 72.19 GB, 77.96 GB, 83.39 GB, 88.58 GB, 93.66 GB, 98.36 GB, 102.67 GB, 106.88 GB, 111.03 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.70 GB, 128.94 GB, 132.06 GB, 135.09 GB, 137.96 GB, 140.71 GB, 143.57 GB, 146.32 GB, 148.97 GB, 151.66 GB, 154.33 GB, 157.01 GB, 159.70 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173326116751) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.01 GB (186843523729) - RowsProduced: 231.13M (231134050) - TotalNetworkReceiveTime: 5h46m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 964.529ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 63.990ms - TotalThreadsVoluntaryContextSwitches: 7 (7) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h14m - OpenTime: 964.532ms - ExecTreeOpenTime: 897.960ms - PrepareTime: 220.727ms - ExecTreePrepareTime: 164.686us KuduTableSink:(Total: 47m8s, non-child: 47m8s, % non-child: 100.00%) - KuduApplyTimer: 37m11s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 81.72 K/sec - TotalNumRows: 231.13M (231133026) SORT_NODE (id=2):(Total: 7h14m, non-child: 1h27m, % non-child: 20.07%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 30m30s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 231.13M (231134050) - RowsReturnedRate: 8.88 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276233531) Buffer pool: - AllocTime: 36m4s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h46m, non-child: 50s678ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s884ms - PeakMemoryUsage: 19.53 MB (20482285) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.40 K/sec Buffer pool: - AllocTime: 3h4m - CumulativeAllocationBytes: 287.43 GB (308628398080) - CumulativeAllocations: 15.07M (15069746) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.76 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.27 GB, 89.51 GB, 94.65 GB, 99.40 GB, 103.75 GB, 108.01 GB, 112.20 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.32 GB, 133.46 GB, 136.52 GB, 139.43 GB, 142.21 GB, 145.10 GB, 147.88 GB, 150.56 GB, 153.28 GB, 155.98 GB, 158.68 GB, 161.41 GB, 163.11 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 897.877ms - TotalBytesReceived: 163.15 GB (175185458180) - TotalGetBatchTime: 5h46m - DataArrivalTimer: 5h46m SenderSide: - DeserializeRowBatchTime: 3h16m - NumBatchesArrived: 7.53M (7534924) - NumBatchesDeferred: 148.23K (148231) - NumBatchesEnqueued: 7.53M (7534873) - NumBatchesReceived: 7.53M (7534873) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 65.227ms, non-child: 65.227ms, % non-child: 100.00%) - CodegenTime: 1.226ms - CompileTime: 5.237ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.493ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 26.805ms Instance 844a0200d7876345:20bb38b900000085 (host=vd1112.foo.com:22000):(Total: 1s184ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 4h49m - Prepare Finished: 217.746ms (217.746ms) - Open Finished: 1s184ms (966.900ms) - First Batch Produced: 4h49m (4h49m) - First Batch Sent: 4h49m (13.322ms) MemoryUsage(8m32s): 22.80 GB, 40.34 GB, 49.69 GB, 57.95 GB, 65.52 GB, 72.19 GB, 77.95 GB, 83.39 GB, 88.58 GB, 93.66 GB, 98.36 GB, 102.67 GB, 106.88 GB, 111.03 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.09 GB, 137.97 GB, 140.72 GB, 143.58 GB, 146.33 GB, 148.98 GB, 151.68 GB, 154.34 GB, 157.02 GB, 159.71 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.39 GB (188320595207) - RowsProduced: 632.89M (632891204) - TotalNetworkReceiveTime: 3h25m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 966.892ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 58.991ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 4h49m - OpenTime: 966.894ms - ExecTreeOpenTime: 906.499ms - PrepareTime: 217.463ms - ExecTreePrepareTime: 148.155us KuduTableSink:(Total: 1h50m, non-child: 1h50m, % non-child: 100.00%) - KuduApplyTimer: 1h29m - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 95.67 K/sec - TotalNumRows: 632.89M (632891204) SORT_NODE (id=2):(Total: 4h49m, non-child: 1h23m, % non-child: 28.81%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 31m59s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 632.89M (632891204) - RowsReturnedRate: 36.44 K/sec - RunsCreated: 2 (2) - SortDataSize: 161.38 GB (173276815289) Buffer pool: - AllocTime: 34m19s - CumulativeAllocationBytes: 161.41 GB (173310738432) - CumulativeAllocations: 82.64K (82641) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 3h26m, non-child: 50s452ms, % non-child: 0.41%) - ConvertRowBatchTime: 20s832ms - PeakMemoryUsage: 19.53 MB (20481554) - RowsReturned: 632.97M (632972676) - RowsReturnedRate: 51.20 K/sec Buffer pool: - AllocTime: 2h50m - CumulativeAllocationBytes: 287.45 GB (308649533440) - CumulativeAllocations: 15.07M (15070778) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.04 GB, 40.75 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.27 GB, 89.51 GB, 94.65 GB, 99.40 GB, 103.76 GB, 108.01 GB, 112.21 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.05 GB, 130.32 GB, 133.47 GB, 136.53 GB, 139.44 GB, 142.22 GB, 145.11 GB, 147.89 GB, 150.57 GB, 153.29 GB, 155.99 GB, 158.69 GB, 161.42 GB, 163.12 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.16 GB, 163.17 GB, 163.17 GB, 163.17 GB, 163.17 GB, 163.17 GB, 163.17 GB, 163.17 GB, 163.17 GB - FirstBatchArrivalWaitTime: 906.420ms - TotalBytesReceived: 163.17 GB (175198042785) - TotalGetBatchTime: 3h25m - DataArrivalTimer: 3h25m SenderSide: - DeserializeRowBatchTime: 3h - NumBatchesArrived: 7.54M (7535389) - NumBatchesDeferred: 120.20K (120197) - NumBatchesEnqueued: 7.54M (7535389) - NumBatchesReceived: 7.54M (7535389) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 59.147ms, non-child: 59.147ms, % non-child: 100.00%) - CodegenTime: 1.257ms - CompileTime: 5.176ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 30.966ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.379ms Instance 844a0200d7876345:20bb38b9000000bb (host=vd1111.foo.com:22000):(Total: 1s184ms, non-child: 202.691ms, % non-child: 17.12%) Fragment Instance Lifecycle Event Timeline: 1s184ms - Prepare Finished: 201.875ms (201.875ms) - Open Finished: 1s184ms (982.535ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.19 GB, 52.70 GB, 59.57 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.54 GB, 85.16 GB, 89.43 GB, 93.35 GB, 97.18 GB, 100.95 GB, 104.50 GB, 107.97 GB, 111.18 GB, 114.29 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.43 GB, 127.92 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.72 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157577986486) - PeakReservation: 146.72 GB (157538058240) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.68 GB (172525062263) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h59m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 982.519ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 45.993ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 982.522ms - ExecTreeOpenTime: 935.672ms - PrepareTime: 201.622ms - ExecTreePrepareTime: 184.348us KuduTableSink:(Total: 12.532us, non-child: 12.532us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 935.854ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157538066432) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 25m24s - CumulativeAllocationBytes: 146.72 GB (157538058240) - CumulativeAllocations: 75.12K (75120) - PeakReservation: 146.72 GB (157538058240) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157538058240) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h59m, non-child: 45s219ms, % non-child: 0.21%) - ConvertRowBatchTime: 19s046ms - PeakMemoryUsage: 19.53 MB (20483529) - RowsReturned: 575.37M (575371356) - RowsReturnedRate: 26.65 K/sec Buffer pool: - AllocTime: 2h33m - CumulativeAllocationBytes: 261.29 GB (280562032640) - CumulativeAllocations: 13.70M (13699318) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.37 GB, 94.33 GB, 98.20 GB, 102.01 GB, 105.60 GB, 109.10 GB, 112.36 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.76 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.28 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 935.555ms - TotalBytesReceived: 148.32 GB (159253699885) - TotalGetBatchTime: 5h59m - DataArrivalTimer: 5h59m SenderSide: - DeserializeRowBatchTime: 2h41m - NumBatchesArrived: 6.85M (6849659) - NumBatchesDeferred: 32.58K (32582) - NumBatchesEnqueued: 6.85M (6849659) - NumBatchesReceived: 6.85M (6849659) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 45.648ms, non-child: 45.648ms, % non-child: 100.00%) - CodegenTime: 901.211us - CompileTime: 4.341ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 24.747ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 16.111ms Instance 844a0200d7876345:20bb38b9000000ea (host=ve1125.foo.com:22000):(Total: 1s183ms, non-child: 203.336ms, % non-child: 17.18%) Fragment Instance Lifecycle Event Timeline: 1s183ms - Prepare Finished: 202.643ms (202.643ms) - Open Finished: 1s183ms (981.215ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.43 GB, 137.87 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157577986486) - PeakReservation: 146.72 GB (157538058240) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.24 GB (170986142182) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h50m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 981.174ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 57.992ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 981.176ms - ExecTreeOpenTime: 921.572ms - PrepareTime: 202.222ms - ExecTreePrepareTime: 134.804us KuduTableSink:(Total: 11.560us, non-child: 11.560us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 921.704ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157538066432) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 32m36s - CumulativeAllocationBytes: 146.72 GB (157538058240) - CumulativeAllocations: 75.12K (75120) - PeakReservation: 146.72 GB (157538058240) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157538058240) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h50m, non-child: 48s232ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s010ms - PeakMemoryUsage: 19.53 MB (20483745) - RowsReturned: 575.37M (575366820) - RowsReturnedRate: 27.35 K/sec Buffer pool: - AllocTime: 2h38m - CumulativeAllocationBytes: 261.29 GB (280559820800) - CumulativeAllocations: 13.70M (13699210) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.18 GB, 102.00 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 921.512ms - TotalBytesReceived: 148.32 GB (159253074702) - TotalGetBatchTime: 5h50m - DataArrivalTimer: 5h49m SenderSide: - DeserializeRowBatchTime: 2h51m - NumBatchesArrived: 6.85M (6849605) - NumBatchesDeferred: 80.69K (80695) - NumBatchesEnqueued: 6.85M (6849605) - NumBatchesReceived: 6.85M (6849605) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 58.432ms, non-child: 58.432ms, % non-child: 100.00%) - CodegenTime: 1.027ms - CompileTime: 5.007ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.440ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 20.311ms Instance 844a0200d7876345:20bb38b9000000c6 (host=vd1142.foo.com:22000):(Total: 1s183ms, non-child: 205.864ms, % non-child: 17.40%) Fragment Instance Lifecycle Event Timeline: 1s183ms - Prepare Finished: 204.862ms (204.862ms) - Open Finished: 1s183ms (978.686ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.43 GB, 137.87 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157577986486) - PeakReservation: 146.72 GB (157538058240) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.43 GB (171182603493) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h52m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 978.661ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 52.992ms - TotalThreadsVoluntaryContextSwitches: 18 (18) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 978.664ms - ExecTreeOpenTime: 925.100ms - PrepareTime: 204.464ms - ExecTreePrepareTime: 158.070us KuduTableSink:(Total: 16.879us, non-child: 16.879us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 925.255ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157538066432) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 32m15s - CumulativeAllocationBytes: 146.72 GB (157538058240) - CumulativeAllocations: 75.12K (75120) - PeakReservation: 146.72 GB (157538058240) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157538058240) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 48s730ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s947ms - PeakMemoryUsage: 19.53 MB (20483337) - RowsReturned: 575.37M (575366988) - RowsReturnedRate: 27.16 K/sec Buffer pool: - AllocTime: 3h34m - CumulativeAllocationBytes: 261.29 GB (280559902720) - CumulativeAllocations: 13.70M (13699214) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.19 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 925.011ms - TotalBytesReceived: 148.32 GB (159252926637) - TotalGetBatchTime: 5h52m - DataArrivalTimer: 5h52m SenderSide: - DeserializeRowBatchTime: 3h43m - NumBatchesArrived: 6.85M (6849607) - NumBatchesDeferred: 84.00K (84004) - NumBatchesEnqueued: 6.85M (6849607) - NumBatchesReceived: 6.85M (6849607) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 52.053ms, non-child: 52.053ms, % non-child: 100.00%) - CodegenTime: 1.028ms - CompileTime: 4.085ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 25.806ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 21.576ms Instance 844a0200d7876345:20bb38b9000000c3 (host=ve1128.foo.com:22000):(Total: 1s183ms, non-child: 202.081ms, % non-child: 17.08%) Fragment Instance Lifecycle Event Timeline: 1s183ms - Prepare Finished: 201.188ms (201.188ms) - Open Finished: 1s183ms (982.219ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.17 GB, 52.68 GB, 59.56 GB, 65.62 GB, 70.86 GB, 75.80 GB, 80.52 GB, 85.14 GB, 89.42 GB, 93.33 GB, 97.16 GB, 100.93 GB, 104.48 GB, 107.95 GB, 111.16 GB, 114.27 GB, 117.21 GB, 120.04 GB, 122.80 GB, 125.41 GB, 127.91 GB, 130.50 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.72 GB, 145.17 GB, 146.70 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157565403574) - PeakReservation: 146.71 GB (157525475328) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.42 GB (171177553509) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h53m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 982.203ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 57.991ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 982.206ms - ExecTreeOpenTime: 922.007ms - PrepareTime: 200.904ms - ExecTreePrepareTime: 234.494us KuduTableSink:(Total: 19.899us, non-child: 19.899us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 922.238ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157525483520) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 31m22s - CumulativeAllocationBytes: 146.71 GB (157525475328) - CumulativeAllocations: 75.11K (75114) - PeakReservation: 146.71 GB (157525475328) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157525475328) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 49s549ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s550ms - PeakMemoryUsage: 19.53 MB (20483512) - RowsReturned: 575.33M (575328516) - RowsReturnedRate: 27.13 K/sec Buffer pool: - AllocTime: 2h34m - CumulativeAllocationBytes: 261.27 GB (280541143040) - CumulativeAllocations: 13.70M (13698298) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.03 GB, 45.62 GB, 53.21 GB, 60.17 GB, 66.30 GB, 71.59 GB, 76.59 GB, 81.36 GB, 86.03 GB, 90.35 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.58 GB, 109.08 GB, 112.34 GB, 115.48 GB, 118.45 GB, 121.31 GB, 124.10 GB, 126.74 GB, 129.26 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.26 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 921.937ms - TotalBytesReceived: 148.31 GB (159242096685) - TotalGetBatchTime: 5h52m - DataArrivalTimer: 5h52m SenderSide: - DeserializeRowBatchTime: 2h45m - NumBatchesArrived: 6.85M (6849149) - NumBatchesDeferred: 48.12K (48117) - NumBatchesEnqueued: 6.85M (6849149) - NumBatchesReceived: 6.85M (6849149) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 58.830ms, non-child: 58.830ms, % non-child: 100.00%) - CodegenTime: 1.129ms - CompileTime: 4.824ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.609ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.819ms Instance 844a0200d7876345:20bb38b90000008c (host=vc1511.foo.com:22000):(Total: 1s182ms, non-child: 203.082ms, % non-child: 17.17%) Fragment Instance Lifecycle Event Timeline: 1s183ms - Prepare Finished: 202.227ms (202.227ms) - Open Finished: 1s183ms (981.068ms) MemoryUsage(8m32s): 22.79 GB, 40.34 GB, 49.69 GB, 57.95 GB, 65.51 GB, 72.19 GB, 77.95 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.36 GB, 102.66 GB, 106.87 GB, 111.02 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.70 GB, 128.94 GB, 132.05 GB, 135.08 GB, 137.96 GB, 140.71 GB, 143.56 GB, 146.31 GB, 148.96 GB, 151.66 GB, 154.32 GB, 157.00 GB, 159.69 GB, 161.37 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173321306550) - PeakReservation: 161.38 GB (173281378304) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.34 GB (188267575484) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h49m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 981.052ms - TotalThreadsSysTime: 999.000us - TotalThreadsUserTime: 52.992ms - TotalThreadsVoluntaryContextSwitches: 22 (22) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 981.055ms - ExecTreeOpenTime: 926.490ms - PrepareTime: 201.705ms - ExecTreePrepareTime: 152.091us KuduTableSink:(Total: 14.967us, non-child: 14.967us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 926.624ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173281386496) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 32m58s - CumulativeAllocationBytes: 161.38 GB (173281378304) - CumulativeAllocations: 82.63K (82627) - PeakReservation: 161.38 GB (173281378304) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173281378304) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h50m, non-child: 51s055ms, % non-child: 0.24%) - ConvertRowBatchTime: 21s373ms - PeakMemoryUsage: 19.53 MB (20482424) - RowsReturned: 632.87M (632873388) - RowsReturnedRate: 30.11 K/sec Buffer pool: - AllocTime: 3h18m - CumulativeAllocationBytes: 287.41 GB (308601118720) - CumulativeAllocations: 15.07M (15068414) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.04 GB, 40.75 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.65 GB, 99.39 GB, 103.75 GB, 108.00 GB, 112.20 GB, 116.15 GB, 120.01 GB, 123.58 GB, 127.04 GB, 130.31 GB, 133.46 GB, 136.52 GB, 139.43 GB, 142.20 GB, 145.09 GB, 147.87 GB, 150.55 GB, 153.27 GB, 155.97 GB, 158.67 GB, 161.39 GB, 163.09 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 926.380ms - TotalBytesReceived: 163.14 GB (175170047368) - TotalGetBatchTime: 5h49m - DataArrivalTimer: 5h49m SenderSide: - DeserializeRowBatchTime: 3h28m - NumBatchesArrived: 7.53M (7534207) - NumBatchesDeferred: 94.42K (94419) - NumBatchesEnqueued: 7.53M (7534207) - NumBatchesReceived: 7.53M (7534207) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 53.094ms, non-child: 53.094ms, % non-child: 100.00%) - CodegenTime: 1.191ms - CompileTime: 4.682ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.131ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.687ms Instance 844a0200d7876345:20bb38b9000000ce (host=ve1140.foo.com:22000):(Total: 1s181ms, non-child: 213.301ms, % non-child: 18.06%) Fragment Instance Lifecycle Event Timeline: 1s181ms - Prepare Finished: 212.605ms (212.605ms) - Open Finished: 1s181ms (968.971ms) MemoryUsage(8m32s): 20.75 GB, 36.69 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.41 GB, 93.33 GB, 97.15 GB, 100.93 GB, 104.48 GB, 107.95 GB, 111.16 GB, 114.27 GB, 117.21 GB, 120.04 GB, 122.80 GB, 125.42 GB, 127.91 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.73 GB, 145.17 GB, 146.71 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157567500726) - PeakReservation: 146.71 GB (157527572480) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.44 GB (171202174685) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h58m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 968.942ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 55.991ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 968.945ms - ExecTreeOpenTime: 910.409ms - PrepareTime: 212.069ms - ExecTreePrepareTime: 162.404us KuduTableSink:(Total: 21.933us, non-child: 21.933us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 910.568ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157527580672) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 25m53s - CumulativeAllocationBytes: 146.71 GB (157527572480) - CumulativeAllocations: 75.11K (75115) - PeakReservation: 146.71 GB (157527572480) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157527572480) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h59m, non-child: 49s636ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s523ms - PeakMemoryUsage: 19.53 MB (20482914) - RowsReturned: 575.33M (575332464) - RowsReturnedRate: 26.68 K/sec Buffer pool: - AllocTime: 1h45m - CumulativeAllocationBytes: 261.28 GB (280543068160) - CumulativeAllocations: 13.70M (13698392) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.03 GB, 90.35 GB, 94.31 GB, 98.17 GB, 101.99 GB, 105.58 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.45 GB, 121.31 GB, 124.10 GB, 126.74 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 910.324ms - TotalBytesReceived: 148.31 GB (159243951255) - TotalGetBatchTime: 5h58m - DataArrivalTimer: 5h58m SenderSide: - DeserializeRowBatchTime: 1h53m - NumBatchesArrived: 6.85M (6849196) - NumBatchesDeferred: 11.60K (11597) - NumBatchesEnqueued: 6.85M (6849196) - NumBatchesReceived: 6.85M (6849196) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 57.202ms, non-child: 57.202ms, % non-child: 100.00%) - CodegenTime: 1.029ms - CompileTime: 4.536ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 27.448ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.664ms Instance 844a0200d7876345:20bb38b9000000ac (host=va1029.foo.com:22000):(Total: 1s179ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h21m - Prepare Finished: 208.345ms (208.345ms) - Open Finished: 1s180ms (971.997ms) - First Batch Produced: 7h21m (7h21m) - First Batch Sent: 7h21m (8.333ms) MemoryUsage(8m32s): 22.81 GB, 40.35 GB, 49.69 GB, 57.96 GB, 65.53 GB, 72.20 GB, 77.97 GB, 83.40 GB, 88.59 GB, 93.67 GB, 98.37 GB, 102.68 GB, 106.89 GB, 111.03 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.09 GB, 137.97 GB, 140.71 GB, 143.57 GB, 146.32 GB, 148.97 GB, 151.67 GB, 154.33 GB, 157.01 GB, 159.71 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173326988912) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.11 GB (188019547778) - RowsProduced: 235.96M (235956060) - TotalNetworkReceiveTime: 5h50m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 971.977ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 53.991ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h21m - OpenTime: 971.983ms - ExecTreeOpenTime: 914.259ms - PrepareTime: 207.859ms - ExecTreePrepareTime: 189.446us KuduTableSink:(Total: 40m8s, non-child: 40m8s, % non-child: 100.00%) - KuduApplyTimer: 33m14s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 97.97 K/sec - TotalNumRows: 235.96M (235955036) SORT_NODE (id=2):(Total: 7h21m, non-child: 1h30m, % non-child: 20.48%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 37m46s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 235.96M (235956060) - RowsReturnedRate: 8.91 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276485830) Buffer pool: - AllocTime: 31m11s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h50m, non-child: 51s215ms, % non-child: 0.24%) - ConvertRowBatchTime: 21s146ms - PeakMemoryUsage: 19.53 MB (20481595) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.06 K/sec Buffer pool: - AllocTime: 2h56m - CumulativeAllocationBytes: 287.43 GB (308628398080) - CumulativeAllocations: 15.07M (15069746) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.02 GB, 40.75 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.95 GB, 78.78 GB, 84.27 GB, 89.51 GB, 94.66 GB, 99.40 GB, 103.76 GB, 108.01 GB, 112.20 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.32 GB, 133.46 GB, 136.52 GB, 139.43 GB, 142.21 GB, 145.09 GB, 147.88 GB, 150.56 GB, 153.28 GB, 155.98 GB, 158.68 GB, 161.41 GB, 163.11 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 914.163ms - TotalBytesReceived: 163.15 GB (175185711366) - TotalGetBatchTime: 5h50m - DataArrivalTimer: 5h50m SenderSide: - DeserializeRowBatchTime: 3h14m - NumBatchesArrived: 7.53M (7534997) - NumBatchesDeferred: 95.44K (95438) - NumBatchesEnqueued: 7.53M (7534873) - NumBatchesReceived: 7.53M (7534873) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.258ms, non-child: 56.258ms, % non-child: 100.00%) - CodegenTime: 1.161ms - CompileTime: 4.432ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.814ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.462ms Instance 844a0200d7876345:20bb38b9000000cd (host=vc1525.foo.com:22000):(Total: 1s179ms, non-child: 209.925ms, % non-child: 17.80%) Fragment Instance Lifecycle Event Timeline: 1s179ms - Prepare Finished: 209.263ms (209.263ms) - Open Finished: 1s179ms (970.278ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.49 GB, 107.96 GB, 111.18 GB, 114.28 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.43 GB, 127.92 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157577986486) - PeakReservation: 146.72 GB (157538058240) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.62 GB (172469602738) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h53m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 970.256ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 60.991ms - TotalThreadsVoluntaryContextSwitches: 6 (6) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 970.259ms - ExecTreeOpenTime: 908.566ms - PrepareTime: 208.865ms - ExecTreePrepareTime: 163.259us KuduTableSink:(Total: 13.852us, non-child: 13.852us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 908.727ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157538066432) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 29m53s - CumulativeAllocationBytes: 146.72 GB (157538058240) - CumulativeAllocations: 75.12K (75120) - PeakReservation: 146.72 GB (157538058240) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157538058240) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 48s564ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s358ms - PeakMemoryUsage: 19.53 MB (20483272) - RowsReturned: 575.37M (575368332) - RowsReturnedRate: 27.13 K/sec Buffer pool: - AllocTime: 3h16m - CumulativeAllocationBytes: 261.29 GB (280560558080) - CumulativeAllocations: 13.70M (13699246) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.63 GB, 53.22 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.37 GB, 86.05 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.59 GB, 109.09 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.76 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 908.511ms - TotalBytesReceived: 148.32 GB (159253092747) - TotalGetBatchTime: 5h53m - DataArrivalTimer: 5h52m SenderSide: - DeserializeRowBatchTime: 3h26m - NumBatchesArrived: 6.85M (6849623) - NumBatchesDeferred: 68.49K (68488) - NumBatchesEnqueued: 6.85M (6849623) - NumBatchesReceived: 6.85M (6849623) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 60.519ms, non-child: 60.519ms, % non-child: 100.00%) - CodegenTime: 1.182ms - CompileTime: 5.171ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 30.749ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.069ms Instance 844a0200d7876345:20bb38b900000097 (host=vd1103.foo.com:22000):(Total: 1s178ms, non-child: 206.346ms, % non-child: 17.51%) Fragment Instance Lifecycle Event Timeline: 1s179ms - Prepare Finished: 205.630ms (205.630ms) - Open Finished: 1s179ms (973.386ms) MemoryUsage(8m32s): 22.79 GB, 40.34 GB, 49.69 GB, 57.96 GB, 65.52 GB, 72.19 GB, 77.96 GB, 83.39 GB, 88.58 GB, 93.66 GB, 98.36 GB, 102.67 GB, 106.87 GB, 111.02 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.70 GB, 128.94 GB, 132.05 GB, 135.08 GB, 137.95 GB, 140.70 GB, 143.55 GB, 146.30 GB, 148.96 GB, 151.65 GB, 154.31 GB, 156.99 GB, 159.68 GB, 161.37 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.41 GB (173317112246) - PeakReservation: 161.38 GB (173277184000) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.37 GB (188302406418) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h48m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 973.361ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 62.990ms - TotalThreadsVoluntaryContextSwitches: 6 (6) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 973.364ms - ExecTreeOpenTime: 910.365ms - PrepareTime: 205.209ms - ExecTreePrepareTime: 163.162us KuduTableSink:(Total: 17.430us, non-child: 17.430us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 910.526ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173277192192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 33m1s - CumulativeAllocationBytes: 161.38 GB (173277184000) - CumulativeAllocations: 82.62K (82625) - PeakReservation: 161.38 GB (173277184000) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173277184000) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h49m, non-child: 49s906ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s675ms - PeakMemoryUsage: 19.53 MB (20481666) - RowsReturned: 632.85M (632853648) - RowsReturnedRate: 30.19 K/sec Buffer pool: - AllocTime: 2h46m - CumulativeAllocationBytes: 287.40 GB (308591493120) - CumulativeAllocations: 15.07M (15067944) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.04 GB, 40.75 GB, 50.20 GB, 58.56 GB, 66.21 GB, 72.95 GB, 78.77 GB, 84.27 GB, 89.51 GB, 94.65 GB, 99.40 GB, 103.75 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.03 GB, 130.31 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.08 GB, 147.86 GB, 150.55 GB, 153.27 GB, 155.96 GB, 158.67 GB, 161.39 GB, 163.09 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB - FirstBatchArrivalWaitTime: 910.276ms - TotalBytesReceived: 163.13 GB (175164620584) - TotalGetBatchTime: 5h48m - DataArrivalTimer: 5h48m SenderSide: - DeserializeRowBatchTime: 3h1m - NumBatchesArrived: 7.53M (7533972) - NumBatchesDeferred: 113.15K (113149) - NumBatchesEnqueued: 7.53M (7533972) - NumBatchesReceived: 7.53M (7533972) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 61.744ms, non-child: 61.744ms, % non-child: 100.00%) - CodegenTime: 1.345ms - CompileTime: 5.126ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.960ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.966ms Instance 844a0200d7876345:20bb38b900000091 (host=vc1519.foo.com:22000):(Total: 1s177ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h16m - Prepare Finished: 212.498ms (212.498ms) - Open Finished: 1s178ms (965.523ms) - First Batch Produced: 7h16m (7h16m) - First Batch Sent: 7h16m (11.276ms) MemoryUsage(8m32s): 22.82 GB, 40.36 GB, 49.70 GB, 57.96 GB, 65.52 GB, 72.19 GB, 77.96 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.36 GB, 102.67 GB, 106.88 GB, 111.02 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.70 GB, 128.94 GB, 132.05 GB, 135.08 GB, 137.96 GB, 140.71 GB, 143.56 GB, 146.32 GB, 148.97 GB, 151.66 GB, 154.33 GB, 157.00 GB, 159.70 GB, 161.38 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.35 GB (188281490303) - RowsProduced: 230.33M (230333162) - TotalNetworkReceiveTime: 5h48m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 965.504ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 54.991ms - TotalThreadsVoluntaryContextSwitches: 18 (18) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h16m - OpenTime: 965.508ms - ExecTreeOpenTime: 908.174ms - PrepareTime: 212.107ms - ExecTreePrepareTime: 191.317us KuduTableSink:(Total: 44m18s, non-child: 44m18s, % non-child: 100.00%) - KuduApplyTimer: 36m22s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 86.64 K/sec - TotalNumRows: 230.33M (230332138) SORT_NODE (id=2):(Total: 7h16m, non-child: 1h27m, % non-child: 20.06%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 33m16s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 230.33M (230333162) - RowsReturnedRate: 8.79 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276876815) Buffer pool: - AllocTime: 33m57s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h49m, non-child: 51s776ms, % non-child: 0.25%) - ConvertRowBatchTime: 21s206ms - PeakMemoryUsage: 19.53 MB (20481651) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.20 K/sec Buffer pool: - AllocTime: 3h42m - CumulativeAllocationBytes: 287.42 GB (308619632640) - CumulativeAllocations: 15.07M (15069318) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.02 GB, 40.76 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.64 GB, 99.39 GB, 103.75 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.03 GB, 130.31 GB, 133.46 GB, 136.52 GB, 139.43 GB, 142.21 GB, 145.09 GB, 147.87 GB, 150.56 GB, 153.28 GB, 155.97 GB, 158.68 GB, 161.40 GB, 163.10 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 908.093ms - TotalBytesReceived: 163.15 GB (175181129812) - TotalGetBatchTime: 5h48m - DataArrivalTimer: 5h48m SenderSide: - DeserializeRowBatchTime: 3h54m - NumBatchesArrived: 7.53M (7534659) - NumBatchesDeferred: 141.72K (141716) - NumBatchesEnqueued: 7.53M (7534659) - NumBatchesReceived: 7.53M (7534659) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 55.932ms, non-child: 55.932ms, % non-child: 100.00%) - CodegenTime: 1.083ms - CompileTime: 5.822ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.371ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 21.175ms Instance 844a0200d7876345:20bb38b9000000d7 (host=vd1340.foo.com:22000):(Total: 1s177ms, non-child: 194.236ms, % non-child: 16.50%) Fragment Instance Lifecycle Event Timeline: 1s177ms - Prepare Finished: 193.226ms (193.226ms) - Open Finished: 1s177ms (984.275ms) MemoryUsage(8m32s): 20.73 GB, 36.68 GB, 45.17 GB, 52.69 GB, 59.56 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.14 GB, 89.41 GB, 93.33 GB, 97.15 GB, 100.92 GB, 104.48 GB, 107.94 GB, 111.16 GB, 114.27 GB, 117.21 GB, 120.04 GB, 122.79 GB, 125.41 GB, 127.91 GB, 130.50 GB, 133.00 GB, 135.41 GB, 137.86 GB, 140.28 GB, 142.72 GB, 145.17 GB, 146.70 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157561209270) - PeakReservation: 146.70 GB (157521281024) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.41 GB (171162066918) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h48m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 984.252ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 62.991ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 984.255ms - ExecTreeOpenTime: 917.344ms - PrepareTime: 192.844ms - ExecTreePrepareTime: 150.599us KuduTableSink:(Total: 17.791us, non-child: 17.791us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 917.492ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.70 GB (157521289216) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 36m24s - CumulativeAllocationBytes: 146.70 GB (157521281024) - CumulativeAllocations: 75.11K (75112) - PeakReservation: 146.70 GB (157521281024) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.70 GB (157521281024) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h49m, non-child: 48s676ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s710ms - PeakMemoryUsage: 19.53 MB (20483748) - RowsReturned: 575.30M (575304240) - RowsReturnedRate: 27.45 K/sec Buffer pool: - AllocTime: 4h25m - CumulativeAllocationBytes: 261.26 GB (280529305600) - CumulativeAllocations: 13.70M (13697720) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.35 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.58 GB, 109.08 GB, 112.34 GB, 115.48 GB, 118.45 GB, 121.31 GB, 124.10 GB, 126.74 GB, 129.26 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.71 GB, 148.26 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB - FirstBatchArrivalWaitTime: 917.269ms - TotalBytesReceived: 148.30 GB (159236188050) - TotalGetBatchTime: 5h48m - DataArrivalTimer: 5h48m SenderSide: - DeserializeRowBatchTime: 4h33m - NumBatchesArrived: 6.85M (6848860) - NumBatchesDeferred: 132.24K (132240) - NumBatchesEnqueued: 6.85M (6848860) - NumBatchesReceived: 6.85M (6848860) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 65.414ms, non-child: 65.414ms, % non-child: 100.00%) - CodegenTime: 1.307ms - CompileTime: 5.760ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 34.592ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.476ms Instance 844a0200d7876345:20bb38b9000000ec (host=vc1510.foo.com:22000):(Total: 1s176ms, non-child: 209.287ms, % non-child: 17.79%) Fragment Instance Lifecycle Event Timeline: 1s176ms - Prepare Finished: 208.541ms (208.541ms) - Open Finished: 1s176ms (967.887ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.29 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.43 GB, 127.93 GB, 130.52 GB, 133.02 GB, 135.44 GB, 137.88 GB, 140.31 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157584277942) - PeakReservation: 146.72 GB (157544349696) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.57 GB (172408807361) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h55m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 967.867ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 56.991ms - TotalThreadsVoluntaryContextSwitches: 13 (13) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 967.870ms - ExecTreeOpenTime: 909.974ms - PrepareTime: 208.223ms - ExecTreePrepareTime: 246.469us KuduTableSink:(Total: 12.268us, non-child: 12.268us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 910.217ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157544357888) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 27m37s - CumulativeAllocationBytes: 146.72 GB (157544349696) - CumulativeAllocations: 75.12K (75123) - PeakReservation: 146.72 GB (157544349696) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157544349696) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h56m, non-child: 47s969ms, % non-child: 0.22%) - ConvertRowBatchTime: 20s039ms - PeakMemoryUsage: 19.53 MB (20482827) - RowsReturned: 575.39M (575387820) - RowsReturnedRate: 26.92 K/sec Buffer pool: - AllocTime: 2h29m - CumulativeAllocationBytes: 261.30 GB (280570060800) - CumulativeAllocations: 13.70M (13699710) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.95 GB, 37.05 GB, 45.64 GB, 53.24 GB, 60.19 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.37 GB, 94.33 GB, 98.19 GB, 102.01 GB, 105.60 GB, 109.11 GB, 112.36 GB, 115.50 GB, 118.47 GB, 121.33 GB, 124.12 GB, 126.76 GB, 129.29 GB, 131.91 GB, 134.44 GB, 136.88 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.74 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 909.903ms - TotalBytesReceived: 148.32 GB (159259295253) - TotalGetBatchTime: 5h55m - DataArrivalTimer: 5h55m SenderSide: - DeserializeRowBatchTime: 2h40m - NumBatchesArrived: 6.85M (6849855) - NumBatchesDeferred: 31.18K (31183) - NumBatchesEnqueued: 6.85M (6849855) - NumBatchesReceived: 6.85M (6849855) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.641ms, non-child: 56.641ms, % non-child: 100.00%) - CodegenTime: 1.076ms - CompileTime: 4.865ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 30.027ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 21.236ms Instance 844a0200d7876345:20bb38b9000000ed (host=ve1130.foo.com:22000):(Total: 1s176ms, non-child: 216.292ms, % non-child: 18.39%) Fragment Instance Lifecycle Event Timeline: 1s176ms - Prepare Finished: 215.500ms (215.500ms) - Open Finished: 1s176ms (961.015ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.35 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.29 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.43 GB, 127.93 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157580083638) - PeakReservation: 146.72 GB (157540155392) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.45 GB (171210071191) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h52m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 960.991ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 61.991ms - TotalThreadsVoluntaryContextSwitches: 9 (9) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 960.995ms - ExecTreeOpenTime: 898.118ms - PrepareTime: 214.966ms - ExecTreePrepareTime: 177.341us KuduTableSink:(Total: 12.522us, non-child: 12.522us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 898.293ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157540163584) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 31m17s - CumulativeAllocationBytes: 146.72 GB (157540155392) - CumulativeAllocations: 75.12K (75121) - PeakReservation: 146.72 GB (157540155392) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157540155392) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 49s056ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s468ms - PeakMemoryUsage: 19.54 MB (20484826) - RowsReturned: 575.38M (575384544) - RowsReturnedRate: 27.14 K/sec Buffer pool: - AllocTime: 3h42m - CumulativeAllocationBytes: 261.30 GB (280568463360) - CumulativeAllocations: 13.70M (13699632) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.18 GB, 66.32 GB, 71.62 GB, 76.61 GB, 81.38 GB, 86.05 GB, 90.37 GB, 94.33 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.36 GB, 115.50 GB, 118.47 GB, 121.33 GB, 124.12 GB, 126.76 GB, 129.29 GB, 131.91 GB, 134.44 GB, 136.88 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 898.042ms - TotalBytesReceived: 148.32 GB (159257724878) - TotalGetBatchTime: 5h52m - DataArrivalTimer: 5h52m SenderSide: - DeserializeRowBatchTime: 3h54m - NumBatchesArrived: 6.85M (6849816) - NumBatchesDeferred: 93.59K (93594) - NumBatchesEnqueued: 6.85M (6849816) - NumBatchesReceived: 6.85M (6849816) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 61.450ms, non-child: 61.450ms, % non-child: 100.00%) - CodegenTime: 1.219ms - CompileTime: 5.387ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 29.318ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 26.177ms Instance 844a0200d7876345:20bb38b9000000a6 (host=vd1141.foo.com:22000):(Total: 1s175ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 4h57m - Prepare Finished: 207.844ms (207.844ms) - Open Finished: 1s175ms (967.764ms) - First Batch Produced: 4h57m (4h57m) - First Batch Sent: 4h57m (9.335ms) MemoryUsage(8m32s): 22.81 GB, 40.35 GB, 49.70 GB, 57.97 GB, 65.54 GB, 72.21 GB, 77.97 GB, 83.41 GB, 88.59 GB, 93.68 GB, 98.38 GB, 102.68 GB, 106.89 GB, 111.04 GB, 114.95 GB, 118.77 GB, 122.31 GB, 125.72 GB, 128.97 GB, 132.08 GB, 135.11 GB, 137.99 GB, 140.73 GB, 143.59 GB, 146.34 GB, 148.99 GB, 151.69 GB, 154.35 GB, 157.03 GB, 159.72 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173327049346) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.13 GB (186968232352) - RowsProduced: 632.89M (632891204) - TotalNetworkReceiveTime: 3h29m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 967.747ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 43.993ms - TotalThreadsVoluntaryContextSwitches: 5 (5) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 4h57m - OpenTime: 967.750ms - ExecTreeOpenTime: 921.653ms - PrepareTime: 207.480ms - ExecTreePrepareTime: 141.566us KuduTableSink:(Total: 1h46m, non-child: 1h46m, % non-child: 100.00%) - KuduApplyTimer: 1h28m - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 99.29 K/sec - TotalNumRows: 632.89M (632891204) SORT_NODE (id=2):(Total: 4h57m, non-child: 1h27m, % non-child: 29.29%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 39m51s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 632.89M (632891204) - RowsReturnedRate: 35.48 K/sec - RunsCreated: 2 (2) - SortDataSize: 161.38 GB (173276069885) Buffer pool: - AllocTime: 29m11s - CumulativeAllocationBytes: 161.42 GB (173319127040) - CumulativeAllocations: 82.64K (82645) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 3h30m, non-child: 48s704ms, % non-child: 0.39%) - ConvertRowBatchTime: 20s369ms - PeakMemoryUsage: 19.53 MB (20481283) - RowsReturned: 633.00M (632998716) - RowsReturnedRate: 50.19 K/sec Buffer pool: - AllocTime: 3h2m - CumulativeAllocationBytes: 287.46 GB (308662231040) - CumulativeAllocations: 15.07M (15071398) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.02 GB, 40.75 GB, 50.20 GB, 58.56 GB, 66.21 GB, 72.96 GB, 78.78 GB, 84.28 GB, 89.52 GB, 94.66 GB, 99.41 GB, 103.76 GB, 108.02 GB, 112.21 GB, 116.17 GB, 120.02 GB, 123.60 GB, 127.06 GB, 130.33 GB, 133.48 GB, 136.54 GB, 139.45 GB, 142.23 GB, 145.11 GB, 147.90 GB, 150.58 GB, 153.30 GB, 156.00 GB, 158.70 GB, 161.42 GB, 163.12 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.17 GB, 163.17 GB, 163.17 GB, 163.17 GB, 163.17 GB, 163.17 GB, 163.17 GB, 163.17 GB, 163.17 GB - FirstBatchArrivalWaitTime: 921.579ms - TotalBytesReceived: 163.17 GB (175204499481) - TotalGetBatchTime: 3h29m - DataArrivalTimer: 3h29m SenderSide: - DeserializeRowBatchTime: 3h11m - NumBatchesArrived: 7.54M (7535699) - NumBatchesDeferred: 80.01K (80010) - NumBatchesEnqueued: 7.54M (7535699) - NumBatchesReceived: 7.54M (7535699) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 44.694ms, non-child: 44.694ms, % non-child: 100.00%) - CodegenTime: 871.184us - CompileTime: 3.546ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 22.233ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 18.428ms Instance 844a0200d7876345:20bb38b9000000b6 (host=vc1515.foo.com:22000):(Total: 1s175ms, non-child: 206.275ms, % non-child: 17.55%) Fragment Instance Lifecycle Event Timeline: 1s175ms - Prepare Finished: 205.529ms (205.529ms) - Open Finished: 1s175ms (970.122ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.28 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.43 GB, 127.93 GB, 130.52 GB, 133.02 GB, 135.44 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157580083638) - PeakReservation: 146.72 GB (157540155392) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.60 GB (172438098193) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h57m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 970.109ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 59.991ms - TotalThreadsVoluntaryContextSwitches: 13 (13) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 970.112ms - ExecTreeOpenTime: 908.526ms - PrepareTime: 205.051ms - ExecTreePrepareTime: 165.108us KuduTableSink:(Total: 13.572us, non-child: 13.572us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 908.688ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157540163584) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 27m18s - CumulativeAllocationBytes: 146.72 GB (157540155392) - CumulativeAllocations: 75.12K (75121) - PeakReservation: 146.72 GB (157540155392) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157540155392) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h57m, non-child: 48s164ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s997ms - PeakMemoryUsage: 19.53 MB (20482720) - RowsReturned: 575.38M (575382864) - RowsReturnedRate: 26.79 K/sec Buffer pool: - AllocTime: 2h32m - CumulativeAllocationBytes: 261.30 GB (280567644160) - CumulativeAllocations: 13.70M (13699592) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.64 GB, 53.24 GB, 60.19 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.37 GB, 94.33 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.36 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.76 GB, 129.28 GB, 131.91 GB, 134.44 GB, 136.88 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 908.442ms - TotalBytesReceived: 148.32 GB (159257398203) - TotalGetBatchTime: 5h57m - DataArrivalTimer: 5h57m SenderSide: - DeserializeRowBatchTime: 2h42m - NumBatchesArrived: 6.85M (6849796) - NumBatchesDeferred: 30.76K (30759) - NumBatchesEnqueued: 6.85M (6849796) - NumBatchesReceived: 6.85M (6849796) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 60.240ms, non-child: 60.240ms, % non-child: 100.00%) - CodegenTime: 1.274ms - CompileTime: 4.867ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.441ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.335ms Instance 844a0200d7876345:20bb38b9000000eb (host=vc1535.foo.com:22000):(Total: 1s174ms, non-child: 197.629ms, % non-child: 16.83%) Fragment Instance Lifecycle Event Timeline: 1s175ms - Prepare Finished: 196.854ms (196.854ms) - Open Finished: 1s175ms (978.213ms) MemoryUsage(8m32s): 20.73 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.28 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.02 GB, 135.43 GB, 137.87 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157573792182) - PeakReservation: 146.71 GB (157533863936) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.71 GB (172564369436) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h55m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 978.189ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 55.991ms - TotalThreadsVoluntaryContextSwitches: 20 (20) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 978.192ms - ExecTreeOpenTime: 921.093ms - PrepareTime: 196.322ms - ExecTreePrepareTime: 243.658us KuduTableSink:(Total: 13.541us, non-child: 13.541us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 921.334ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157533872128) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 27m45s - CumulativeAllocationBytes: 146.71 GB (157533863936) - CumulativeAllocations: 75.12K (75118) - PeakReservation: 146.71 GB (157533863936) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157533863936) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h55m, non-child: 50s095ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s387ms - PeakMemoryUsage: 19.53 MB (20483705) - RowsReturned: 575.36M (575360604) - RowsReturnedRate: 26.95 K/sec Buffer pool: - AllocTime: 2h40m - CumulativeAllocationBytes: 261.29 GB (280556789760) - CumulativeAllocations: 13.70M (13699062) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.95 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.18 GB, 66.32 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.36 GB, 94.32 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.46 GB, 121.32 GB, 124.11 GB, 126.75 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 921.044ms - TotalBytesReceived: 148.31 GB (159251345561) - TotalGetBatchTime: 5h55m - DataArrivalTimer: 5h55m SenderSide: - DeserializeRowBatchTime: 2h51m - NumBatchesArrived: 6.85M (6849531) - NumBatchesDeferred: 66.14K (66144) - NumBatchesEnqueued: 6.85M (6849531) - NumBatchesReceived: 6.85M (6849531) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 55.629ms, non-child: 55.629ms, % non-child: 100.00%) - CodegenTime: 1.090ms - CompileTime: 5.927ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.033ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 21.106ms Instance 844a0200d7876345:20bb38b900000092 (host=vc1532.foo.com:22000):(Total: 1s173ms, non-child: 204.662ms, % non-child: 17.44%) Fragment Instance Lifecycle Event Timeline: 1s174ms - Prepare Finished: 203.967ms (203.967ms) - Open Finished: 1s174ms (970.418ms) MemoryUsage(8m32s): 22.81 GB, 40.35 GB, 49.70 GB, 57.96 GB, 65.52 GB, 72.19 GB, 77.96 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.36 GB, 102.67 GB, 106.87 GB, 111.02 GB, 114.94 GB, 118.74 GB, 122.28 GB, 125.70 GB, 128.94 GB, 132.05 GB, 135.08 GB, 137.96 GB, 140.70 GB, 143.56 GB, 146.31 GB, 148.96 GB, 151.65 GB, 154.32 GB, 157.00 GB, 159.69 GB, 161.37 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173321306550) - PeakReservation: 161.38 GB (173281378304) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.34 GB (188264764604) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h47m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 970.400ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 64.989ms - TotalThreadsVoluntaryContextSwitches: 18 (18) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 970.403ms - ExecTreeOpenTime: 904.696ms - PrepareTime: 203.315ms - ExecTreePrepareTime: 156.167us KuduTableSink:(Total: 15.429us, non-child: 15.429us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 904.849ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.38 GB (173281386496) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 35m13s - CumulativeAllocationBytes: 161.38 GB (173281378304) - CumulativeAllocations: 82.63K (82627) - PeakReservation: 161.38 GB (173281378304) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173281378304) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h47m, non-child: 50s830ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s958ms - PeakMemoryUsage: 19.53 MB (20482320) - RowsReturned: 632.88M (632875572) - RowsReturnedRate: 30.33 K/sec Buffer pool: - AllocTime: 3h5m - CumulativeAllocationBytes: 287.41 GB (308602183680) - CumulativeAllocations: 15.07M (15068466) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.02 GB, 40.75 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.64 GB, 99.39 GB, 103.75 GB, 108.00 GB, 112.19 GB, 116.15 GB, 120.00 GB, 123.58 GB, 127.03 GB, 130.31 GB, 133.45 GB, 136.51 GB, 139.42 GB, 142.20 GB, 145.08 GB, 147.87 GB, 150.55 GB, 153.27 GB, 155.96 GB, 158.67 GB, 161.39 GB, 163.09 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB - FirstBatchArrivalWaitTime: 904.636ms - TotalBytesReceived: 163.14 GB (175170293259) - TotalGetBatchTime: 5h47m - DataArrivalTimer: 5h46m SenderSide: - DeserializeRowBatchTime: 3h16m - NumBatchesArrived: 7.53M (7534233) - NumBatchesDeferred: 141.31K (141310) - NumBatchesEnqueued: 7.53M (7534233) - NumBatchesReceived: 7.53M (7534233) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 64.251ms, non-child: 64.251ms, % non-child: 100.00%) - CodegenTime: 1.088ms - CompileTime: 5.578ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 33.265ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.850ms Instance 844a0200d7876345:20bb38b9000000a3 (host=va1030.foo.com:22000):(Total: 1s172ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h20m - Prepare Finished: 199.691ms (199.691ms) - Open Finished: 1s173ms (973.902ms) - First Batch Produced: 7h20m (7h20m) - First Batch Sent: 7h20m (10.311ms) MemoryUsage(8m32s): 22.79 GB, 40.34 GB, 49.69 GB, 57.96 GB, 65.52 GB, 72.19 GB, 77.95 GB, 83.39 GB, 88.57 GB, 93.66 GB, 98.36 GB, 102.67 GB, 106.88 GB, 111.03 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.09 GB, 137.96 GB, 140.71 GB, 143.57 GB, 146.32 GB, 148.97 GB, 151.66 GB, 154.33 GB, 157.00 GB, 159.70 GB, 161.38 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173325509046) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.33 GB (188264018705) - RowsProduced: 234.70M (234695894) - TotalNetworkReceiveTime: 5h50m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 973.873ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 55.991ms - TotalThreadsVoluntaryContextSwitches: 25 (25) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h20m - OpenTime: 973.876ms - ExecTreeOpenTime: 915.567ms - PrepareTime: 198.988ms - ExecTreePrepareTime: 155.134us KuduTableSink:(Total: 41m6s, non-child: 41m6s, % non-child: 100.00%) - KuduApplyTimer: 34m6s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 95.14 K/sec - TotalNumRows: 234.69M (234694870) SORT_NODE (id=2):(Total: 7h20m, non-child: 1h29m, % non-child: 20.39%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 36m49s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 234.70M (234695894) - RowsReturnedRate: 8.88 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173275699907) Buffer pool: - AllocTime: 32m25s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h50m, non-child: 51s164ms, % non-child: 0.24%) - ConvertRowBatchTime: 21s255ms - PeakMemoryUsage: 19.53 MB (20481400) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.08 K/sec Buffer pool: - AllocTime: 3h3m - CumulativeAllocationBytes: 287.42 GB (308619264000) - CumulativeAllocations: 15.07M (15069300) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.76 GB, 50.20 GB, 58.56 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.51 GB, 94.65 GB, 99.40 GB, 103.76 GB, 108.01 GB, 112.20 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.32 GB, 133.46 GB, 136.52 GB, 139.43 GB, 142.21 GB, 145.10 GB, 147.88 GB, 150.56 GB, 153.28 GB, 155.97 GB, 158.68 GB, 161.40 GB, 163.10 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 915.459ms - TotalBytesReceived: 163.15 GB (175179742457) - TotalGetBatchTime: 5h50m - DataArrivalTimer: 5h49m SenderSide: - DeserializeRowBatchTime: 3h17m - NumBatchesArrived: 7.53M (7534650) - NumBatchesDeferred: 77.42K (77419) - NumBatchesEnqueued: 7.53M (7534650) - NumBatchesReceived: 7.53M (7534650) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.790ms, non-child: 56.790ms, % non-child: 100.00%) - CodegenTime: 1.452ms - CompileTime: 6.635ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.261ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 21.072ms Instance 844a0200d7876345:20bb38b9000000c4 (host=ve1138.foo.com:22000):(Total: 1s172ms, non-child: 203.374ms, % non-child: 17.34%) Fragment Instance Lifecycle Event Timeline: 1s172ms - Prepare Finished: 202.479ms (202.479ms) - Open Finished: 1s172ms (970.423ms) MemoryUsage(8m32s): 20.74 GB, 36.69 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.33 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.91 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.73 GB, 145.17 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157569597878) - PeakReservation: 146.71 GB (157529669632) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.45 GB (171208570279) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h55m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 970.405ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 59.991ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 970.408ms - ExecTreeOpenTime: 908.341ms - PrepareTime: 202.113ms - ExecTreePrepareTime: 173.016us KuduTableSink:(Total: 12.179us, non-child: 12.179us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 908.511ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157529677824) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 27m21s - CumulativeAllocationBytes: 146.71 GB (157529669632) - CumulativeAllocations: 75.12K (75116) - PeakReservation: 146.71 GB (157529669632) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157529669632) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h56m, non-child: 47s565ms, % non-child: 0.22%) - ConvertRowBatchTime: 19s992ms - PeakMemoryUsage: 19.53 MB (20483121) - RowsReturned: 575.34M (575339688) - RowsReturnedRate: 26.90 K/sec Buffer pool: - AllocTime: 2h21m - CumulativeAllocationBytes: 261.28 GB (280546590720) - CumulativeAllocations: 13.70M (13698564) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.63 GB, 53.22 GB, 60.18 GB, 66.30 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.35 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.34 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 908.280ms - TotalBytesReceived: 148.31 GB (159245955610) - TotalGetBatchTime: 5h55m - DataArrivalTimer: 5h55m SenderSide: - DeserializeRowBatchTime: 2h32m - NumBatchesArrived: 6.85M (6849282) - NumBatchesDeferred: 44.72K (44720) - NumBatchesEnqueued: 6.85M (6849282) - NumBatchesReceived: 6.85M (6849282) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 60.681ms, non-child: 60.681ms, % non-child: 100.00%) - CodegenTime: 1.107ms - CompileTime: 5.794ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 34.487ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.789ms Instance 844a0200d7876345:20bb38b9000000de (host=vd1336.foo.com:22000):(Total: 1s171ms, non-child: 207.545ms, % non-child: 17.72%) Fragment Instance Lifecycle Event Timeline: 1s171ms - Prepare Finished: 206.458ms (206.458ms) - Open Finished: 1s171ms (965.129ms) MemoryUsage(8m32s): 20.72 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.28 GB, 117.23 GB, 120.05 GB, 122.81 GB, 125.43 GB, 127.92 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157582180790) - PeakReservation: 146.72 GB (157542252544) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.42 GB (171171240595) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h54m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 965.110ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 54.992ms - TotalThreadsVoluntaryContextSwitches: 8 (8) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 965.113ms - ExecTreeOpenTime: 908.727ms - PrepareTime: 206.180ms - ExecTreePrepareTime: 232.193us KuduTableSink:(Total: 13.949us, non-child: 13.949us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 908.956ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157542260736) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 29m9s - CumulativeAllocationBytes: 146.72 GB (157542252544) - CumulativeAllocations: 75.12K (75122) - PeakReservation: 146.72 GB (157542252544) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157542252544) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h55m, non-child: 48s401ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s910ms - PeakMemoryUsage: 19.53 MB (20483823) - RowsReturned: 575.39M (575389500) - RowsReturnedRate: 26.98 K/sec Buffer pool: - AllocTime: 3h7m - CumulativeAllocationBytes: 261.30 GB (280570880000) - CumulativeAllocations: 13.70M (13699750) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.19 GB, 66.31 GB, 71.61 GB, 76.61 GB, 81.37 GB, 86.05 GB, 90.37 GB, 94.33 GB, 98.19 GB, 102.01 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.76 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 908.611ms - TotalBytesReceived: 148.32 GB (159259122452) - TotalGetBatchTime: 5h54m - DataArrivalTimer: 5h54m SenderSide: - DeserializeRowBatchTime: 3h16m - NumBatchesArrived: 6.85M (6849875) - NumBatchesDeferred: 61.76K (61764) - NumBatchesEnqueued: 6.85M (6849875) - NumBatchesReceived: 6.85M (6849875) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 54.839ms, non-child: 54.839ms, % non-child: 100.00%) - CodegenTime: 1.295ms - CompileTime: 5.681ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 29.203ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 19.478ms Instance 844a0200d7876345:20bb38b9000000dc (host=vc1522.foo.com:22000):(Total: 1s171ms, non-child: 197.504ms, % non-child: 16.86%) Fragment Instance Lifecycle Event Timeline: 1s171ms - Prepare Finished: 196.474ms (196.474ms) - Open Finished: 1s171ms (974.996ms) MemoryUsage(8m32s): 20.73 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.14 GB, 89.42 GB, 93.33 GB, 97.16 GB, 100.93 GB, 104.48 GB, 107.94 GB, 111.16 GB, 114.27 GB, 117.21 GB, 120.04 GB, 122.79 GB, 125.41 GB, 127.91 GB, 130.50 GB, 133.00 GB, 135.42 GB, 137.86 GB, 140.29 GB, 142.72 GB, 145.17 GB, 146.70 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.73 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.74 GB (157561209270) - PeakReservation: 146.70 GB (157521281024) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.54 GB (172381036288) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h54m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 974.975ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 52.992ms - TotalThreadsVoluntaryContextSwitches: 10 (10) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 974.980ms - ExecTreeOpenTime: 919.993ms - PrepareTime: 196.206ms - ExecTreePrepareTime: 178.244us KuduTableSink:(Total: 12.957us, non-child: 12.957us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 920.168ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.70 GB (157521289216) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 28m54s - CumulativeAllocationBytes: 146.70 GB (157521281024) - CumulativeAllocations: 75.11K (75112) - PeakReservation: 146.70 GB (157521281024) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.70 GB (157521281024) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h54m, non-child: 48s318ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s119ms - PeakMemoryUsage: 19.53 MB (20482403) - RowsReturned: 575.31M (575307936) - RowsReturnedRate: 27.05 K/sec Buffer pool: - AllocTime: 3h2m - CumulativeAllocationBytes: 261.26 GB (280531107840) - CumulativeAllocations: 13.70M (13697808) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.95 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.19 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.35 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.08 GB, 112.34 GB, 115.48 GB, 118.45 GB, 121.31 GB, 124.09 GB, 126.74 GB, 129.27 GB, 131.89 GB, 134.42 GB, 136.86 GB, 139.33 GB, 141.78 GB, 144.24 GB, 146.72 GB, 148.26 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB - FirstBatchArrivalWaitTime: 919.888ms - TotalBytesReceived: 148.30 GB (159236707888) - TotalGetBatchTime: 5h54m - DataArrivalTimer: 5h53m SenderSide: - DeserializeRowBatchTime: 3h12m - NumBatchesArrived: 6.85M (6848904) - NumBatchesDeferred: 39.50K (39496) - NumBatchesEnqueued: 6.85M (6848904) - NumBatchesReceived: 6.85M (6848904) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 53.559ms, non-child: 53.559ms, % non-child: 100.00%) - CodegenTime: 1.106ms - CompileTime: 5.427ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 26.304ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 21.286ms Instance 844a0200d7876345:20bb38b9000000e7 (host=ve1127.foo.com:22000):(Total: 1s171ms, non-child: 200.242ms, % non-child: 17.10%) Fragment Instance Lifecycle Event Timeline: 1s171ms - Prepare Finished: 199.500ms (199.500ms) - Open Finished: 1s171ms (971.918ms) MemoryUsage(8m32s): 20.73 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.56 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.33 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.42 GB, 137.87 GB, 140.29 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157573792182) - PeakReservation: 146.71 GB (157533863936) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.42 GB (171172551918) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 6h - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 971.901ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 58.991ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 971.906ms - ExecTreeOpenTime: 910.815ms - PrepareTime: 199.051ms - ExecTreePrepareTime: 189.070us KuduTableSink:(Total: 15.703us, non-child: 15.703us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 911.000ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157533872128) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 23m47s - CumulativeAllocationBytes: 146.71 GB (157533863936) - CumulativeAllocations: 75.12K (75118) - PeakReservation: 146.71 GB (157533863936) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157533863936) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 6h, non-child: 48s903ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s275ms - PeakMemoryUsage: 19.53 MB (20482864) - RowsReturned: 575.35M (575353968) - RowsReturnedRate: 26.58 K/sec Buffer pool: - AllocTime: 1h31m - CumulativeAllocationBytes: 261.29 GB (280553553920) - CumulativeAllocations: 13.70M (13698904) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.94 GB, 37.05 GB, 45.64 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.36 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.90 GB, 134.42 GB, 136.86 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 910.735ms - TotalBytesReceived: 148.31 GB (159249862007) - TotalGetBatchTime: 6h - DataArrivalTimer: 6h SenderSide: - DeserializeRowBatchTime: 1h39m - NumBatchesArrived: 6.85M (6849452) - NumBatchesDeferred: 10.79K (10792) - NumBatchesEnqueued: 6.85M (6849452) - NumBatchesReceived: 6.85M (6849452) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 59.760ms, non-child: 59.760ms, % non-child: 100.00%) - CodegenTime: 1.084ms - CompileTime: 4.810ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 29.530ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.879ms Instance 844a0200d7876345:20bb38b9000000bf (host=vd1337.foo.com:22000):(Total: 1s170ms, non-child: 210.341ms, % non-child: 17.97%) Fragment Instance Lifecycle Event Timeline: 1s171ms - Prepare Finished: 209.605ms (209.605ms) - Open Finished: 1s171ms (961.600ms) MemoryUsage(8m32s): 20.74 GB, 36.68 GB, 45.17 GB, 52.69 GB, 59.56 GB, 65.63 GB, 70.87 GB, 75.81 GB, 80.52 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.43 GB, 137.87 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.75 GB (157573792182) - PeakReservation: 146.71 GB (157533863936) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.44 GB (171201769455) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h53m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 961.576ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 61.990ms - TotalThreadsVoluntaryContextSwitches: 5 (5) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 961.578ms - ExecTreeOpenTime: 897.306ms - PrepareTime: 209.183ms - ExecTreePrepareTime: 182.003us KuduTableSink:(Total: 11.730us, non-child: 11.730us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 897.485ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.71 GB (157533872128) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 30m57s - CumulativeAllocationBytes: 146.71 GB (157533863936) - CumulativeAllocations: 75.12K (75118) - PeakReservation: 146.71 GB (157533863936) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.71 GB (157533863936) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 48s559ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s167ms - PeakMemoryUsage: 19.53 MB (20483459) - RowsReturned: 575.36M (575357328) - RowsReturnedRate: 27.09 K/sec Buffer pool: - AllocTime: 3h35m - CumulativeAllocationBytes: 261.29 GB (280555192320) - CumulativeAllocations: 13.70M (13698984) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.62 GB, 53.22 GB, 60.17 GB, 66.31 GB, 71.60 GB, 76.60 GB, 81.36 GB, 86.04 GB, 90.36 GB, 94.31 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.72 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB - FirstBatchArrivalWaitTime: 897.252ms - TotalBytesReceived: 148.31 GB (159250087688) - TotalGetBatchTime: 5h53m - DataArrivalTimer: 5h53m SenderSide: - DeserializeRowBatchTime: 3h44m - NumBatchesArrived: 6.85M (6849492) - NumBatchesDeferred: 60.21K (60209) - NumBatchesEnqueued: 6.85M (6849492) - NumBatchesReceived: 6.85M (6849492) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.981ms, non-child: 62.981ms, % non-child: 100.00%) - CodegenTime: 1.234ms - CompileTime: 5.515ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.595ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.190ms Instance 844a0200d7876345:20bb38b900000082 (host=vd1113.foo.com:22000):(Total: 1s169ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h14m - Prepare Finished: 212.127ms (212.127ms) - Open Finished: 1s170ms (957.965ms) - First Batch Produced: 7h14m (7h14m) - First Batch Sent: 7h14m (9.540ms) MemoryUsage(8m32s): 22.82 GB, 40.36 GB, 49.70 GB, 57.96 GB, 65.53 GB, 72.20 GB, 77.96 GB, 83.40 GB, 88.58 GB, 93.67 GB, 98.37 GB, 102.68 GB, 106.88 GB, 111.03 GB, 114.95 GB, 118.75 GB, 122.29 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.09 GB, 137.97 GB, 140.72 GB, 143.57 GB, 146.32 GB, 148.98 GB, 151.67 GB, 154.33 GB, 157.01 GB, 159.70 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173326117356) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.24 GB (188161354886) - RowsProduced: 261.35M (261353962) - TotalNetworkReceiveTime: 5h51m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 957.934ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 61.991ms - TotalThreadsVoluntaryContextSwitches: 10 (10) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h14m - OpenTime: 957.937ms - ExecTreeOpenTime: 895.177ms - PrepareTime: 211.596ms - ExecTreePrepareTime: 159.109us KuduTableSink:(Total: 46m59s, non-child: 46m59s, % non-child: 100.00%) - KuduApplyTimer: 37m41s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 92.70 K/sec - TotalNumRows: 261.35M (261352938) SORT_NODE (id=2):(Total: 7h14m, non-child: 1h22m, % non-child: 19.04%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 30m43s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 261.35M (261353962) - RowsReturnedRate: 10.03 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276461478) Buffer pool: - AllocTime: 31m43s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h51m, non-child: 50s728ms, % non-child: 0.24%) - ConvertRowBatchTime: 21s113ms - PeakMemoryUsage: 19.53 MB (20481840) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.00 K/sec Buffer pool: - AllocTime: 3h1m - CumulativeAllocationBytes: 287.43 GB (308628357120) - CumulativeAllocations: 15.07M (15069744) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.75 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.95 GB, 78.77 GB, 84.27 GB, 89.51 GB, 94.65 GB, 99.40 GB, 103.76 GB, 108.01 GB, 112.20 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.31 GB, 133.46 GB, 136.52 GB, 139.43 GB, 142.21 GB, 145.10 GB, 147.88 GB, 150.56 GB, 153.28 GB, 155.98 GB, 158.69 GB, 161.41 GB, 163.11 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 895.118ms - TotalBytesReceived: 163.15 GB (175185665660) - TotalGetBatchTime: 5h51m - DataArrivalTimer: 5h50m SenderSide: - DeserializeRowBatchTime: 3h12m - NumBatchesArrived: 7.53M (7534923) - NumBatchesDeferred: 72.81K (72812) - NumBatchesEnqueued: 7.53M (7534872) - NumBatchesReceived: 7.53M (7534872) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 61.445ms, non-child: 61.445ms, % non-child: 100.00%) - CodegenTime: 988.249us - CompileTime: 5.186ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.699ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.005ms Instance 844a0200d7876345:20bb38b9000000d1 (host=va1032.foo.com:22000):(Total: 1s168ms, non-child: 206.872ms, % non-child: 17.71%) Fragment Instance Lifecycle Event Timeline: 1s168ms - Prepare Finished: 205.898ms (205.898ms) - Open Finished: 1s168ms (962.560ms) MemoryUsage(8m32s): 20.75 GB, 36.68 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.15 GB, 89.42 GB, 93.34 GB, 97.16 GB, 100.93 GB, 104.49 GB, 107.95 GB, 111.17 GB, 114.27 GB, 117.22 GB, 120.05 GB, 122.80 GB, 125.42 GB, 127.92 GB, 130.51 GB, 133.01 GB, 135.43 GB, 137.87 GB, 140.30 GB, 142.73 GB, 145.18 GB, 146.71 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.74 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157577986486) - PeakReservation: 146.72 GB (157538058240) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.65 GB (172496300525) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h51m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 962.545ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 54.992ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 962.548ms - ExecTreeOpenTime: 904.149ms - PrepareTime: 205.582ms - ExecTreePrepareTime: 182.026us KuduTableSink:(Total: 13.398us, non-child: 13.398us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 904.328ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157538066432) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 32m11s - CumulativeAllocationBytes: 146.72 GB (157538058240) - CumulativeAllocations: 75.12K (75120) - PeakReservation: 146.72 GB (157538058240) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157538058240) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h51m, non-child: 48s253ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s045ms - PeakMemoryUsage: 19.53 MB (20483625) - RowsReturned: 575.37M (575365140) - RowsReturnedRate: 27.27 K/sec Buffer pool: - AllocTime: 2h58m - CumulativeAllocationBytes: 261.29 GB (280559001600) - CumulativeAllocations: 13.70M (13699170) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.93 GB, 37.04 GB, 45.63 GB, 53.23 GB, 60.18 GB, 66.31 GB, 71.61 GB, 76.60 GB, 81.37 GB, 86.04 GB, 90.36 GB, 94.32 GB, 98.18 GB, 101.99 GB, 105.59 GB, 109.09 GB, 112.34 GB, 115.48 GB, 118.46 GB, 121.32 GB, 124.10 GB, 126.75 GB, 129.27 GB, 131.90 GB, 134.42 GB, 136.86 GB, 139.34 GB, 141.79 GB, 144.25 GB, 146.73 GB, 148.27 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 904.040ms - TotalBytesReceived: 148.32 GB (159253019176) - TotalGetBatchTime: 5h51m - DataArrivalTimer: 5h50m SenderSide: - DeserializeRowBatchTime: 3h10m - NumBatchesArrived: 6.85M (6849585) - NumBatchesDeferred: 65.67K (65671) - NumBatchesEnqueued: 6.85M (6849585) - NumBatchesReceived: 6.85M (6849585) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 56.986ms, non-child: 56.986ms, % non-child: 100.00%) - CodegenTime: 1.121ms - CompileTime: 5.265ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.916ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 22.252ms Instance 844a0200d7876345:20bb38b90000009f (host=vd1331.foo.com:22000):(Total: 1s166ms, non-child: 199.497ms, % non-child: 17.10%) Fragment Instance Lifecycle Event Timeline: 1s167ms - Prepare Finished: 198.685ms (198.685ms) - Open Finished: 1s167ms (968.718ms) MemoryUsage(8m32s): 22.80 GB, 40.35 GB, 49.69 GB, 57.95 GB, 65.52 GB, 72.19 GB, 77.95 GB, 83.39 GB, 88.57 GB, 93.65 GB, 98.35 GB, 102.66 GB, 106.86 GB, 111.01 GB, 114.92 GB, 118.73 GB, 122.27 GB, 125.69 GB, 128.93 GB, 132.04 GB, 135.07 GB, 137.95 GB, 140.69 GB, 143.55 GB, 146.30 GB, 148.95 GB, 151.64 GB, 154.31 GB, 156.99 GB, 159.68 GB, 161.36 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB, 161.41 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.41 GB (173312917942) - PeakReservation: 161.37 GB (173272989696) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.11 GB (186944756357) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h53m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 7 (7) - TotalThreadsTotalWallClockTime: 968.690ms - TotalThreadsSysTime: 2.999ms - TotalThreadsUserTime: 54.991ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 968.694ms - ExecTreeOpenTime: 909.815ms - PrepareTime: 198.188ms - ExecTreePrepareTime: 193.463us KuduTableSink:(Total: 34.737us, non-child: 34.737us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 910.006ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.37 GB (173272997888) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 30m51s - CumulativeAllocationBytes: 161.37 GB (173272989696) - CumulativeAllocations: 82.62K (82623) - PeakReservation: 161.37 GB (173272989696) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.37 GB (173272989696) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 48s795ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s480ms - PeakMemoryUsage: 19.53 MB (20482029) - RowsReturned: 632.84M (632841384) - RowsReturnedRate: 29.83 K/sec Buffer pool: - AllocTime: 3h17m - CumulativeAllocationBytes: 287.39 GB (308585512960) - CumulativeAllocations: 15.07M (15067652) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.05 GB, 40.76 GB, 50.20 GB, 58.55 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.26 GB, 89.50 GB, 94.64 GB, 99.39 GB, 103.74 GB, 108.00 GB, 112.19 GB, 116.14 GB, 119.99 GB, 123.57 GB, 127.02 GB, 130.30 GB, 133.44 GB, 136.51 GB, 139.41 GB, 142.19 GB, 145.08 GB, 147.86 GB, 150.54 GB, 153.26 GB, 155.96 GB, 158.66 GB, 161.39 GB, 163.09 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.12 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB - FirstBatchArrivalWaitTime: 909.737ms - TotalBytesReceived: 163.13 GB (175161058390) - TotalGetBatchTime: 5h53m - DataArrivalTimer: 5h52m SenderSide: - DeserializeRowBatchTime: 3h28m - NumBatchesArrived: 7.53M (7533826) - NumBatchesDeferred: 105.78K (105781) - NumBatchesEnqueued: 7.53M (7533826) - NumBatchesReceived: 7.53M (7533826) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 57.446ms, non-child: 57.446ms, % non-child: 100.00%) - CodegenTime: 1.105ms - CompileTime: 4.959ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.838ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.176ms Instance 844a0200d7876345:20bb38b9000000f5 (host=vd1115.foo.com:22000):(Total: 1s166ms, non-child: 193.469ms, % non-child: 16.58%) Fragment Instance Lifecycle Event Timeline: 1s167ms - Prepare Finished: 192.882ms (192.882ms) - Open Finished: 1s167ms (974.280ms) MemoryUsage(8m32s): 20.73 GB, 36.69 GB, 45.18 GB, 52.69 GB, 59.57 GB, 65.64 GB, 70.88 GB, 75.82 GB, 80.53 GB, 85.16 GB, 89.43 GB, 93.34 GB, 97.17 GB, 100.94 GB, 104.50 GB, 107.96 GB, 111.18 GB, 114.28 GB, 117.23 GB, 120.06 GB, 122.81 GB, 125.42 GB, 127.92 GB, 130.52 GB, 133.02 GB, 135.43 GB, 137.88 GB, 140.30 GB, 142.74 GB, 145.19 GB, 146.72 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.75 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB, 146.76 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 146.76 GB (157582180790) - PeakReservation: 146.72 GB (157542252544) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.73 GB (172577584602) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h50m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 974.253ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 63.990ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 974.256ms - ExecTreeOpenTime: 908.714ms - PrepareTime: 192.280ms - ExecTreePrepareTime: 160.724us KuduTableSink:(Total: 16.762us, non-child: 16.762us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 908.872ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 146.72 GB (157542260736) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 32m48s - CumulativeAllocationBytes: 146.72 GB (157542252544) - CumulativeAllocations: 75.12K (75122) - PeakReservation: 146.72 GB (157542252544) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 146.72 GB (157542252544) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h51m, non-child: 47s497ms, % non-child: 0.23%) - ConvertRowBatchTime: 19s815ms - PeakMemoryUsage: 19.53 MB (20483762) - RowsReturned: 575.39M (575389584) - RowsReturnedRate: 27.31 K/sec Buffer pool: - AllocTime: 4h31m - CumulativeAllocationBytes: 261.30 GB (280570920960) - CumulativeAllocations: 13.70M (13699752) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.95 GB, 37.06 GB, 45.64 GB, 53.24 GB, 60.19 GB, 66.32 GB, 71.62 GB, 76.61 GB, 81.38 GB, 86.05 GB, 90.37 GB, 94.33 GB, 98.19 GB, 102.00 GB, 105.60 GB, 109.10 GB, 112.35 GB, 115.49 GB, 118.47 GB, 121.33 GB, 124.11 GB, 126.76 GB, 129.28 GB, 131.90 GB, 134.43 GB, 136.87 GB, 139.35 GB, 141.80 GB, 144.26 GB, 146.73 GB, 148.28 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.31 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB, 148.32 GB - FirstBatchArrivalWaitTime: 908.643ms - TotalBytesReceived: 148.32 GB (159259386202) - TotalGetBatchTime: 5h50m - DataArrivalTimer: 5h50m SenderSide: - DeserializeRowBatchTime: 4h41m - NumBatchesArrived: 6.85M (6849876) - NumBatchesDeferred: 99.05K (99054) - NumBatchesEnqueued: 6.85M (6849876) - NumBatchesReceived: 6.85M (6849876) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 64.231ms, non-child: 64.231ms, % non-child: 100.00%) - CodegenTime: 1.195ms - CompileTime: 5.568ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.320ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 26.722ms Instance 844a0200d7876345:20bb38b9000000af (host=vd1330.foo.com:22000):(Total: 1s159ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h18m - Prepare Finished: 197.183ms (197.183ms) - Open Finished: 1s160ms (962.976ms) - First Batch Produced: 7h18m (7h18m) - First Batch Sent: 7h18m (81.592ms) MemoryUsage(8m32s): 22.80 GB, 40.35 GB, 49.70 GB, 57.97 GB, 65.54 GB, 72.21 GB, 77.97 GB, 83.40 GB, 88.59 GB, 93.67 GB, 98.37 GB, 102.67 GB, 106.88 GB, 111.03 GB, 114.94 GB, 118.75 GB, 122.29 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.09 GB, 137.97 GB, 140.71 GB, 143.57 GB, 146.32 GB, 148.97 GB, 151.67 GB, 154.33 GB, 157.01 GB, 159.70 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173326965518) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.13 GB (186974007320) - RowsProduced: 221.94M (221944914) - TotalNetworkReceiveTime: 5h49m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 962.960ms - TotalThreadsSysTime: 999.000us - TotalThreadsUserTime: 60.990ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h18m - OpenTime: 962.964ms - ExecTreeOpenTime: 901.044ms - PrepareTime: 196.664ms - ExecTreePrepareTime: 161.015us KuduTableSink:(Total: 42m41s, non-child: 42m41s, % non-child: 100.00%) - KuduApplyTimer: 34m41s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 86.64 K/sec - TotalNumRows: 221.94M (221943890) SORT_NODE (id=2):(Total: 7h18m, non-child: 1h29m, % non-child: 20.34%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 35m11s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 221.94M (221944914) - RowsReturnedRate: 8.43 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276244136) Buffer pool: - AllocTime: 34m16s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h49m, non-child: 51s566ms, % non-child: 0.25%) - ConvertRowBatchTime: 21s169ms - PeakMemoryUsage: 19.53 MB (20482260) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 30.18 K/sec Buffer pool: - AllocTime: 3h55m - CumulativeAllocationBytes: 287.43 GB (308628398080) - CumulativeAllocations: 15.07M (15069746) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.05 GB, 40.76 GB, 50.21 GB, 58.57 GB, 66.22 GB, 72.96 GB, 78.79 GB, 84.28 GB, 89.52 GB, 94.66 GB, 99.41 GB, 103.76 GB, 108.01 GB, 112.20 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.32 GB, 133.46 GB, 136.53 GB, 139.44 GB, 142.21 GB, 145.10 GB, 147.88 GB, 150.56 GB, 153.29 GB, 155.98 GB, 158.69 GB, 161.41 GB, 163.11 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 900.957ms - TotalBytesReceived: 163.15 GB (175185469448) - TotalGetBatchTime: 5h49m - DataArrivalTimer: 5h48m SenderSide: - DeserializeRowBatchTime: 4h5m - NumBatchesArrived: 7.53M (7534995) - NumBatchesDeferred: 153.35K (153352) - NumBatchesEnqueued: 7.53M (7534873) - NumBatchesReceived: 7.53M (7534873) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 60.560ms, non-child: 60.560ms, % non-child: 100.00%) - CodegenTime: 1.233ms - CompileTime: 4.740ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 31.774ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 23.278ms Instance 844a0200d7876345:20bb38b9000000a5 (host=vc1537.foo.com:22000):(Total: 1s158ms, non-child: 0.000ns, % non-child: 0.00%) Fragment Instance Lifecycle Event Timeline: 7h22m - Prepare Finished: 191.882ms (191.882ms) - Open Finished: 1s158ms (966.932ms) - First Batch Produced: 7h22m (7h22m) - First Batch Sent: 7h22m (10.117ms) MemoryUsage(8m32s): 22.82 GB, 40.35 GB, 49.70 GB, 57.96 GB, 65.53 GB, 72.20 GB, 77.96 GB, 83.40 GB, 88.58 GB, 93.67 GB, 98.37 GB, 102.68 GB, 106.88 GB, 111.03 GB, 114.95 GB, 118.76 GB, 122.29 GB, 125.71 GB, 128.95 GB, 132.06 GB, 135.09 GB, 137.97 GB, 140.72 GB, 143.57 GB, 146.32 GB, 148.98 GB, 151.67 GB, 154.33 GB, 157.01 GB, 159.70 GB, 161.39 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB, 161.42 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.42 GB (173326081546) - PeakReservation: 161.38 GB (173285572608) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.25 GB (188169539928) - RowsProduced: 205.89M (205892468) - TotalNetworkReceiveTime: 5h39m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 966.917ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 62.990ms - TotalThreadsVoluntaryContextSwitches: 7 (7) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7h22m - OpenTime: 966.921ms - ExecTreeOpenTime: 903.113ms - PrepareTime: 191.461ms - ExecTreePrepareTime: 202.036us KuduTableSink:(Total: 39m6s, non-child: 39m6s, % non-child: 100.00%) - KuduApplyTimer: 31m48s - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 87.73 K/sec - TotalNumRows: 205.89M (205891444) SORT_NODE (id=2):(Total: 7h22m, non-child: 1h42m, % non-child: 23.17%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: (Avg: 632.89M (632891204) ; Min: 632.89M (632891204) ; Max: 632.89M (632891204) ; Number of samples: 1) - InMemorySortTime: 38m42s - PeakMemoryUsage: 161.38 GB (173285588992) - RowsReturned: 205.89M (205892468) - RowsReturnedRate: 7.76 K/sec - RunsCreated: 1 (1) - SortDataSize: 161.38 GB (173276444307) Buffer pool: - AllocTime: 41m53s - CumulativeAllocationBytes: 161.38 GB (173285572608) - CumulativeAllocations: 82.63K (82629) - PeakReservation: 161.38 GB (173285572608) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.38 GB (173285572608) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h39m, non-child: 50s881ms, % non-child: 0.25%) - ConvertRowBatchTime: 20s844ms - PeakMemoryUsage: 19.53 MB (20481905) - RowsReturned: 632.89M (632891392) - RowsReturnedRate: 31.04 K/sec Buffer pool: - AllocTime: 4h36m - CumulativeAllocationBytes: 287.43 GB (308628357120) - CumulativeAllocations: 15.07M (15069744) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.03 GB, 40.75 GB, 50.20 GB, 58.56 GB, 66.20 GB, 72.94 GB, 78.77 GB, 84.27 GB, 89.51 GB, 94.65 GB, 99.40 GB, 103.76 GB, 108.01 GB, 112.20 GB, 116.16 GB, 120.01 GB, 123.59 GB, 127.04 GB, 130.32 GB, 133.46 GB, 136.53 GB, 139.44 GB, 142.22 GB, 145.10 GB, 147.88 GB, 150.56 GB, 153.28 GB, 155.98 GB, 158.68 GB, 161.41 GB, 163.11 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.14 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB, 163.15 GB - FirstBatchArrivalWaitTime: 903.039ms - TotalBytesReceived: 163.15 GB (175185652641) - TotalGetBatchTime: 5h39m - DataArrivalTimer: 5h38m SenderSide: - DeserializeRowBatchTime: 4h55m - NumBatchesArrived: 7.53M (7534920) - NumBatchesDeferred: 298.60K (298604) - NumBatchesEnqueued: 7.53M (7534872) - NumBatchesReceived: 7.53M (7534872) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 62.391ms, non-child: 62.391ms, % non-child: 100.00%) - CodegenTime: 1.062ms - CompileTime: 4.764ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 32.688ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.238ms Instance 844a0200d7876345:20bb38b900000090 (host=ve1136.foo.com:22000):(Total: 1s156ms, non-child: 199.873ms, % non-child: 17.28%) Fragment Instance Lifecycle Event Timeline: 1s157ms - Prepare Finished: 199.451ms (199.451ms) - Open Finished: 1s157ms (957.597ms) MemoryUsage(8m32s): 22.81 GB, 40.34 GB, 49.69 GB, 57.95 GB, 65.51 GB, 72.18 GB, 77.95 GB, 83.38 GB, 88.57 GB, 93.65 GB, 98.35 GB, 102.66 GB, 106.87 GB, 111.02 GB, 114.93 GB, 118.74 GB, 122.28 GB, 125.69 GB, 128.93 GB, 132.04 GB, 135.07 GB, 137.95 GB, 140.70 GB, 143.55 GB, 146.30 GB, 148.95 GB, 151.64 GB, 154.31 GB, 156.98 GB, 159.67 GB, 161.36 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.39 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB, 161.40 GB ThreadUsage(8m32s): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 161.40 GB (173304529334) - PeakReservation: 161.37 GB (173264601088) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.06 GB (186900104333) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 5h49m - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 957.581ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 55.991ms - TotalThreadsVoluntaryContextSwitches: 15 (15) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 0.000ns - OpenTime: 957.584ms - ExecTreeOpenTime: 898.285ms - PrepareTime: 198.814ms - ExecTreePrepareTime: 176.373us KuduTableSink:(Total: 12.789us, non-child: 12.789us, % non-child: 100.00%) - KuduApplyTimer: 0.000ns - NumRowErrors: 0 (0) - PeakMemoryUsage: 20.00 MB (20971520) - RowsProcessedRate: 0 - TotalNumRows: 0 (0) SORT_NODE (id=2):(Total: 898.458ms, non-child: 0.000ns, % non-child: 0.00%) SortType: Partial ExecOption: Codegen Enabled - NumRowsPerRun: 0 (0) (Number of samples: 0) - InMemorySortTime: 0.000ns - PeakMemoryUsage: 161.37 GB (173264609280) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - RunsCreated: 1 (1) - SortDataSize: 0 Buffer pool: - AllocTime: 32m54s - CumulativeAllocationBytes: 161.37 GB (173264601088) - CumulativeAllocations: 82.62K (82619) - PeakReservation: 161.37 GB (173264601088) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 161.37 GB (173264601088) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns EXCHANGE_NODE (id=1):(Total: 5h49m, non-child: 50s428ms, % non-child: 0.24%) - ConvertRowBatchTime: 20s895ms - PeakMemoryUsage: 19.53 MB (20481135) - RowsReturned: 632.82M (632817612) - RowsReturnedRate: 30.18 K/sec Buffer pool: - AllocTime: 2h50m - CumulativeAllocationBytes: 287.38 GB (308573921280) - CumulativeAllocations: 15.07M (15067086) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 23.02 GB, 40.74 GB, 50.19 GB, 58.54 GB, 66.19 GB, 72.93 GB, 78.76 GB, 84.25 GB, 89.49 GB, 94.64 GB, 99.39 GB, 103.74 GB, 107.99 GB, 112.19 GB, 116.14 GB, 120.00 GB, 123.57 GB, 127.03 GB, 130.30 GB, 133.44 GB, 136.51 GB, 139.42 GB, 142.19 GB, 145.08 GB, 147.86 GB, 150.54 GB, 153.26 GB, 155.95 GB, 158.66 GB, 161.38 GB, 163.08 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.11 GB, 163.12 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB, 163.13 GB - FirstBatchArrivalWaitTime: 898.223ms - TotalBytesReceived: 163.13 GB (175154790586) - TotalGetBatchTime: 5h49m - DataArrivalTimer: 5h48m SenderSide: - DeserializeRowBatchTime: 2h59m - NumBatchesArrived: 7.53M (7533543) - NumBatchesDeferred: 108.58K (108578) - NumBatchesEnqueued: 7.53M (7533543) - NumBatchesReceived: 7.53M (7533543) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0) CodeGen:(Total: 58.111ms, non-child: 58.111ms, % non-child: 100.00%) - CodegenTime: 1.001ms - CompileTime: 4.800ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 32 (32) - NumInstructions: 612 (612) - OptimizationTime: 28.271ms - PeakMemoryUsage: 306.00 KB (313344) - PrepareTime: 24.519ms Averaged Fragment F00:(Total: 246.759ms, non-child: 0.000ns, % non-child: 0.00%) split sizes: min: 54.11 GB, max: 55.17 GB, avg: 54.69 GB, stddev: 196.43 MB completion times: min:8h1m max:8h1m mean: 8h1m stddev:40.928ms execution rates: min:1.92 MB/sec max:1.95 MB/sec mean:1.94 MB/sec stddev:6.96 KB/sec num instances: 129 - AverageThreadTokens: 86.02 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.50 GB (14495323934) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 165.88 GB (178117160539) - RowsProduced: 597.67M (597666379) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 26m32s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 31.975ms - TotalThreadsSysTime: 1.302ms - TotalThreadsUserTime: 30.111ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s532ms - OpenTime: 31.976ms - ExecTreeOpenTime: 200.684us - PrepareTime: 214.703ms - ExecTreePrepareTime: 1.970ms KrpcDataStreamSender (dst_id=1):(Total: 11m17s, non-child: 11m17s, % non-child: 100.00%) - BytesSent: 78.45 GB (84234996688) - EosSent: 0 (0) - OverallThroughput: 118.70 MB/sec - PeakMemoryUsage: 3.60 MB (3779851) - RowsReturned: 597.67M (597665355) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m15s - UncompressedRowBatchSize: 154.06 GB (165423704104) HDFS_SCAN_NODE (id=0):(Total: 10s019ms, non-child: 10s019ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 85.02 - BytesRead: 33.69 GB (36177682140) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m13s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 170 (170) - NumScannerThreadsStarted: 85 (85) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.50 GB (14492206093) - PerReadThreadRawHdfsThroughput: 51.12 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 597.88M (597876243) - RowsReturned: 597.67M (597666379) - RowsReturnedRate: 64.25 M/sec - ScanRangesComplete: 85 (85) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m35s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 24s496ms - TotalRawHdfsReadTime(*): 11m18s - TotalReadThroughput: 1.19 MB/sec CodeGen:(Total: 31.467ms, non-child: 31.467ms, % non-child: 100.00%) - CodegenTime: 428.499us - CompileTime: 4.328ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.353ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 19.227ms Fragment F00: Instance 844a0200d7876345:20bb38b900000063 (host=va1026.foo.com:22000):(Total: 304.784ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:29/7.05 GB 0:19/4.70 GB 5:30/7.32 GB 3:24/5.75 GB 4:30/7.31 GB 1:39/9.55 GB 2:32/7.85 GB 6:21/5.16 GB Fragment Instance Lifecycle Event Timeline: 1s813ms - Prepare Finished: 279.259ms (279.259ms) - Open Finished: 305.395ms (26.135ms) - First Batch Produced: 1s811ms (1s506ms) - First Batch Sent: 1s813ms (2.012ms) MemoryUsage(8m32s): 13.23 GB, 13.98 GB, 13.98 GB, 13.97 GB, 13.89 GB, 13.65 GB, 13.31 GB, 13.21 GB, 13.14 GB, 12.91 GB, 12.41 GB, 11.66 GB, 11.28 GB, 10.72 GB, 10.39 GB, 10.30 GB, 10.17 GB, 9.88 GB, 9.62 GB, 8.97 GB, 8.68 GB, 8.70 GB, 8.63 GB, 8.38 GB, 8.17 GB, 8.07 GB, 8.66 GB, 9.48 GB, 10.84 GB, 13.09 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.98 GB (15012267519) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.62 GB (172464205556) - RowsProduced: 577.41M (577412649) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h17m - TotalStorageWaitTime: 28m15s - TotalThreadsInvoluntaryContextSwitches: 27 (27) - TotalThreadsTotalWallClockTime: 26.132ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 24.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 6s459ms - OpenTime: 26.133ms - ExecTreeOpenTime: 267.327us - PrepareTime: 278.586ms - ExecTreePrepareTime: 1.806ms KrpcDataStreamSender (dst_id=1):(Total: 11m50s, non-child: 11m50s, % non-child: 100.00%) - BytesSent: 75.84 GB (81434801297) - EosSent: 0 (0) - OverallThroughput: 109.26 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 577.41M (577411625) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m49s - UncompressedRowBatchSize: 148.84 GB (159817779311) HDFS_SCAN_NODE (id=0):(Total: 5s878ms, non-child: 5s878ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:29/7.05 GB 0:19/4.70 GB 5:30/7.32 GB 3:24/5.75 GB 4:30/7.31 GB 1:39/9.55 GB 2:32/7.85 GB 6:21/5.16 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.76 GB, 16.14 GB, 16.99 GB, 17.54 GB, 18.21 GB, 19.40 GB, 19.82 GB, 20.10 GB, 20.44 GB, 20.53 GB, 20.70 GB, 21.00 GB, 21.09 GB, 21.16 GB, 21.26 GB, 21.45 GB, 21.48 GB, 21.48 GB, 21.70 GB, 21.91 GB, 22.18 GB, 22.47 GB, 22.84 GB, 23.07 GB, 23.40 GB, 23.85 GB, 25.04 GB, 26.46 GB, 28.91 GB, 32.81 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB, 34.11 GB - FooterProcessingTime: (Avg: 2s193ms ; Min: 60.906ms ; Max: 15s245ms ; Number of samples: 175) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.11 GB (36625764813) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 4m48s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 175 (175) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.98 GB (15009331516) - PerReadThreadRawHdfsThroughput: 52.19 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 577.63M (577625641) - RowsReturned: 577.41M (577412649) - RowsReturnedRate: 98.23 M/sec - ScanRangesComplete: 87 (87) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m36s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 44s656ms - TotalRawHdfsReadTime(*): 11m9s - TotalReadThroughput: 1.21 MB/sec CodeGen:(Total: 25.609ms, non-child: 25.609ms, % non-child: 100.00%) - CodegenTime: 411.360us - CompileTime: 4.592ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.592ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 13.905ms Instance 844a0200d7876345:20bb38b90000003c (host=vc1504.foo.com:22000):(Total: 290.816ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:28/6.90 GB 7:28/6.90 GB 3:33/8.05 GB 1:27/6.61 GB 6:27/6.64 GB 2:24/5.85 GB 4:28/6.86 GB 0:29/7.05 GB Fragment Instance Lifecycle Event Timeline: 1s488ms - Prepare Finished: 265.298ms (265.298ms) - Open Finished: 291.315ms (26.017ms) - First Batch Produced: 1s486ms (1s195ms) - First Batch Sent: 1s488ms (1.926ms) MemoryUsage(8m32s): 13.22 GB, 13.99 GB, 13.99 GB, 13.96 GB, 13.84 GB, 13.58 GB, 13.36 GB, 13.32 GB, 13.16 GB, 12.71 GB, 12.01 GB, 11.48 GB, 10.99 GB, 10.59 GB, 10.50 GB, 10.37 GB, 10.10 GB, 9.76 GB, 9.30 GB, 8.83 GB, 8.65 GB, 8.53 GB, 8.19 GB, 7.80 GB, 7.87 GB, 8.42 GB, 9.37 GB, 11.03 GB, 12.75 GB, 13.81 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15025363109) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.71 GB (172563104240) - RowsProduced: 589.30M (589296618) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 29m16s - TotalThreadsInvoluntaryContextSwitches: 2 (2) - TotalThreadsTotalWallClockTime: 26.014ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 24.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s502ms - OpenTime: 26.016ms - ExecTreeOpenTime: 249.787us - PrepareTime: 264.759ms - ExecTreePrepareTime: 1.512ms KrpcDataStreamSender (dst_id=1):(Total: 11m9s, non-child: 11m9s, % non-child: 100.00%) - BytesSent: 77.34 GB (83048166211) - EosSent: 0 (0) - OverallThroughput: 118.36 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 589.30M (589295594) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m7s - UncompressedRowBatchSize: 151.91 GB (163107177849) HDFS_SCAN_NODE (id=0):(Total: 7s999ms, non-child: 7s999ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:28/6.90 GB 7:28/6.90 GB 3:33/8.05 GB 1:27/6.61 GB 6:27/6.64 GB 2:24/5.85 GB 4:28/6.86 GB 0:29/7.05 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.86 GB, 16.34 GB, 17.14 GB, 17.64 GB, 18.64 GB, 19.64 GB, 20.02 GB, 20.36 GB, 20.56 GB, 20.72 GB, 21.00 GB, 21.17 GB, 21.24 GB, 21.35 GB, 21.51 GB, 21.59 GB, 21.64 GB, 21.90 GB, 22.13 GB, 22.16 GB, 22.35 GB, 22.69 GB, 22.75 GB, 22.85 GB, 23.49 GB, 24.71 GB, 26.41 GB, 29.37 GB, 32.32 GB, 34.18 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB, 34.44 GB - FooterProcessingTime: (Avg: 2s584ms ; Min: 27.773ms ; Max: 9s824ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.44 GB (36981748784) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 12m4s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.99 GB (15022198140) - PerReadThreadRawHdfsThroughput: 54.39 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 589.51M (589509610) - RowsReturned: 589.30M (589296618) - RowsReturnedRate: 73.66 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m49s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 22s935ms - TotalRawHdfsReadTime(*): 10m48s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 25.496ms, non-child: 25.496ms, % non-child: 100.00%) - CodegenTime: 396.555us - CompileTime: 3.890ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.618ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.575ms Instance 844a0200d7876345:20bb38b900000051 (host=vc1538.foo.com:22000):(Total: 285.262ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:23/5.49 GB 4:26/6.28 GB 7:23/5.69 GB 5:23/5.65 GB 0:36/8.87 GB 3:31/7.63 GB 2:33/8.10 GB 6:29/7.07 GB Fragment Instance Lifecycle Event Timeline: 1s722ms - Prepare Finished: 261.183ms (261.183ms) - Open Finished: 285.692ms (24.509ms) - First Batch Produced: 1s719ms (1s434ms) - First Batch Sent: 1s722ms (2.332ms) MemoryUsage(8m32s): 13.13 GB, 13.97 GB, 13.94 GB, 13.86 GB, 13.64 GB, 13.30 GB, 13.17 GB, 12.99 GB, 12.64 GB, 11.93 GB, 11.37 GB, 10.88 GB, 10.54 GB, 10.34 GB, 10.15 GB, 9.94 GB, 9.47 GB, 9.06 GB, 8.79 GB, 8.82 GB, 8.54 GB, 8.47 GB, 8.44 GB, 8.69 GB, 9.38 GB, 10.25 GB, 12.20 GB, 13.35 GB, 13.93 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.98 GB (15007952742) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.70 GB (172549140051) - RowsProduced: 606.65M (606654024) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 32m54s - TotalThreadsInvoluntaryContextSwitches: 2 (2) - TotalThreadsTotalWallClockTime: 24.504ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 23.997ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s873ms - OpenTime: 24.506ms - ExecTreeOpenTime: 247.536us - PrepareTime: 260.685ms - ExecTreePrepareTime: 1.868ms KrpcDataStreamSender (dst_id=1):(Total: 11m7s, non-child: 11m7s, % non-child: 100.00%) - BytesSent: 79.69 GB (85561356527) - EosSent: 0 (0) - OverallThroughput: 122.29 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 606.65M (606653000) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m5s - UncompressedRowBatchSize: 156.38 GB (167911145029) HDFS_SCAN_NODE (id=0):(Total: 10s363ms, non-child: 10s363ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:23/5.49 GB 4:26/6.28 GB 7:23/5.69 GB 5:23/5.65 GB 0:36/8.87 GB 3:31/7.63 GB 2:33/8.10 GB 6:29/7.07 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.25 GB, 16.78 GB, 17.47 GB, 18.13 GB, 19.31 GB, 19.89 GB, 20.22 GB, 20.50 GB, 20.63 GB, 20.88 GB, 21.11 GB, 21.22 GB, 21.41 GB, 21.57 GB, 21.64 GB, 21.79 GB, 21.92 GB, 22.29 GB, 22.56 GB, 22.96 GB, 23.12 GB, 23.66 GB, 24.25 GB, 25.03 GB, 26.37 GB, 27.95 GB, 31.35 GB, 33.37 GB, 34.42 GB, 34.56 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB, 34.67 GB - FooterProcessingTime: (Avg: 3s187ms ; Min: 13.963ms ; Max: 12s239ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.67 GB (37229583087) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m25s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (15004805925) - PerReadThreadRawHdfsThroughput: 48.15 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 606.87M (606867016) - RowsReturned: 606.65M (606654024) - RowsReturnedRate: 58.53 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m40s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 36s151ms - TotalRawHdfsReadTime(*): 12m17s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 23.996ms, non-child: 23.996ms, % non-child: 100.00%) - CodegenTime: 353.846us - CompileTime: 4.180ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.583ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 12.773ms Instance 844a0200d7876345:20bb38b900000018 (host=vd1101.foo.com:22000):(Total: 280.739ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:24/5.90 GB 0:33/8.10 GB 3:36/8.64 GB 2:33/8.09 GB 7:30/7.28 GB 6:24/5.90 GB 4:14/3.38 GB 1:30/7.23 GB Fragment Instance Lifecycle Event Timeline: 1s712ms - Prepare Finished: 240.901ms (240.901ms) - Open Finished: 281.236ms (40.334ms) - First Batch Produced: 1s711ms (1s429ms) - First Batch Sent: 1s712ms (1.818ms) MemoryUsage(8m32s): 13.13 GB, 13.98 GB, 13.98 GB, 13.96 GB, 13.81 GB, 13.50 GB, 13.31 GB, 13.24 GB, 12.94 GB, 12.48 GB, 11.70 GB, 11.27 GB, 10.81 GB, 10.51 GB, 10.37 GB, 10.05 GB, 9.68 GB, 9.14 GB, 8.53 GB, 8.57 GB, 8.46 GB, 8.32 GB, 8.10 GB, 8.07 GB, 8.80 GB, 9.77 GB, 10.72 GB, 12.39 GB, 13.56 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15017378019) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.27 GB (188192130616) - RowsProduced: 597.34M (597342806) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 32m14s - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 40.330ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 37.994ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s901ms - OpenTime: 40.331ms - ExecTreeOpenTime: 266.450us - PrepareTime: 240.342ms - ExecTreePrepareTime: 2.186ms KrpcDataStreamSender (dst_id=1):(Total: 11m13s, non-child: 11m13s, % non-child: 100.00%) - BytesSent: 78.40 GB (84180428638) - EosSent: 0 (0) - OverallThroughput: 119.27 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 597.34M (597341782) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m11s - UncompressedRowBatchSize: 153.98 GB (165334313667) HDFS_SCAN_NODE (id=0):(Total: 8s382ms, non-child: 8s382ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:24/5.90 GB 0:33/8.10 GB 3:36/8.64 GB 2:33/8.09 GB 7:30/7.28 GB 6:24/5.90 GB 4:14/3.38 GB 1:30/7.23 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.94 GB, 16.56 GB, 17.25 GB, 17.93 GB, 18.98 GB, 19.80 GB, 20.11 GB, 20.46 GB, 20.65 GB, 20.83 GB, 21.10 GB, 21.22 GB, 21.32 GB, 21.44 GB, 21.57 GB, 21.60 GB, 21.60 GB, 21.60 GB, 21.68 GB, 22.29 GB, 22.58 GB, 22.92 GB, 23.11 GB, 23.69 GB, 25.31 GB, 27.02 GB, 28.76 GB, 31.66 GB, 33.67 GB, 34.21 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB - FooterProcessingTime: (Avg: 2s827ms ; Min: 29.427ms ; Max: 19s174ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.35 GB (36887605428) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m43s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.98 GB (15014303065) - PerReadThreadRawHdfsThroughput: 37.74 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 597.55M (597545558) - RowsReturned: 597.34M (597342806) - RowsReturnedRate: 71.26 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m10s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 41s879ms - TotalRawHdfsReadTime(*): 15m32s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 39.749ms, non-child: 39.749ms, % non-child: 100.00%) - CodegenTime: 454.386us - CompileTime: 4.541ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.142ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 27.603ms Instance 844a0200d7876345:20bb38b90000001a (host=vc1534.foo.com:22000):(Total: 280.518ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 6:31/7.17 GB 4:30/7.34 GB 3:25/6.14 GB 5:21/4.97 GB 0:34/8.30 GB 1:29/7.10 GB 2:29/7.14 GB 7:27/6.52 GB Fragment Instance Lifecycle Event Timeline: 1s865ms - Prepare Finished: 253.190ms (253.190ms) - Open Finished: 281.006ms (27.816ms) - First Batch Produced: 1s863ms (1s582ms) - First Batch Sent: 1s865ms (2.495ms) MemoryUsage(8m32s): 13.09 GB, 13.75 GB, 13.77 GB, 13.87 GB, 13.78 GB, 13.50 GB, 13.28 GB, 13.21 GB, 13.05 GB, 12.66 GB, 11.82 GB, 11.43 GB, 10.90 GB, 10.50 GB, 10.39 GB, 10.35 GB, 10.10 GB, 9.88 GB, 9.15 GB, 9.07 GB, 8.98 GB, 8.79 GB, 8.56 GB, 8.26 GB, 8.21 GB, 8.72 GB, 9.44 GB, 11.15 GB, 13.36 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.92 GB (14945796419) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.18 GB (188095084451) - RowsProduced: 590.46M (590455071) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 27m45s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 27.809ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 26.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 11s807ms - OpenTime: 27.811ms - ExecTreeOpenTime: 247.725us - PrepareTime: 252.644ms - ExecTreePrepareTime: 2.102ms KrpcDataStreamSender (dst_id=1):(Total: 11m15s, non-child: 11m15s, % non-child: 100.00%) - BytesSent: 77.48 GB (83188674969) - EosSent: 0 (0) - OverallThroughput: 117.48 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 590.45M (590454047) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m14s - UncompressedRowBatchSize: 152.21 GB (163428929379) HDFS_SCAN_NODE (id=0):(Total: 11s281ms, non-child: 11s281ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:31/7.17 GB 4:30/7.34 GB 3:25/6.14 GB 5:21/4.97 GB 0:34/8.30 GB 1:29/7.10 GB 2:29/7.14 GB 7:27/6.52 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.74 GB, 16.23 GB, 16.94 GB, 17.65 GB, 18.67 GB, 19.64 GB, 19.94 GB, 20.34 GB, 20.53 GB, 20.67 GB, 20.98 GB, 21.14 GB, 21.19 GB, 21.29 GB, 21.51 GB, 21.70 GB, 21.75 GB, 22.08 GB, 22.26 GB, 22.69 GB, 22.79 GB, 23.05 GB, 23.30 GB, 23.58 GB, 24.06 GB, 25.06 GB, 26.36 GB, 29.49 GB, 33.35 GB, 34.08 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB, 34.15 GB - FooterProcessingTime: (Avg: 2s522ms ; Min: 28.261ms ; Max: 13s976ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.15 GB (36664405058) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 11m55s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.92 GB (14942754360) - PerReadThreadRawHdfsThroughput: 52.98 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 590.67M (590668063) - RowsReturned: 590.46M (590455071) - RowsReturnedRate: 52.34 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m41s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 29s532ms - TotalRawHdfsReadTime(*): 11m - TotalReadThroughput: 1.21 MB/sec CodeGen:(Total: 27.274ms, non-child: 27.274ms, % non-child: 100.00%) - CodegenTime: 349.921us - CompileTime: 4.535ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.797ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 15.479ms Instance 844a0200d7876345:20bb38b900000072 (host=ve1131.foo.com:22000):(Total: 278.750ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:24/5.59 GB 1:27/6.36 GB 7:26/6.32 GB 6:31/7.59 GB 4:31/7.60 GB 3:27/6.61 GB 0:30/7.34 GB 5:31/7.28 GB Fragment Instance Lifecycle Event Timeline: 1s940ms - Prepare Finished: 245.884ms (245.884ms) - Open Finished: 279.215ms (33.331ms) - First Batch Produced: 1s939ms (1s660ms) - First Batch Sent: 1s940ms (1.062ms) MemoryUsage(8m32s): 12.03 GB, 12.65 GB, 12.63 GB, 12.51 GB, 12.23 GB, 12.08 GB, 11.97 GB, 11.67 GB, 10.95 GB, 10.37 GB, 9.81 GB, 9.53 GB, 9.33 GB, 9.02 GB, 8.60 GB, 8.02 GB, 7.90 GB, 7.70 GB, 7.62 GB, 7.80 GB, 8.24 GB, 9.22 GB, 10.93 GB, 12.40 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.68 GB (13618036046) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.41 GB (171161855986) - RowsProduced: 596.20M (596199049) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h18m - TotalStorageWaitTime: 24m - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 33.328ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 30.995ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 6s527ms - OpenTime: 33.329ms - ExecTreeOpenTime: 215.537us - PrepareTime: 245.359ms - ExecTreePrepareTime: 2.861ms KrpcDataStreamSender (dst_id=1):(Total: 11m21s, non-child: 11m21s, % non-child: 100.00%) - BytesSent: 78.27 GB (84045328257) - EosSent: 0 (0) - OverallThroughput: 117.62 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 596.20M (596198025) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m20s - UncompressedRowBatchSize: 153.68 GB (165017018950) HDFS_SCAN_NODE (id=0):(Total: 5s981ms, non-child: 5s981ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:24/5.59 GB 1:27/6.36 GB 7:26/6.32 GB 6:31/7.59 GB 4:31/7.60 GB 3:27/6.61 GB 0:30/7.34 GB 5:31/7.28 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.17 GB, 15.27 GB, 15.90 GB, 16.64 GB, 17.78 GB, 18.29 GB, 18.69 GB, 18.83 GB, 19.08 GB, 19.32 GB, 19.39 GB, 19.56 GB, 19.68 GB, 19.69 GB, 19.95 GB, 20.24 GB, 20.55 GB, 20.83 GB, 21.41 GB, 22.43 GB, 23.41 GB, 25.15 GB, 28.17 GB, 30.76 GB, 31.33 GB, 31.43 GB, 31.58 GB, 31.82 GB, 32.11 GB, 32.35 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB - FooterProcessingTime: (Avg: 2s249ms ; Min: 16.027ms ; Max: 12s002ms ; Number of samples: 162) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.48 GB (34875476727) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 5m45s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 162 (162) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.68 GB (13614870279) - PerReadThreadRawHdfsThroughput: 50.93 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 596.40M (596403849) - RowsReturned: 596.20M (596199049) - RowsReturnedRate: 99.67 M/sec - ScanRangesComplete: 82 (82) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m13s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 21s426ms - TotalRawHdfsReadTime(*): 10m53s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 32.809ms, non-child: 32.809ms, % non-child: 100.00%) - CodegenTime: 407.034us - CompileTime: 4.184ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.705ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 21.414ms Instance 844a0200d7876345:20bb38b900000034 (host=vc1528.foo.com:22000):(Total: 274.043ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:27/6.51 GB 7:27/6.64 GB 0:29/6.98 GB 6:29/7.11 GB 1:26/6.32 GB 2:27/6.61 GB 5:28/6.80 GB 3:32/7.73 GB Fragment Instance Lifecycle Event Timeline: 1s737ms - Prepare Finished: 249.432ms (249.432ms) - Open Finished: 274.556ms (25.124ms) - First Batch Produced: 1s735ms (1s460ms) - First Batch Sent: 1s737ms (2.274ms) MemoryUsage(8m32s): 13.14 GB, 13.89 GB, 13.87 GB, 13.91 GB, 13.70 GB, 13.40 GB, 13.28 GB, 13.11 GB, 12.76 GB, 12.11 GB, 11.48 GB, 11.06 GB, 10.65 GB, 10.45 GB, 10.22 GB, 9.94 GB, 9.51 GB, 8.86 GB, 8.58 GB, 8.51 GB, 8.47 GB, 8.25 GB, 8.29 GB, 8.92 GB, 9.53 GB, 10.62 GB, 11.92 GB, 13.35 GB, 13.88 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.95 GB (14979353499) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.63 GB (172475100443) - RowsProduced: 607.26M (607261289) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h17m - TotalStorageWaitTime: 29m32s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 25.120ms - TotalThreadsSysTime: 4.999ms - TotalThreadsUserTime: 19.997ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7s999ms - OpenTime: 25.121ms - ExecTreeOpenTime: 145.074us - PrepareTime: 248.836ms - ExecTreePrepareTime: 2.124ms KrpcDataStreamSender (dst_id=1):(Total: 11m28s, non-child: 11m28s, % non-child: 100.00%) - BytesSent: 79.77 GB (85652103595) - EosSent: 0 (0) - OverallThroughput: 118.68 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 607.26M (607260265) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m26s - UncompressedRowBatchSize: 156.54 GB (168079931128) HDFS_SCAN_NODE (id=0):(Total: 7s447ms, non-child: 7s447ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:27/6.51 GB 7:27/6.64 GB 0:29/6.98 GB 6:29/7.11 GB 1:26/6.32 GB 2:27/6.61 GB 5:28/6.80 GB 3:32/7.73 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.31 GB, 16.70 GB, 17.33 GB, 18.21 GB, 19.33 GB, 19.97 GB, 20.27 GB, 20.56 GB, 20.74 GB, 20.97 GB, 21.19 GB, 21.30 GB, 21.39 GB, 21.52 GB, 21.62 GB, 21.73 GB, 21.79 GB, 21.79 GB, 22.19 GB, 22.57 GB, 23.01 GB, 23.26 GB, 23.85 GB, 25.38 GB, 26.63 GB, 28.57 GB, 30.87 GB, 33.36 GB, 34.29 GB, 34.43 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB - FooterProcessingTime: (Avg: 2s878ms ; Min: 26.507ms ; Max: 12s935ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.51 GB (37052318416) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 6m24s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.95 GB (14976262161) - PerReadThreadRawHdfsThroughput: 51.80 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 607.47M (607474281) - RowsReturned: 607.26M (607261289) - RowsReturnedRate: 81.53 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m30s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 24s710ms - TotalRawHdfsReadTime(*): 11m22s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 24.676ms, non-child: 24.676ms, % non-child: 100.00%) - CodegenTime: 375.255us - CompileTime: 3.843ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.411ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 13.972ms Instance 844a0200d7876345:20bb38b90000006e (host=vd1122.foo.com:22000):(Total: 272.465ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:29/7.01 GB 1:29/7.10 GB 6:32/7.92 GB 3:20/4.91 GB 7:30/7.38 GB 5:31/7.59 GB 2:32/7.66 GB 0:22/5.38 GB Fragment Instance Lifecycle Event Timeline: 1s216ms - Prepare Finished: 239.757ms (239.757ms) - Open Finished: 273.258ms (33.500ms) - First Batch Produced: 1s214ms (941.514ms) - First Batch Sent: 1s216ms (2.149ms) MemoryUsage(8m32s): 13.13 GB, 13.99 GB, 13.98 GB, 13.88 GB, 13.69 GB, 13.42 GB, 13.30 GB, 13.14 GB, 12.79 GB, 12.08 GB, 11.56 GB, 11.07 GB, 10.65 GB, 10.43 GB, 10.25 GB, 10.03 GB, 9.73 GB, 9.10 GB, 8.78 GB, 8.55 GB, 8.46 GB, 8.40 GB, 8.34 GB, 8.54 GB, 9.01 GB, 10.06 GB, 12.05 GB, 13.20 GB, 13.80 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15019503427) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.65 GB (172495600241) - RowsProduced: 605.48M (605482246) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 29m1s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 33.489ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 29.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s752ms - OpenTime: 33.490ms - ExecTreeOpenTime: 197.285us - PrepareTime: 238.878ms - ExecTreePrepareTime: 1.786ms KrpcDataStreamSender (dst_id=1):(Total: 11m4s, non-child: 11m4s, % non-child: 100.00%) - BytesSent: 79.42 GB (85281508556) - EosSent: 0 (0) - OverallThroughput: 122.40 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 605.48M (605481222) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m3s - UncompressedRowBatchSize: 156.08 GB (167587236401) HDFS_SCAN_NODE (id=0):(Total: 10s278ms, non-child: 10s278ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:29/7.01 GB 1:29/7.10 GB 6:32/7.92 GB 3:20/4.91 GB 7:30/7.38 GB 5:31/7.59 GB 2:32/7.66 GB 0:22/5.38 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.33 GB, 16.79 GB, 17.47 GB, 18.16 GB, 19.39 GB, 19.97 GB, 20.31 GB, 20.58 GB, 20.70 GB, 20.98 GB, 21.21 GB, 21.29 GB, 21.38 GB, 21.55 GB, 21.62 GB, 21.74 GB, 22.06 GB, 22.24 GB, 22.36 GB, 22.47 GB, 22.87 GB, 23.37 GB, 24.03 GB, 24.73 GB, 25.73 GB, 27.63 GB, 31.14 GB, 33.17 GB, 34.27 GB, 34.47 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB - FooterProcessingTime: (Avg: 2s346ms ; Min: 16.179ms ; Max: 13s287ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.52 GB (37067752433) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m40s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.99 GB (15016367306) - PerReadThreadRawHdfsThroughput: 49.62 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 605.70M (605695238) - RowsReturned: 605.48M (605482246) - RowsReturnedRate: 58.91 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m24s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 21s469ms - TotalRawHdfsReadTime(*): 11m52s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 33.020ms, non-child: 33.020ms, % non-child: 100.00%) - CodegenTime: 457.837us - CompileTime: 4.321ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.790ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 20.406ms Instance 844a0200d7876345:20bb38b90000002a (host=vc1541.foo.com:22000):(Total: 271.721ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:26/6.36 GB 7:32/7.88 GB 3:20/4.82 GB 2:23/5.55 GB 5:26/6.35 GB 6:27/6.61 GB 0:33/8.13 GB 1:37/9.04 GB Fragment Instance Lifecycle Event Timeline: 1s654ms - Prepare Finished: 245.005ms (245.005ms) - Open Finished: 272.198ms (27.192ms) - First Batch Produced: 1s652ms (1s380ms) - First Batch Sent: 1s654ms (2.006ms) MemoryUsage(8m32s): 13.24 GB, 13.94 GB, 13.93 GB, 13.89 GB, 13.67 GB, 13.35 GB, 13.19 GB, 13.11 GB, 12.84 GB, 12.26 GB, 11.57 GB, 11.20 GB, 10.75 GB, 10.47 GB, 10.33 GB, 10.09 GB, 9.69 GB, 9.32 GB, 9.03 GB, 8.86 GB, 8.69 GB, 8.47 GB, 8.21 GB, 8.32 GB, 8.69 GB, 9.55 GB, 10.94 GB, 12.17 GB, 13.39 GB, 13.95 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.96 GB (14989497843) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.38 GB (188314335618) - RowsProduced: 595.56M (595558440) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h13m - TotalStorageWaitTime: 26m49s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 27.190ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 24.996ms - TotalThreadsVoluntaryContextSwitches: 6 (6) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 12s225ms - OpenTime: 27.191ms - ExecTreeOpenTime: 146.014us - PrepareTime: 244.428ms - ExecTreePrepareTime: 1.844ms KrpcDataStreamSender (dst_id=1):(Total: 11m7s, non-child: 11m7s, % non-child: 100.00%) - BytesSent: 78.21 GB (83982568578) - EosSent: 0 (0) - OverallThroughput: 119.92 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 595.56M (595557416) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m6s - UncompressedRowBatchSize: 153.52 GB (164840063523) HDFS_SCAN_NODE (id=0):(Total: 11s715ms, non-child: 11s715ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:26/6.36 GB 7:32/7.88 GB 3:20/4.82 GB 2:23/5.55 GB 5:26/6.35 GB 6:27/6.61 GB 0:33/8.13 GB 1:37/9.04 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.16 GB, 16.54 GB, 17.21 GB, 17.81 GB, 18.91 GB, 19.69 GB, 20.04 GB, 20.33 GB, 20.50 GB, 20.70 GB, 20.94 GB, 21.20 GB, 21.29 GB, 21.42 GB, 21.53 GB, 21.58 GB, 21.61 GB, 21.93 GB, 22.31 GB, 22.63 GB, 22.87 GB, 23.06 GB, 23.27 GB, 23.95 GB, 24.99 GB, 26.58 GB, 29.01 GB, 31.17 GB, 33.32 GB, 34.37 GB, 34.46 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB - FooterProcessingTime: (Avg: 2s226ms ; Min: 58.041ms ; Max: 8s564ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.47 GB (37007317886) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 13m24s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.96 GB (14986153016) - PerReadThreadRawHdfsThroughput: 54.35 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 595.77M (595771432) - RowsReturned: 595.56M (595558440) - RowsReturnedRate: 50.83 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 10m54s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 26s486ms - TotalRawHdfsReadTime(*): 10m49s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 26.763ms, non-child: 26.763ms, % non-child: 100.00%) - CodegenTime: 407.955us - CompileTime: 3.937ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.389ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 15.913ms Instance 844a0200d7876345:20bb38b900000020 (host=vc1505.foo.com:22000):(Total: 271.529ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:27/6.57 GB 7:28/6.85 GB 1:31/7.63 GB 0:21/5.15 GB 3:32/7.77 GB 4:28/6.84 GB 6:31/7.56 GB 5:26/6.44 GB Fragment Instance Lifecycle Event Timeline: 1s803ms - Prepare Finished: 244.395ms (244.395ms) - Open Finished: 272.048ms (27.652ms) - First Batch Produced: 1s801ms (1s528ms) - First Batch Sent: 1s803ms (2.240ms) MemoryUsage(8m32s): 13.15 GB, 13.95 GB, 13.94 GB, 13.87 GB, 13.65 GB, 13.28 GB, 13.15 GB, 12.94 GB, 12.61 GB, 11.91 GB, 11.29 GB, 10.86 GB, 10.41 GB, 10.20 GB, 9.99 GB, 9.71 GB, 9.26 GB, 8.96 GB, 8.76 GB, 8.79 GB, 8.66 GB, 8.57 GB, 8.81 GB, 9.11 GB, 9.91 GB, 10.91 GB, 12.44 GB, 13.80 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB, 13.99 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15020040093) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.41 GB (188345462065) - RowsProduced: 608.43M (608430534) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 30m32s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 27.642ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 26.996ms - TotalThreadsVoluntaryContextSwitches: 5 (5) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s175ms - OpenTime: 27.643ms - ExecTreeOpenTime: 137.068us - PrepareTime: 243.809ms - ExecTreePrepareTime: 2.042ms KrpcDataStreamSender (dst_id=1):(Total: 11m14s, non-child: 11m14s, % non-child: 100.00%) - BytesSent: 79.83 GB (85719370133) - EosSent: 0 (0) - OverallThroughput: 121.23 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 608.43M (608429510) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m12s - UncompressedRowBatchSize: 156.84 GB (168402968627) HDFS_SCAN_NODE (id=0):(Total: 8s698ms, non-child: 8s698ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:27/6.57 GB 7:28/6.85 GB 1:31/7.63 GB 0:21/5.15 GB 3:32/7.77 GB 4:28/6.84 GB 6:31/7.56 GB 5:26/6.44 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.31 GB, 16.77 GB, 17.42 GB, 18.11 GB, 19.23 GB, 19.84 GB, 20.13 GB, 20.45 GB, 20.59 GB, 20.80 GB, 21.05 GB, 21.11 GB, 21.20 GB, 21.34 GB, 21.49 GB, 21.56 GB, 21.57 GB, 22.13 GB, 22.59 GB, 23.02 GB, 23.31 GB, 23.77 GB, 24.76 GB, 25.70 GB, 27.19 GB, 28.96 GB, 31.61 GB, 33.97 GB, 34.40 GB, 34.58 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB, 34.68 GB - FooterProcessingTime: (Avg: 2s501ms ; Min: 47.486ms ; Max: 12s030ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.68 GB (37235965145) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m46s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.99 GB (15016883232) - PerReadThreadRawHdfsThroughput: 52.25 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 608.64M (608643526) - RowsReturned: 608.43M (608430534) - RowsReturnedRate: 69.94 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m18s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 21s425ms - TotalRawHdfsReadTime(*): 11m19s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 27.178ms, non-child: 27.178ms, % non-child: 100.00%) - CodegenTime: 452.464us - CompileTime: 4.055ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.572ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 15.095ms Instance 844a0200d7876345:20bb38b900000052 (host=vd1328.foo.com:22000):(Total: 271.307ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 3:24/5.87 GB 4:33/7.98 GB 1:23/5.66 GB 6:40/9.84 GB 5:28/6.84 GB 2:33/8.09 GB 0:28/6.80 GB 7:15/3.68 GB Fragment Instance Lifecycle Event Timeline: 1s643ms - Prepare Finished: 237.591ms (237.591ms) - Open Finished: 271.768ms (34.177ms) - First Batch Produced: 1s641ms (1s369ms) - First Batch Sent: 1s643ms (2.147ms) MemoryUsage(8m32s): 11.97 GB, 12.61 GB, 12.58 GB, 12.45 GB, 12.17 GB, 12.02 GB, 11.77 GB, 11.17 GB, 10.47 GB, 9.92 GB, 9.54 GB, 9.33 GB, 9.00 GB, 8.49 GB, 8.05 GB, 7.84 GB, 7.71 GB, 7.61 GB, 7.88 GB, 8.55 GB, 9.84 GB, 11.32 GB, 12.24 GB, 12.68 GB, 12.68 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.68 GB (13616424652) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.42 GB (171175729835) - RowsProduced: 606.43M (606433472) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 26m20s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 34.174ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 11s031ms - OpenTime: 34.176ms - ExecTreeOpenTime: 181.646us - PrepareTime: 237.048ms - ExecTreePrepareTime: 1.786ms KrpcDataStreamSender (dst_id=1):(Total: 11m11s, non-child: 11m11s, % non-child: 100.00%) - BytesSent: 79.56 GB (85422870858) - EosSent: 0 (0) - OverallThroughput: 121.26 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 606.43M (606432448) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m10s - UncompressedRowBatchSize: 156.32 GB (167849513182) HDFS_SCAN_NODE (id=0):(Total: 10s533ms, non-child: 10s533ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:24/5.87 GB 4:33/7.98 GB 1:23/5.66 GB 6:40/9.84 GB 5:28/6.84 GB 2:33/8.09 GB 0:28/6.80 GB 7:15/3.68 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.53 GB, 15.45 GB, 16.14 GB, 17.34 GB, 18.11 GB, 18.49 GB, 18.73 GB, 18.95 GB, 19.21 GB, 19.34 GB, 19.48 GB, 19.59 GB, 19.61 GB, 19.82 GB, 20.10 GB, 20.49 GB, 20.89 GB, 21.44 GB, 22.40 GB, 23.85 GB, 26.14 GB, 28.71 GB, 30.40 GB, 31.27 GB, 31.41 GB, 31.59 GB, 31.83 GB, 32.03 GB, 32.31 GB, 32.64 GB, 32.92 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB, 32.93 GB - FooterProcessingTime: (Avg: 2s342ms ; Min: 32.741ms ; Max: 12s531ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.93 GB (35362379673) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m40s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.68 GB (13613275984) - PerReadThreadRawHdfsThroughput: 52.69 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 606.64M (606638272) - RowsReturned: 606.43M (606433472) - RowsReturnedRate: 57.57 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m23s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 17s233ms - TotalRawHdfsReadTime(*): 10m39s - TotalReadThroughput: 1.17 MB/sec CodeGen:(Total: 33.691ms, non-child: 33.691ms, % non-child: 100.00%) - CodegenTime: 450.382us - CompileTime: 4.623ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.602ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 20.853ms Instance 844a0200d7876345:20bb38b90000001d (host=vc1524.foo.com:22000):(Total: 270.482ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 6:24/5.86 GB 7:16/3.76 GB 4:33/7.85 GB 1:30/7.21 GB 0:25/6.08 GB 5:36/8.51 GB 2:27/6.61 GB 3:37/8.98 GB Fragment Instance Lifecycle Event Timeline: 2s202ms - Prepare Finished: 244.202ms (244.202ms) - Open Finished: 270.975ms (26.773ms) - First Batch Produced: 2s200ms (1s929ms) - First Batch Sent: 2s202ms (2.010ms) MemoryUsage(8m32s): 13.05 GB, 13.87 GB, 13.83 GB, 13.73 GB, 13.51 GB, 13.26 GB, 13.10 GB, 12.91 GB, 12.64 GB, 12.18 GB, 11.48 GB, 11.13 GB, 10.71 GB, 10.57 GB, 10.45 GB, 10.19 GB, 10.01 GB, 9.50 GB, 8.90 GB, 8.78 GB, 8.59 GB, 8.33 GB, 8.39 GB, 8.64 GB, 9.19 GB, 10.04 GB, 10.93 GB, 12.67 GB, 13.88 GB, 13.90 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.92 GB (14951175486) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.31 GB (188237586452) - RowsProduced: 602.36M (602359905) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 31m44s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 26.780ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 24.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 13s872ms - OpenTime: 26.781ms - ExecTreeOpenTime: 140.955us - PrepareTime: 243.629ms - ExecTreePrepareTime: 1.999ms KrpcDataStreamSender (dst_id=1):(Total: 11m10s, non-child: 11m10s, % non-child: 100.00%) - BytesSent: 79.00 GB (84829119578) - EosSent: 0 (0) - OverallThroughput: 120.65 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 602.36M (602358881) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m9s - UncompressedRowBatchSize: 155.27 GB (166722204462) HDFS_SCAN_NODE (id=0):(Total: 13s387ms, non-child: 13s387ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:24/5.86 GB 7:16/3.76 GB 4:33/7.85 GB 1:30/7.21 GB 0:25/6.08 GB 5:36/8.51 GB 2:27/6.61 GB 3:37/8.98 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.17 GB, 16.63 GB, 17.25 GB, 17.95 GB, 19.00 GB, 19.71 GB, 19.99 GB, 20.30 GB, 20.58 GB, 20.91 GB, 21.18 GB, 21.28 GB, 21.45 GB, 21.66 GB, 21.90 GB, 22.03 GB, 22.36 GB, 22.42 GB, 22.46 GB, 22.74 GB, 22.87 GB, 23.07 GB, 23.71 GB, 24.81 GB, 26.06 GB, 27.53 GB, 29.13 GB, 32.25 GB, 34.35 GB, 34.45 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB - FooterProcessingTime: (Avg: 2s458ms ; Min: 26.001ms ; Max: 12s165ms ; Number of samples: 178) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.51 GB (37054432577) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m5s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 178 (178) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.92 GB (14948018655) - PerReadThreadRawHdfsThroughput: 45.07 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 602.57M (602572897) - RowsReturned: 602.36M (602359905) - RowsReturnedRate: 44.99 M/sec - ScanRangesComplete: 90 (90) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m19s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 21s998ms - TotalRawHdfsReadTime(*): 13m4s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 26.320ms, non-child: 26.320ms, % non-child: 100.00%) - CodegenTime: 427.261us - CompileTime: 4.130ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.502ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 15.152ms Instance 844a0200d7876345:20bb38b900000023 (host=vd1326.foo.com:22000):(Total: 270.035ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:28/6.84 GB 7:22/5.32 GB 2:26/6.35 GB 6:27/6.64 GB 3:37/9.12 GB 0:26/6.44 GB 5:21/5.15 GB 4:37/9.08 GB Fragment Instance Lifecycle Event Timeline: 1s730ms - Prepare Finished: 244.485ms (244.485ms) - Open Finished: 270.478ms (25.993ms) - First Batch Produced: 1s728ms (1s457ms) - First Batch Sent: 1s730ms (1.758ms) MemoryUsage(8m32s): 12.12 GB, 12.73 GB, 12.71 GB, 12.58 GB, 12.24 GB, 12.12 GB, 11.91 GB, 11.40 GB, 10.66 GB, 10.09 GB, 9.65 GB, 9.50 GB, 9.25 GB, 8.78 GB, 8.08 GB, 7.87 GB, 7.69 GB, 7.30 GB, 7.26 GB, 7.73 GB, 8.47 GB, 10.28 GB, 12.01 GB, 12.63 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.74 GB (13675818931) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.11 GB (186952073115) - RowsProduced: 599.42M (599418082) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h10m - TotalStorageWaitTime: 23m15s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 25.990ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 23.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 14s680ms - OpenTime: 25.992ms - ExecTreeOpenTime: 136.114us - PrepareTime: 243.967ms - ExecTreePrepareTime: 1.797ms KrpcDataStreamSender (dst_id=1):(Total: 11m19s, non-child: 11m19s, % non-child: 100.00%) - BytesSent: 78.67 GB (84469751364) - EosSent: 0 (0) - OverallThroughput: 118.57 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 599.42M (599417058) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m17s - UncompressedRowBatchSize: 154.51 GB (165907748009) HDFS_SCAN_NODE (id=0):(Total: 14s142ms, non-child: 14s142ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:28/6.84 GB 7:22/5.32 GB 2:26/6.35 GB 6:27/6.64 GB 3:37/9.12 GB 0:26/6.44 GB 5:21/5.15 GB 4:37/9.08 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.65 GB, 15.54 GB, 16.18 GB, 17.33 GB, 18.14 GB, 18.52 GB, 18.77 GB, 18.94 GB, 19.25 GB, 19.36 GB, 19.47 GB, 19.64 GB, 19.68 GB, 19.68 GB, 19.89 GB, 20.26 GB, 20.54 GB, 20.66 GB, 21.42 GB, 22.55 GB, 23.95 GB, 27.13 GB, 30.05 GB, 31.18 GB, 31.39 GB, 31.53 GB, 31.72 GB, 31.93 GB, 32.13 GB, 32.40 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB - FooterProcessingTime: (Avg: 2s347ms ; Min: 37.968ms ; Max: 12s521ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.55 GB (34952565877) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 13m20s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.73 GB (13672694852) - PerReadThreadRawHdfsThroughput: 48.68 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 599.62M (599622882) - RowsReturned: 599.42M (599418082) - RowsReturnedRate: 42.38 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m47s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 17s619ms - TotalRawHdfsReadTime(*): 11m24s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 25.578ms, non-child: 25.578ms, % non-child: 100.00%) - CodegenTime: 393.229us - CompileTime: 4.028ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.547ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.386ms Instance 844a0200d7876345:20bb38b90000006f (host=vc1520.foo.com:22000):(Total: 269.525ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 6:29/7.14 GB 4:24/5.77 GB 5:27/6.57 GB 1:29/7.15 GB 7:27/6.55 GB 2:37/8.93 GB 3:22/5.20 GB 0:31/7.59 GB Fragment Instance Lifecycle Event Timeline: 1s619ms - Prepare Finished: 242.861ms (242.861ms) - Open Finished: 269.981ms (27.120ms) - First Batch Produced: 1s618ms (1s348ms) - First Batch Sent: 1s619ms (1.429ms) MemoryUsage(8m32s): 13.10 GB, 13.89 GB, 13.88 GB, 13.94 GB, 13.83 GB, 13.55 GB, 13.33 GB, 13.29 GB, 13.11 GB, 12.69 GB, 11.91 GB, 11.44 GB, 10.95 GB, 10.55 GB, 10.46 GB, 10.28 GB, 9.93 GB, 9.50 GB, 8.98 GB, 8.87 GB, 8.79 GB, 8.55 GB, 8.24 GB, 8.06 GB, 8.25 GB, 9.13 GB, 9.91 GB, 11.36 GB, 13.17 GB, 13.84 GB, 13.84 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.85 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (15003938289) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.59 GB (172429530616) - RowsProduced: 592.47M (592465041) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 29m48s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 27.122ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 25.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s631ms - OpenTime: 27.124ms - ExecTreeOpenTime: 289.235us - PrepareTime: 242.327ms - ExecTreePrepareTime: 2.055ms KrpcDataStreamSender (dst_id=1):(Total: 11m14s, non-child: 11m14s, % non-child: 100.00%) - BytesSent: 77.80 GB (83538441910) - EosSent: 0 (0) - OverallThroughput: 118.04 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 592.46M (592464017) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m13s - UncompressedRowBatchSize: 152.72 GB (163983933945) HDFS_SCAN_NODE (id=0):(Total: 8s109ms, non-child: 8s109ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:29/7.14 GB 4:24/5.77 GB 5:27/6.57 GB 1:29/7.15 GB 7:27/6.55 GB 2:37/8.93 GB 3:22/5.20 GB 0:31/7.59 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.75 GB, 16.22 GB, 17.02 GB, 17.72 GB, 18.69 GB, 19.68 GB, 20.04 GB, 20.35 GB, 20.59 GB, 20.76 GB, 21.02 GB, 21.21 GB, 21.29 GB, 21.38 GB, 21.55 GB, 21.63 GB, 21.63 GB, 21.68 GB, 21.94 GB, 22.49 GB, 22.71 GB, 22.89 GB, 22.93 GB, 23.31 GB, 24.24 GB, 25.93 GB, 27.32 GB, 29.86 GB, 33.04 GB, 34.23 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB - FooterProcessingTime: (Avg: 2s298ms ; Min: 32.459ms ; Max: 9s097ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.32 GB (36847406957) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m25s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (15000920681) - PerReadThreadRawHdfsThroughput: 52.18 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 592.68M (592678033) - RowsReturned: 592.47M (592465041) - RowsReturnedRate: 73.06 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m25s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 23s356ms - TotalRawHdfsReadTime(*): 11m13s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 26.549ms, non-child: 26.549ms, % non-child: 100.00%) - CodegenTime: 405.920us - CompileTime: 4.449ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.528ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.962ms Instance 844a0200d7876345:20bb38b900000044 (host=vd1133.foo.com:22000):(Total: 269.480ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:21/5.16 GB 4:26/6.33 GB 3:24/5.87 GB 0:43/10.30 GB 5:33/8.04 GB 2:23/5.69 GB 7:26/6.44 GB 6:26/6.44 GB Fragment Instance Lifecycle Event Timeline: 1s689ms - Prepare Finished: 233.195ms (233.195ms) - Open Finished: 270.280ms (37.084ms) - First Batch Produced: 1s686ms (1s416ms) - First Batch Sent: 1s689ms (2.643ms) MemoryUsage(8m32s): 12.15 GB, 12.72 GB, 12.68 GB, 12.52 GB, 12.23 GB, 12.12 GB, 11.90 GB, 11.31 GB, 10.55 GB, 9.99 GB, 9.61 GB, 9.47 GB, 9.26 GB, 8.78 GB, 8.04 GB, 7.74 GB, 7.57 GB, 7.27 GB, 7.22 GB, 7.99 GB, 9.29 GB, 11.26 GB, 12.60 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.72 GB (13654669247) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.45 GB (171203678675) - RowsProduced: 605.90M (605902437) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 20m30s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 37.078ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 34.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s643ms - OpenTime: 37.079ms - ExecTreeOpenTime: 186.781us - PrepareTime: 232.301ms - ExecTreePrepareTime: 1.966ms KrpcDataStreamSender (dst_id=1):(Total: 11m35s, non-child: 11m35s, % non-child: 100.00%) - BytesSent: 79.51 GB (85371885974) - EosSent: 0 (0) - OverallThroughput: 117.00 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 605.90M (605901413) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m34s - UncompressedRowBatchSize: 156.19 GB (167702846480) HDFS_SCAN_NODE (id=0):(Total: 9s103ms, non-child: 9s103ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:21/5.16 GB 4:26/6.33 GB 3:24/5.87 GB 0:43/10.30 GB 5:33/8.04 GB 2:23/5.69 GB 7:26/6.44 GB 6:26/6.44 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.72 GB, 15.51 GB, 16.12 GB, 17.34 GB, 18.11 GB, 18.50 GB, 18.72 GB, 18.92 GB, 19.22 GB, 19.32 GB, 19.46 GB, 19.62 GB, 19.73 GB, 19.92 GB, 20.04 GB, 20.14 GB, 20.50 GB, 20.83 GB, 21.48 GB, 23.02 GB, 25.31 GB, 28.83 GB, 31.09 GB, 31.31 GB, 31.41 GB, 31.58 GB, 31.85 GB, 32.04 GB, 32.21 GB, 32.53 GB, 32.80 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB, 32.81 GB - FooterProcessingTime: (Avg: 1s677ms ; Min: 24.831ms ; Max: 12s542ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.81 GB (35225165874) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m56s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.71 GB (13651537150) - PerReadThreadRawHdfsThroughput: 51.59 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 606.11M (606107237) - RowsReturned: 605.90M (605902437) - RowsReturnedRate: 66.56 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m59s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 16s325ms - TotalRawHdfsReadTime(*): 10m51s - TotalReadThroughput: 1.16 MB/sec CodeGen:(Total: 36.504ms, non-child: 36.504ms, % non-child: 100.00%) - CodegenTime: 505.556us - CompileTime: 4.320ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.763ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 23.871ms Instance 844a0200d7876345:20bb38b90000000d (host=vd1107.foo.com:22000):(Total: 268.022ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 6:27/6.67 GB 2:32/7.92 GB 0:28/6.93 GB 4:24/5.90 GB 7:29/7.14 GB 5:31/7.63 GB 1:29/7.01 GB 3:23/5.39 GB Fragment Instance Lifecycle Event Timeline: 1s659ms - Prepare Finished: 234.036ms (234.036ms) - Open Finished: 268.562ms (34.526ms) - First Batch Produced: 1s657ms (1s388ms) - First Batch Sent: 1s659ms (2.208ms) MemoryUsage(8m32s): 13.24 GB, 13.89 GB, 13.88 GB, 13.79 GB, 13.62 GB, 13.39 GB, 13.31 GB, 13.17 GB, 12.83 GB, 12.16 GB, 11.52 GB, 11.10 GB, 10.66 GB, 10.48 GB, 10.31 GB, 10.20 GB, 9.94 GB, 9.29 GB, 8.79 GB, 8.67 GB, 8.43 GB, 8.18 GB, 8.04 GB, 8.38 GB, 9.03 GB, 10.04 GB, 11.65 GB, 13.29 GB, 13.99 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 14.00 GB (15029850361) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.40 GB (188339399324) - RowsProduced: 602.50M (602502793) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 25m39s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 34.520ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s102ms - OpenTime: 34.521ms - ExecTreeOpenTime: 140.389us - PrepareTime: 233.403ms - ExecTreePrepareTime: 1.981ms KrpcDataStreamSender (dst_id=1):(Total: 11m12s, non-child: 11m12s, % non-child: 100.00%) - BytesSent: 79.10 GB (84932050086) - EosSent: 0 (0) - OverallThroughput: 120.53 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 602.50M (602501769) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m10s - UncompressedRowBatchSize: 155.31 GB (166762177806) HDFS_SCAN_NODE (id=0):(Total: 9s584ms, non-child: 9s584ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:27/6.67 GB 2:32/7.92 GB 0:28/6.93 GB 4:24/5.90 GB 7:29/7.14 GB 5:31/7.63 GB 1:29/7.01 GB 3:23/5.39 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.32 GB, 16.68 GB, 17.25 GB, 17.95 GB, 19.24 GB, 19.92 GB, 20.22 GB, 20.51 GB, 20.64 GB, 20.86 GB, 21.14 GB, 21.24 GB, 21.32 GB, 21.48 GB, 21.59 GB, 21.89 GB, 22.08 GB, 22.17 GB, 22.27 GB, 22.48 GB, 22.65 GB, 22.86 GB, 23.26 GB, 24.37 GB, 25.66 GB, 27.46 GB, 30.31 GB, 33.17 GB, 34.37 GB, 34.47 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB - FooterProcessingTime: (Avg: 1s819ms ; Min: 15.455ms ; Max: 7s510ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.54 GB (37087068824) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m22s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.99 GB (15026709870) - PerReadThreadRawHdfsThroughput: 53.77 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 602.72M (602715785) - RowsReturned: 602.50M (602502793) - RowsReturnedRate: 62.87 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m1s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 25s386ms - TotalRawHdfsReadTime(*): 10m57s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 34.089ms, non-child: 34.089ms, % non-child: 100.00%) - CodegenTime: 445.658us - CompileTime: 4.161ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.907ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.524ms Instance 844a0200d7876345:20bb38b900000027 (host=vc1501.foo.com:22000):(Total: 267.641ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:29/7.14 GB 1:24/5.90 GB 2:26/6.36 GB 4:30/7.35 GB 6:25/6.19 GB 5:27/6.58 GB 0:36/8.80 GB 3:27/6.53 GB Fragment Instance Lifecycle Event Timeline: 1s819ms - Prepare Finished: 241.784ms (241.784ms) - Open Finished: 268.236ms (26.452ms) - First Batch Produced: 1s817ms (1s548ms) - First Batch Sent: 1s819ms (2.661ms) MemoryUsage(8m32s): 13.36 GB, 14.01 GB, 14.00 GB, 13.96 GB, 13.73 GB, 13.45 GB, 13.38 GB, 13.23 GB, 12.82 GB, 11.99 GB, 11.52 GB, 10.99 GB, 10.65 GB, 10.51 GB, 10.36 GB, 10.02 GB, 9.37 GB, 8.78 GB, 8.50 GB, 8.38 GB, 8.04 GB, 7.89 GB, 7.90 GB, 8.06 GB, 9.05 GB, 10.48 GB, 12.58 GB, 13.87 GB, 13.93 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 14.01 GB (15040770748) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.35 GB (188284393114) - RowsProduced: 606.60M (606595452) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 23m50s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 26.437ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 22.997ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 11s110ms - OpenTime: 26.449ms - ExecTreeOpenTime: 139.777us - PrepareTime: 241.111ms - ExecTreePrepareTime: 1.807ms KrpcDataStreamSender (dst_id=1):(Total: 11m11s, non-child: 11m11s, % non-child: 100.00%) - BytesSent: 79.62 GB (85486976604) - EosSent: 0 (0) - OverallThroughput: 121.38 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 606.59M (606594428) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m10s - UncompressedRowBatchSize: 156.36 GB (167895081742) HDFS_SCAN_NODE (id=0):(Total: 10s610ms, non-child: 10s610ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:29/7.14 GB 1:24/5.90 GB 2:26/6.36 GB 4:30/7.35 GB 6:25/6.19 GB 5:27/6.58 GB 0:36/8.80 GB 3:27/6.53 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.60 GB, 16.87 GB, 17.50 GB, 18.23 GB, 19.57 GB, 20.04 GB, 20.41 GB, 20.63 GB, 20.76 GB, 21.09 GB, 21.27 GB, 21.32 GB, 21.45 GB, 21.64 GB, 21.67 GB, 21.67 GB, 21.67 GB, 21.95 GB, 21.99 GB, 22.19 GB, 22.37 GB, 22.83 GB, 23.54 GB, 24.13 GB, 25.90 GB, 28.37 GB, 32.07 GB, 34.26 GB, 34.42 GB, 34.50 GB, 34.57 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB - FooterProcessingTime: (Avg: 2s053ms ; Min: 48.757ms ; Max: 9s103ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.58 GB (37125591479) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m56s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 14.00 GB (15037646642) - PerReadThreadRawHdfsThroughput: 56.75 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 606.81M (606808444) - RowsReturned: 606.60M (606595452) - RowsReturnedRate: 57.17 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m50s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 29s668ms - TotalRawHdfsReadTime(*): 10m23s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 26.021ms, non-child: 26.021ms, % non-child: 100.00%) - CodegenTime: 362.671us - CompileTime: 3.920ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.631ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.967ms Instance 844a0200d7876345:20bb38b900000057 (host=vc1540.foo.com:22000):(Total: 266.729ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:30/7.43 GB 3:24/5.94 GB 1:26/6.30 GB 0:24/5.85 GB 6:33/8.13 GB 7:31/7.64 GB 4:25/5.97 GB 5:31/7.59 GB Fragment Instance Lifecycle Event Timeline: 1s583ms - Prepare Finished: 242.630ms (242.630ms) - Open Finished: 267.338ms (24.707ms) - First Batch Produced: 1s581ms (1s313ms) - First Batch Sent: 1s583ms (2.257ms) MemoryUsage(8m32s): 13.25 GB, 13.90 GB, 13.89 GB, 13.89 GB, 13.83 GB, 13.51 GB, 13.40 GB, 13.33 GB, 13.08 GB, 12.46 GB, 11.72 GB, 11.28 GB, 10.77 GB, 10.61 GB, 10.50 GB, 10.21 GB, 9.86 GB, 9.14 GB, 8.83 GB, 8.70 GB, 8.33 GB, 7.94 GB, 7.83 GB, 7.94 GB, 8.79 GB, 9.88 GB, 11.87 GB, 13.72 GB, 13.95 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.95 GB (14981220273) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.69 GB (172535195133) - RowsProduced: 603.36M (603357021) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h18m - TotalStorageWaitTime: 25m28s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 24.683ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 23.997ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7s739ms - OpenTime: 24.685ms - ExecTreeOpenTime: 162.628us - PrepareTime: 241.964ms - ExecTreePrepareTime: 1.803ms KrpcDataStreamSender (dst_id=1):(Total: 11m7s, non-child: 11m7s, % non-child: 100.00%) - BytesSent: 79.11 GB (84946553658) - EosSent: 0 (0) - OverallThroughput: 121.43 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 603.36M (603355997) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m5s - UncompressedRowBatchSize: 155.53 GB (166999058470) HDFS_SCAN_NODE (id=0):(Total: 7s308ms, non-child: 7s308ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:30/7.43 GB 3:24/5.94 GB 1:26/6.30 GB 0:24/5.85 GB 6:33/8.13 GB 7:31/7.64 GB 4:25/5.97 GB 5:31/7.59 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.91 GB, 16.42 GB, 17.11 GB, 17.73 GB, 19.07 GB, 19.85 GB, 20.17 GB, 20.53 GB, 20.65 GB, 20.89 GB, 21.20 GB, 21.28 GB, 21.37 GB, 21.55 GB, 21.67 GB, 21.67 GB, 21.83 GB, 21.91 GB, 22.25 GB, 22.40 GB, 22.42 GB, 22.48 GB, 22.95 GB, 23.72 GB, 25.30 GB, 27.24 GB, 30.73 GB, 33.91 GB, 34.34 GB, 34.46 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB - FooterProcessingTime: (Avg: 1s826ms ; Min: 17.073ms ; Max: 9s615ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.56 GB (37108549311) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 6m8s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.95 GB (14978137148) - PerReadThreadRawHdfsThroughput: 55.47 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 603.57M (603570013) - RowsReturned: 603.36M (603357021) - RowsReturnedRate: 82.55 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m33s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 19s151ms - TotalRawHdfsReadTime(*): 10m38s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 24.274ms, non-child: 24.274ms, % non-child: 100.00%) - CodegenTime: 303.575us - CompileTime: 4.327ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 5.891ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 13.551ms Instance 844a0200d7876345:20bb38b900000009 (host=va1028.foo.com:22000):(Total: 265.860ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:17/4.12 GB 4:29/7.10 GB 0:28/6.59 GB 3:30/7.24 GB 1:21/5.13 GB 6:32/7.77 GB 7:35/8.59 GB 2:33/8.09 GB Fragment Instance Lifecycle Event Timeline: 1s893ms - Prepare Finished: 230.463ms (230.463ms) - Open Finished: 266.282ms (35.819ms) - First Batch Produced: 1s891ms (1s625ms) - First Batch Sent: 1s893ms (2.020ms) MemoryUsage(8m32s): 13.18 GB, 13.96 GB, 13.93 GB, 13.86 GB, 13.65 GB, 13.35 GB, 13.23 GB, 13.08 GB, 12.73 GB, 12.04 GB, 11.42 GB, 10.94 GB, 10.59 GB, 10.43 GB, 10.25 GB, 10.09 GB, 9.59 GB, 9.13 GB, 8.81 GB, 8.71 GB, 8.50 GB, 8.22 GB, 8.20 GB, 8.68 GB, 9.39 GB, 10.58 GB, 12.10 GB, 13.09 GB, 13.79 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.96 GB (14994637149) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.25 GB (188172965432) - RowsProduced: 605.46M (605463818) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 29m1s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 35.814ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 33.995ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s786ms - OpenTime: 35.816ms - ExecTreeOpenTime: 186.701us - PrepareTime: 229.968ms - ExecTreePrepareTime: 2.124ms KrpcDataStreamSender (dst_id=1):(Total: 11m11s, non-child: 11m11s, % non-child: 100.00%) - BytesSent: 79.47 GB (85334274766) - EosSent: 0 (0) - OverallThroughput: 121.16 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 605.46M (605462794) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m10s - UncompressedRowBatchSize: 156.07 GB (167582381187) HDFS_SCAN_NODE (id=0):(Total: 9s319ms, non-child: 9s319ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:17/4.12 GB 4:29/7.10 GB 0:28/6.59 GB 3:30/7.24 GB 1:21/5.13 GB 6:32/7.77 GB 7:35/8.59 GB 2:33/8.09 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.33 GB, 16.75 GB, 17.37 GB, 18.05 GB, 19.22 GB, 19.84 GB, 20.18 GB, 20.44 GB, 20.60 GB, 20.84 GB, 21.05 GB, 21.15 GB, 21.34 GB, 21.54 GB, 21.62 GB, 21.88 GB, 21.92 GB, 22.17 GB, 22.38 GB, 22.73 GB, 22.94 GB, 23.16 GB, 23.75 GB, 24.93 GB, 26.35 GB, 28.51 GB, 31.20 GB, 32.97 GB, 34.23 GB, 34.40 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB - FooterProcessingTime: (Avg: 2s380ms ; Min: 59.376ms ; Max: 10s191ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.49 GB (37033578671) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m46s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.96 GB (14991545749) - PerReadThreadRawHdfsThroughput: 49.45 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 605.68M (605676810) - RowsReturned: 605.46M (605463818) - RowsReturnedRate: 64.97 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m26s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 37s689ms - TotalRawHdfsReadTime(*): 11m54s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 35.245ms, non-child: 35.245ms, % non-child: 100.00%) - CodegenTime: 492.118us - CompileTime: 4.332ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.301ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 21.964ms Instance 844a0200d7876345:20bb38b900000048 (host=vc1516.foo.com:22000):(Total: 265.435ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:25/6.00 GB 5:26/6.31 GB 0:40/9.79 GB 4:30/7.35 GB 6:29/7.10 GB 7:28/6.86 GB 1:21/5.08 GB 3:27/6.68 GB Fragment Instance Lifecycle Event Timeline: 1s795ms - Prepare Finished: 239.004ms (239.004ms) - Open Finished: 266.030ms (27.026ms) - First Batch Produced: 1s793ms (1s527ms) - First Batch Sent: 1s795ms (2.182ms) MemoryUsage(8m32s): 13.30 GB, 13.97 GB, 13.96 GB, 13.89 GB, 13.61 GB, 13.29 GB, 13.22 GB, 13.04 GB, 12.61 GB, 11.78 GB, 11.26 GB, 10.75 GB, 10.44 GB, 10.32 GB, 10.10 GB, 9.74 GB, 9.16 GB, 8.69 GB, 8.46 GB, 8.46 GB, 8.33 GB, 7.98 GB, 8.27 GB, 9.18 GB, 10.14 GB, 12.04 GB, 13.55 GB, 13.92 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (15001563772) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.69 GB (172540772510) - RowsProduced: 613.32M (613320076) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 24m45s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 27.001ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 24.997ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s451ms - OpenTime: 27.003ms - ExecTreeOpenTime: 153.062us - PrepareTime: 238.358ms - ExecTreePrepareTime: 1.940ms KrpcDataStreamSender (dst_id=1):(Total: 11m5s, non-child: 11m5s, % non-child: 100.00%) - BytesSent: 80.53 GB (86464286780) - EosSent: 0 (0) - OverallThroughput: 123.94 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 613.32M (613319052) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m4s - UncompressedRowBatchSize: 158.10 GB (169756893466) HDFS_SCAN_NODE (id=0):(Total: 7s959ms, non-child: 7s959ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:25/6.00 GB 5:26/6.31 GB 0:40/9.79 GB 4:30/7.35 GB 6:29/7.10 GB 7:28/6.86 GB 1:21/5.08 GB 3:27/6.68 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.48 GB, 16.84 GB, 17.45 GB, 18.20 GB, 19.44 GB, 19.93 GB, 20.28 GB, 20.49 GB, 20.65 GB, 20.95 GB, 21.10 GB, 21.18 GB, 21.31 GB, 21.46 GB, 21.55 GB, 21.63 GB, 21.81 GB, 22.03 GB, 22.14 GB, 22.68 GB, 23.13 GB, 23.23 GB, 24.24 GB, 25.92 GB, 27.66 GB, 30.98 GB, 33.59 GB, 34.27 GB, 34.40 GB, 34.54 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB, 34.65 GB - FooterProcessingTime: (Avg: 1s983ms ; Min: 23.338ms ; Max: 7s666ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.65 GB (37204210187) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m44s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (14998439779) - PerReadThreadRawHdfsThroughput: 53.06 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 613.53M (613533068) - RowsReturned: 613.32M (613320076) - RowsReturnedRate: 77.05 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m27s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 29s642ms - TotalRawHdfsReadTime(*): 11m8s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 26.534ms, non-child: 26.534ms, % non-child: 100.00%) - CodegenTime: 392.243us - CompileTime: 3.907ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.659ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 15.484ms Instance 844a0200d7876345:20bb38b900000028 (host=vd1129.foo.com:22000):(Total: 264.990ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:28/6.86 GB 7:18/4.41 GB 4:22/5.40 GB 1:27/6.68 GB 3:24/5.91 GB 6:32/7.80 GB 5:35/8.35 GB 0:37/9.13 GB Fragment Instance Lifecycle Event Timeline: 1s723ms - Prepare Finished: 227.536ms (227.536ms) - Open Finished: 265.409ms (37.872ms) - First Batch Produced: 1s721ms (1s456ms) - First Batch Sent: 1s723ms (2.076ms) MemoryUsage(8m32s): 12.07 GB, 12.75 GB, 12.73 GB, 12.63 GB, 12.36 GB, 12.19 GB, 12.00 GB, 11.61 GB, 10.90 GB, 10.34 GB, 9.87 GB, 9.61 GB, 9.39 GB, 9.02 GB, 8.35 GB, 7.80 GB, 7.59 GB, 7.26 GB, 7.26 GB, 7.45 GB, 8.18 GB, 9.39 GB, 11.03 GB, 12.28 GB, 12.66 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.76 GB (13697965197) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.09 GB (186929979149) - RowsProduced: 596.53M (596532144) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 25m12s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 37.863ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 34.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 12s719ms - OpenTime: 37.872ms - ExecTreeOpenTime: 170.973us - PrepareTime: 227.045ms - ExecTreePrepareTime: 2.092ms KrpcDataStreamSender (dst_id=1):(Total: 11m30s, non-child: 11m30s, % non-child: 100.00%) - BytesSent: 78.36 GB (84141164956) - EosSent: 0 (0) - OverallThroughput: 116.22 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 596.53M (596531120) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m29s - UncompressedRowBatchSize: 153.77 GB (165109221072) HDFS_SCAN_NODE (id=0):(Total: 12s193ms, non-child: 12s193ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:28/6.86 GB 7:18/4.41 GB 4:22/5.40 GB 1:27/6.68 GB 3:24/5.91 GB 6:32/7.80 GB 5:35/8.35 GB 0:37/9.13 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.47 GB, 15.48 GB, 16.15 GB, 17.13 GB, 18.12 GB, 18.51 GB, 18.81 GB, 18.97 GB, 19.24 GB, 19.43 GB, 19.52 GB, 19.67 GB, 19.77 GB, 19.78 GB, 19.78 GB, 19.96 GB, 20.21 GB, 20.31 GB, 21.11 GB, 22.00 GB, 23.55 GB, 25.65 GB, 28.52 GB, 30.68 GB, 31.37 GB, 31.51 GB, 31.64 GB, 31.83 GB, 32.10 GB, 32.35 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB - FooterProcessingTime: (Avg: 2s178ms ; Min: 22.235ms ; Max: 13s303ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.52 GB (34922086951) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m32s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.75 GB (13694998612) - PerReadThreadRawHdfsThroughput: 48.40 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 596.74M (596736944) - RowsReturned: 596.53M (596532144) - RowsReturnedRate: 48.92 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m4s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 23s021ms - TotalRawHdfsReadTime(*): 11m28s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 37.245ms, non-child: 37.245ms, % non-child: 100.00%) - CodegenTime: 557.650us - CompileTime: 4.279ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.665ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 24.475ms Instance 844a0200d7876345:20bb38b90000004e (host=va1035.foo.com:22000):(Total: 264.431ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 6:24/5.94 GB 2:29/6.88 GB 4:30/7.25 GB 0:25/6.11 GB 1:40/9.86 GB 3:24/5.75 GB 5:27/6.56 GB 7:26/6.37 GB Fragment Instance Lifecycle Event Timeline: 2s225ms - Prepare Finished: 231.689ms (231.689ms) - Open Finished: 265.141ms (33.451ms) - First Batch Produced: 2s223ms (1s957ms) - First Batch Sent: 2s225ms (2.000ms) MemoryUsage(8m32s): 13.26 GB, 13.97 GB, 13.97 GB, 13.89 GB, 13.62 GB, 13.31 GB, 13.24 GB, 13.07 GB, 12.68 GB, 11.84 GB, 11.33 GB, 10.82 GB, 10.47 GB, 10.32 GB, 10.09 GB, 9.89 GB, 9.35 GB, 8.91 GB, 8.70 GB, 8.51 GB, 8.25 GB, 8.14 GB, 8.17 GB, 8.83 GB, 9.51 GB, 10.99 GB, 12.94 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.98 GB (15006484025) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.64 GB (172484812903) - RowsProduced: 609.43M (609427016) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h17m - TotalStorageWaitTime: 25m22s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 33.444ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 30.996ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s663ms - OpenTime: 33.445ms - ExecTreeOpenTime: 133.729us - PrepareTime: 230.877ms - ExecTreePrepareTime: 2.243ms KrpcDataStreamSender (dst_id=1):(Total: 11m11s, non-child: 11m11s, % non-child: 100.00%) - BytesSent: 79.96 GB (85858136184) - EosSent: 0 (0) - OverallThroughput: 121.95 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 609.43M (609425992) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m10s - UncompressedRowBatchSize: 157.09 GB (168679456573) HDFS_SCAN_NODE (id=0):(Total: 9s143ms, non-child: 9s143ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:24/5.94 GB 2:29/6.88 GB 4:30/7.25 GB 0:25/6.11 GB 1:40/9.86 GB 3:24/5.75 GB 5:27/6.56 GB 7:26/6.37 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.42 GB, 16.81 GB, 17.42 GB, 18.15 GB, 19.43 GB, 19.92 GB, 20.25 GB, 20.51 GB, 20.64 GB, 20.92 GB, 21.12 GB, 21.19 GB, 21.29 GB, 21.47 GB, 21.51 GB, 21.71 GB, 21.85 GB, 22.29 GB, 22.42 GB, 22.57 GB, 22.79 GB, 23.28 GB, 23.98 GB, 25.37 GB, 26.58 GB, 29.22 GB, 32.67 GB, 34.27 GB, 34.39 GB, 34.50 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB - FooterProcessingTime: (Avg: 2s259ms ; Min: 15.430ms ; Max: 9s991ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.55 GB (37102724069) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 6m48s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (15003335309) - PerReadThreadRawHdfsThroughput: 50.62 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 609.64M (609640008) - RowsReturned: 609.43M (609427016) - RowsReturnedRate: 66.65 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m3s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 25s630ms - TotalRawHdfsReadTime(*): 11m39s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 33.017ms, non-child: 33.017ms, % non-child: 100.00%) - CodegenTime: 398.894us - CompileTime: 3.959ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.406ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.087ms Instance 844a0200d7876345:20bb38b900000026 (host=vd1334.foo.com:22000):(Total: 263.658ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 3:23/5.69 GB 5:31/7.67 GB 4:19/4.58 GB 0:31/7.56 GB 2:33/8.13 GB 6:28/6.78 GB 1:29/7.06 GB 7:30/7.31 GB Fragment Instance Lifecycle Event Timeline: 1s927ms - Prepare Finished: 226.841ms (226.841ms) - Open Finished: 264.222ms (37.380ms) - First Batch Produced: 1s925ms (1s661ms) - First Batch Sent: 1s927ms (1.923ms) MemoryUsage(8m32s): 11.82 GB, 12.70 GB, 12.68 GB, 12.61 GB, 12.36 GB, 12.04 GB, 11.91 GB, 11.67 GB, 11.19 GB, 10.41 GB, 9.93 GB, 9.47 GB, 9.28 GB, 8.99 GB, 8.67 GB, 8.19 GB, 7.97 GB, 8.05 GB, 7.95 GB, 7.78 GB, 8.09 GB, 8.55 GB, 9.35 GB, 11.12 GB, 12.54 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.71 GB (13646234426) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.11 GB (186950953992) - RowsProduced: 587.27M (587266359) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 34m2s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 37.374ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 33.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s827ms - OpenTime: 37.376ms - ExecTreeOpenTime: 236.214us - PrepareTime: 226.211ms - ExecTreePrepareTime: 1.969ms KrpcDataStreamSender (dst_id=1):(Total: 11m33s, non-child: 11m33s, % non-child: 100.00%) - BytesSent: 77.07 GB (82753781178) - EosSent: 0 (0) - OverallThroughput: 113.85 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 587.27M (587265335) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m31s - UncompressedRowBatchSize: 151.38 GB (162545431258) HDFS_SCAN_NODE (id=0):(Total: 9s274ms, non-child: 9s274ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:23/5.69 GB 5:31/7.67 GB 4:19/4.58 GB 0:31/7.56 GB 2:33/8.13 GB 6:28/6.78 GB 1:29/7.06 GB 7:30/7.31 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 11.81 GB, 15.28 GB, 15.94 GB, 16.61 GB, 17.76 GB, 18.21 GB, 18.56 GB, 18.75 GB, 18.90 GB, 19.20 GB, 19.28 GB, 19.36 GB, 19.53 GB, 19.56 GB, 19.57 GB, 20.04 GB, 20.55 GB, 21.04 GB, 21.45 GB, 21.81 GB, 22.98 GB, 23.93 GB, 25.35 GB, 28.46 GB, 30.95 GB, 31.37 GB, 31.52 GB, 31.70 GB, 31.95 GB, 32.20 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.36 GB, 32.37 GB, 32.37 GB, 32.37 GB, 32.37 GB, 32.37 GB, 32.37 GB, 32.37 GB, 32.37 GB - FooterProcessingTime: (Avg: 3s204ms ; Min: 18.023ms ; Max: 20s653ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.37 GB (34752062112) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m52s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.71 GB (13643225157) - PerReadThreadRawHdfsThroughput: 44.30 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 587.47M (587471159) - RowsReturned: 587.27M (587266359) - RowsReturnedRate: 63.32 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m29s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 29s510ms - TotalRawHdfsReadTime(*): 12m28s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 36.840ms, non-child: 36.840ms, % non-child: 100.00%) - CodegenTime: 460.386us - CompileTime: 4.422ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.902ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.893ms Instance 844a0200d7876345:20bb38b900000070 (host=vd1104.foo.com:22000):(Total: 262.332ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:25/6.15 GB 7:35/8.60 GB 5:27/6.57 GB 4:31/7.67 GB 1:25/6.03 GB 3:27/6.54 GB 6:24/5.90 GB 0:29/7.15 GB Fragment Instance Lifecycle Event Timeline: 1s669ms - Prepare Finished: 229.348ms (229.348ms) - Open Finished: 262.851ms (33.502ms) - First Batch Produced: 1s667ms (1s404ms) - First Batch Sent: 1s669ms (2.137ms) MemoryUsage(8m32s): 13.25 GB, 14.01 GB, 14.00 GB, 13.96 GB, 13.82 GB, 13.51 GB, 13.38 GB, 13.26 GB, 13.02 GB, 12.42 GB, 11.72 GB, 11.30 GB, 10.74 GB, 10.55 GB, 10.40 GB, 10.15 GB, 9.75 GB, 9.11 GB, 8.60 GB, 8.52 GB, 8.39 GB, 8.09 GB, 7.97 GB, 8.04 GB, 8.54 GB, 9.31 GB, 11.01 GB, 13.07 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB, 13.90 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 14.01 GB (15046663099) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.68 GB (172524821487) - RowsProduced: 601.00M (601001714) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 28m39s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 33.500ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s372ms - OpenTime: 33.501ms - ExecTreeOpenTime: 160.600us - PrepareTime: 228.756ms - ExecTreePrepareTime: 2.037ms KrpcDataStreamSender (dst_id=1):(Total: 11m9s, non-child: 11m9s, % non-child: 100.00%) - BytesSent: 78.92 GB (84742971177) - EosSent: 0 (0) - OverallThroughput: 120.77 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 601.00M (601000690) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m7s - UncompressedRowBatchSize: 154.92 GB (166346936180) HDFS_SCAN_NODE (id=0):(Total: 7s851ms, non-child: 7s851ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:25/6.15 GB 7:35/8.60 GB 5:27/6.57 GB 4:31/7.67 GB 1:25/6.03 GB 3:27/6.54 GB 6:24/5.90 GB 0:29/7.15 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.16 GB, 16.73 GB, 17.37 GB, 17.95 GB, 19.16 GB, 19.92 GB, 20.22 GB, 20.58 GB, 20.70 GB, 20.90 GB, 21.22 GB, 21.30 GB, 21.37 GB, 21.54 GB, 21.68 GB, 21.68 GB, 21.68 GB, 21.86 GB, 22.03 GB, 22.25 GB, 22.54 GB, 22.73 GB, 23.26 GB, 24.00 GB, 25.01 GB, 26.40 GB, 29.38 GB, 32.95 GB, 34.37 GB, 34.45 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB - FooterProcessingTime: (Avg: 2s277ms ; Min: 23.627ms ; Max: 13s465ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.51 GB (37052405176) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m59s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 14.01 GB (15043574007) - PerReadThreadRawHdfsThroughput: 52.01 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 601.21M (601214706) - RowsReturned: 601.00M (601001714) - RowsReturnedRate: 76.55 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m56s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 29s147ms - TotalRawHdfsReadTime(*): 11m19s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 33.030ms, non-child: 33.030ms, % non-child: 100.00%) - CodegenTime: 427.892us - CompileTime: 4.187ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.435ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 20.835ms Instance 844a0200d7876345:20bb38b900000068 (host=vc1508.foo.com:22000):(Total: 262.316ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:24/5.88 GB 0:29/7.14 GB 2:28/6.85 GB 6:30/6.97 GB 4:26/6.00 GB 3:31/7.67 GB 7:29/6.99 GB 5:30/7.34 GB Fragment Instance Lifecycle Event Timeline: 1s656ms - Prepare Finished: 237.190ms (237.190ms) - Open Finished: 262.766ms (25.575ms) - First Batch Produced: 1s654ms (1s391ms) - First Batch Sent: 1s656ms (2.241ms) MemoryUsage(8m32s): 13.11 GB, 13.91 GB, 13.88 GB, 13.77 GB, 13.55 GB, 13.23 GB, 13.08 GB, 12.93 GB, 12.65 GB, 11.95 GB, 11.40 GB, 10.94 GB, 10.48 GB, 10.27 GB, 10.04 GB, 9.79 GB, 9.51 GB, 8.92 GB, 8.86 GB, 8.87 GB, 8.81 GB, 8.76 GB, 8.72 GB, 8.98 GB, 9.43 GB, 10.30 GB, 12.04 GB, 13.75 GB, 13.83 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.94 GB (14962801434) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.57 GB (172411812995) - RowsProduced: 609.99M (609986026) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 30m31s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 25.589ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 23.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s240ms - OpenTime: 25.590ms - ExecTreeOpenTime: 165.631us - PrepareTime: 236.647ms - ExecTreePrepareTime: 1.925ms KrpcDataStreamSender (dst_id=1):(Total: 11m6s, non-child: 11m6s, % non-child: 100.00%) - BytesSent: 80.08 GB (85982266175) - EosSent: 0 (0) - OverallThroughput: 123.11 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 609.99M (609985002) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m4s - UncompressedRowBatchSize: 157.24 GB (168833865662) HDFS_SCAN_NODE (id=0):(Total: 8s735ms, non-child: 8s735ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:24/5.88 GB 0:29/7.14 GB 2:28/6.85 GB 6:30/6.97 GB 4:26/6.00 GB 3:31/7.67 GB 7:29/6.99 GB 5:30/7.34 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.29 GB, 16.77 GB, 17.43 GB, 18.09 GB, 19.24 GB, 19.83 GB, 20.13 GB, 20.51 GB, 20.71 GB, 20.94 GB, 21.20 GB, 21.25 GB, 21.33 GB, 21.50 GB, 21.58 GB, 21.69 GB, 21.99 GB, 22.30 GB, 22.76 GB, 23.12 GB, 23.56 GB, 24.08 GB, 24.68 GB, 25.60 GB, 26.43 GB, 28.03 GB, 31.13 GB, 34.05 GB, 34.29 GB, 34.43 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB - FooterProcessingTime: (Avg: 2s286ms ; Min: 26.203ms ; Max: 11s427ms ; Number of samples: 177) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.51 GB (37051535455) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m1s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 177 (177) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.93 GB (14959710629) - PerReadThreadRawHdfsThroughput: 52.93 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 610.20M (610199018) - RowsReturned: 609.99M (609986026) - RowsReturnedRate: 69.83 M/sec - ScanRangesComplete: 89 (89) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m39s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 22s543ms - TotalRawHdfsReadTime(*): 11m7s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 25.125ms, non-child: 25.125ms, % non-child: 100.00%) - CodegenTime: 417.121us - CompileTime: 3.819ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.648ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.122ms Instance 844a0200d7876345:20bb38b90000007f (host=vc1526.foo.com:22000):(Total: 261.755ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:29/7.05 GB 5:19/4.62 GB 2:24/5.90 GB 4:33/8.04 GB 6:29/7.18 GB 1:33/8.00 GB 3:31/7.50 GB 7:26/6.36 GB Fragment Instance Lifecycle Event Timeline: 1s871ms - Prepare Finished: 235.733ms (235.733ms) - Open Finished: 262.185ms (26.451ms) - First Batch Produced: 1s869ms (1s607ms) - First Batch Sent: 1s871ms (1.803ms) MemoryUsage(8m32s): 13.19 GB, 13.93 GB, 13.93 GB, 13.87 GB, 13.72 GB, 13.39 GB, 13.24 GB, 13.16 GB, 13.02 GB, 12.41 GB, 11.67 GB, 11.24 GB, 10.71 GB, 10.52 GB, 10.40 GB, 10.27 GB, 9.96 GB, 9.25 GB, 8.87 GB, 8.80 GB, 8.57 GB, 8.18 GB, 8.01 GB, 8.07 GB, 8.60 GB, 9.58 GB, 11.24 GB, 13.32 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.95 GB (14976838103) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.70 GB (172554349562) - RowsProduced: 600.32M (600320403) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 28m - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 26.447ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 26.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 5s672ms - OpenTime: 26.448ms - ExecTreeOpenTime: 192.127us - PrepareTime: 235.231ms - ExecTreePrepareTime: 1.903ms KrpcDataStreamSender (dst_id=1):(Total: 10m56s, non-child: 10m56s, % non-child: 100.00%) - BytesSent: 78.80 GB (84607060885) - EosSent: 0 (0) - OverallThroughput: 122.82 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 600.32M (600319379) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 10m55s - UncompressedRowBatchSize: 154.75 GB (166158099258) HDFS_SCAN_NODE (id=0):(Total: 5s225ms, non-child: 5s225ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:29/7.05 GB 5:19/4.62 GB 2:24/5.90 GB 4:33/8.04 GB 6:29/7.18 GB 1:33/8.00 GB 3:31/7.50 GB 7:26/6.36 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.87 GB, 16.45 GB, 17.14 GB, 17.71 GB, 18.86 GB, 19.69 GB, 20.01 GB, 20.39 GB, 20.64 GB, 20.84 GB, 21.15 GB, 21.23 GB, 21.31 GB, 21.48 GB, 21.63 GB, 21.78 GB, 21.90 GB, 21.96 GB, 22.30 GB, 22.54 GB, 22.67 GB, 22.72 GB, 23.13 GB, 23.87 GB, 24.95 GB, 26.72 GB, 29.67 GB, 33.26 GB, 34.33 GB, 34.43 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB - FooterProcessingTime: (Avg: 2s370ms ; Min: 25.418ms ; Max: 10s437ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.51 GB (37056982043) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m30s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.95 GB (14973714054) - PerReadThreadRawHdfsThroughput: 51.95 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 600.53M (600533395) - RowsReturned: 600.32M (600320403) - RowsReturnedRate: 114.87 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m49s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 22s664ms - TotalRawHdfsReadTime(*): 11m20s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 26.007ms, non-child: 26.007ms, % non-child: 100.00%) - CodegenTime: 357.035us - CompileTime: 4.122ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.467ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.937ms Instance 844a0200d7876345:20bb38b900000000 (host=vd1138.foo.com:22000):(Total: 261.568ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 6:34/8.26 GB 2:21/5.07 GB 1:21/5.16 GB 0:30/7.27 GB 3:27/6.56 GB 4:29/7.13 GB 5:30/7.25 GB 7:32/7.92 GB Fragment Instance Lifecycle Event Timeline: 1s884ms - Prepare Finished: 226.146ms (226.146ms) - Open Finished: 262.198ms (36.051ms) - First Batch Produced: 1s882ms (1s620ms) - First Batch Sent: 1s884ms (2.065ms) MemoryUsage(8m32s): 12.08 GB, 12.71 GB, 12.68 GB, 12.59 GB, 12.33 GB, 12.11 GB, 11.94 GB, 11.57 GB, 10.86 GB, 10.28 GB, 9.81 GB, 9.53 GB, 9.38 GB, 9.05 GB, 8.54 GB, 7.97 GB, 7.71 GB, 7.57 GB, 7.40 GB, 7.62 GB, 8.16 GB, 9.08 GB, 10.49 GB, 11.92 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.71 GB (13652366629) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.09 GB (186929579116) - RowsProduced: 592.61M (592605051) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 24m52s - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 36.044ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 12s198ms - OpenTime: 36.045ms - ExecTreeOpenTime: 156.830us - PrepareTime: 225.416ms - ExecTreePrepareTime: 1.755ms KrpcDataStreamSender (dst_id=1):(Total: 11m31s, non-child: 11m31s, % non-child: 100.00%) - BytesSent: 77.78 GB (83513326436) - EosSent: 0 (0) - OverallThroughput: 115.11 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 592.60M (592604027) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m30s - UncompressedRowBatchSize: 152.76 GB (164022086657) HDFS_SCAN_NODE (id=0):(Total: 11s658ms, non-child: 11s658ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:34/8.26 GB 2:21/5.07 GB 1:21/5.16 GB 0:30/7.27 GB 3:27/6.56 GB 4:29/7.13 GB 5:30/7.25 GB 7:32/7.92 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.20 GB, 15.34 GB, 16.00 GB, 16.88 GB, 17.93 GB, 18.33 GB, 18.65 GB, 18.82 GB, 19.06 GB, 19.27 GB, 19.36 GB, 19.51 GB, 19.74 GB, 19.75 GB, 19.79 GB, 20.01 GB, 20.25 GB, 20.57 GB, 20.98 GB, 21.96 GB, 23.29 GB, 24.97 GB, 27.42 GB, 29.93 GB, 31.27 GB, 31.39 GB, 31.53 GB, 31.72 GB, 31.95 GB, 32.24 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.41 GB, 32.42 GB, 32.42 GB, 32.42 GB, 32.42 GB, 32.42 GB, 32.42 GB, 32.42 GB, 32.42 GB - FooterProcessingTime: (Avg: 2s229ms ; Min: 32.856ms ; Max: 9s800ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.42 GB (34806272985) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m43s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.71 GB (13649203604) - PerReadThreadRawHdfsThroughput: 53.33 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 592.81M (592809851) - RowsReturned: 592.61M (592605051) - RowsReturnedRate: 50.83 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m24s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 20s959ms - TotalRawHdfsReadTime(*): 10m22s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 35.530ms, non-child: 35.530ms, % non-child: 100.00%) - CodegenTime: 482.059us - CompileTime: 4.370ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.562ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 23.038ms Instance 844a0200d7876345:20bb38b900000047 (host=vc1523.foo.com:22000):(Total: 261.002ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:36/8.79 GB 7:23/5.62 GB 6:26/6.31 GB 4:21/5.16 GB 3:27/6.29 GB 0:23/5.66 GB 5:38/9.41 GB 2:30/7.34 GB Fragment Instance Lifecycle Event Timeline: 1s901ms - Prepare Finished: 234.689ms (234.689ms) - Open Finished: 261.459ms (26.770ms) - First Batch Produced: 1s899ms (1s638ms) - First Batch Sent: 1s901ms (1.600ms) MemoryUsage(8m32s): 13.13 GB, 13.91 GB, 13.89 GB, 13.82 GB, 13.70 GB, 13.45 GB, 13.37 GB, 13.09 GB, 12.74 GB, 11.95 GB, 11.46 GB, 11.06 GB, 10.70 GB, 10.57 GB, 10.29 GB, 9.94 GB, 9.62 GB, 8.82 GB, 8.42 GB, 8.29 GB, 8.07 GB, 7.97 GB, 8.31 GB, 9.23 GB, 9.94 GB, 10.52 GB, 12.55 GB, 13.73 GB, 13.85 GB, 13.84 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.94 GB (14971016890) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.58 GB (172426092605) - RowsProduced: 610.76M (610761905) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h17m - TotalStorageWaitTime: 28m41s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 26.766ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 24.996ms - TotalThreadsVoluntaryContextSwitches: 8 (8) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s783ms - OpenTime: 26.767ms - ExecTreeOpenTime: 130.217us - PrepareTime: 234.144ms - ExecTreePrepareTime: 2.412ms KrpcDataStreamSender (dst_id=1):(Total: 11m8s, non-child: 11m8s, % non-child: 100.00%) - BytesSent: 80.16 GB (86075495027) - EosSent: 0 (0) - OverallThroughput: 122.79 MB/sec - PeakMemoryUsage: 3.59 MB (3760800) - RowsReturned: 610.76M (610760881) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m7s - UncompressedRowBatchSize: 157.44 GB (169048022463) HDFS_SCAN_NODE (id=0):(Total: 8s278ms, non-child: 8s278ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:36/8.79 GB 7:23/5.62 GB 6:26/6.31 GB 4:21/5.16 GB 3:27/6.29 GB 0:23/5.66 GB 5:38/9.41 GB 2:30/7.34 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.36 GB, 16.80 GB, 17.40 GB, 18.27 GB, 19.53 GB, 20.11 GB, 20.36 GB, 20.67 GB, 20.86 GB, 21.10 GB, 21.33 GB, 21.44 GB, 21.51 GB, 21.71 GB, 21.88 GB, 21.91 GB, 22.04 GB, 22.04 GB, 22.10 GB, 22.33 GB, 22.56 GB, 22.92 GB, 24.20 GB, 26.11 GB, 27.34 GB, 28.45 GB, 32.02 GB, 34.06 GB, 34.33 GB, 34.46 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB - FooterProcessingTime: (Avg: 1s678ms ; Min: 35.185ms ; Max: 6s964ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.52 GB (37067952796) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 6m33s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.94 GB (14967868210) - PerReadThreadRawHdfsThroughput: 50.58 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 610.97M (610974897) - RowsReturned: 610.76M (610761905) - RowsReturnedRate: 73.78 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m35s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 23s346ms - TotalRawHdfsReadTime(*): 11m38s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 26.356ms, non-child: 26.356ms, % non-child: 100.00%) - CodegenTime: 394.595us - CompileTime: 3.940ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.484ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.434ms Instance 844a0200d7876345:20bb38b900000010 (host=vc1519.foo.com:22000):(Total: 260.582ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:23/5.66 GB 2:20/4.95 GB 7:29/6.98 GB 4:31/7.58 GB 0:22/5.41 GB 3:35/8.62 GB 5:36/8.78 GB 6:28/6.79 GB Fragment Instance Lifecycle Event Timeline: 1s513ms - Prepare Finished: 231.109ms (231.109ms) - Open Finished: 261.126ms (30.016ms) - First Batch Produced: 1s511ms (1s250ms) - First Batch Sent: 1s513ms (1.911ms) MemoryUsage(8m32s): 13.24 GB, 13.97 GB, 13.97 GB, 13.93 GB, 13.72 GB, 13.41 GB, 13.31 GB, 13.18 GB, 12.83 GB, 12.17 GB, 11.50 GB, 11.06 GB, 10.64 GB, 10.46 GB, 10.26 GB, 9.90 GB, 9.49 GB, 8.95 GB, 8.63 GB, 8.57 GB, 8.37 GB, 8.11 GB, 8.03 GB, 8.51 GB, 9.28 GB, 10.30 GB, 11.87 GB, 13.09 GB, 13.86 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (15002267716) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.35 GB (188281490303) - RowsProduced: 602.07M (602067309) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 26m45s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 30.011ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 28.995ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 14s872ms - OpenTime: 30.013ms - ExecTreeOpenTime: 374.700us - PrepareTime: 230.489ms - ExecTreePrepareTime: 2.080ms KrpcDataStreamSender (dst_id=1):(Total: 11m5s, non-child: 11m5s, % non-child: 100.00%) - BytesSent: 79.01 GB (84835549922) - EosSent: 0 (0) - OverallThroughput: 121.51 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 602.07M (602066285) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m4s - UncompressedRowBatchSize: 155.20 GB (166641038504) HDFS_SCAN_NODE (id=0):(Total: 14s390ms, non-child: 14s390ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:23/5.66 GB 2:20/4.95 GB 7:29/6.98 GB 4:31/7.58 GB 0:22/5.41 GB 3:35/8.62 GB 5:36/8.78 GB 6:28/6.79 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.29 GB, 16.71 GB, 17.30 GB, 18.02 GB, 19.20 GB, 19.87 GB, 20.17 GB, 20.47 GB, 20.64 GB, 20.86 GB, 21.09 GB, 21.19 GB, 21.29 GB, 21.44 GB, 21.52 GB, 21.53 GB, 21.57 GB, 21.74 GB, 22.06 GB, 22.42 GB, 22.66 GB, 22.82 GB, 23.33 GB, 24.54 GB, 26.09 GB, 27.87 GB, 30.65 GB, 32.78 GB, 34.16 GB, 34.40 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB - FooterProcessingTime: (Avg: 1s711ms ; Min: 34.864ms ; Max: 7s779ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.51 GB (37060106118) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m52s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (14999094470) - PerReadThreadRawHdfsThroughput: 53.56 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 602.28M (602280301) - RowsReturned: 602.07M (602067309) - RowsReturnedRate: 41.84 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m21s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 27s591ms - TotalRawHdfsReadTime(*): 10m59s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 29.307ms, non-child: 29.307ms, % non-child: 100.00%) - CodegenTime: 495.861us - CompileTime: 4.344ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.329ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 17.034ms Instance 844a0200d7876345:20bb38b900000060 (host=vc1517.foo.com:22000):(Total: 260.251ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:28/6.86 GB 0:24/5.82 GB 5:29/7.14 GB 1:16/3.92 GB 3:25/6.03 GB 7:30/7.36 GB 4:39/9.39 GB 6:34/8.29 GB Fragment Instance Lifecycle Event Timeline: 1s539ms - Prepare Finished: 235.453ms (235.453ms) - Open Finished: 260.836ms (25.383ms) - First Batch Produced: 1s537ms (1s276ms) - First Batch Sent: 1s539ms (2.030ms) MemoryUsage(8m32s): 13.05 GB, 13.85 GB, 13.92 GB, 13.89 GB, 13.79 GB, 13.54 GB, 13.30 GB, 13.18 GB, 13.05 GB, 12.60 GB, 11.90 GB, 11.42 GB, 10.91 GB, 10.51 GB, 10.41 GB, 10.33 GB, 10.12 GB, 9.69 GB, 9.19 GB, 8.80 GB, 8.86 GB, 8.81 GB, 8.70 GB, 8.45 GB, 8.25 GB, 8.65 GB, 9.66 GB, 11.01 GB, 12.93 GB, 13.78 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.96 GB (14985505783) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.68 GB (172525689887) - RowsProduced: 590.06M (590058314) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h13m - TotalStorageWaitTime: 33m38s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 25.378ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 22.997ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 15s393ms - OpenTime: 25.379ms - ExecTreeOpenTime: 202.570us - PrepareTime: 234.795ms - ExecTreePrepareTime: 1.937ms KrpcDataStreamSender (dst_id=1):(Total: 11m6s, non-child: 11m6s, % non-child: 100.00%) - BytesSent: 77.41 GB (83116012522) - EosSent: 0 (0) - OverallThroughput: 118.91 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 590.06M (590057290) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m5s - UncompressedRowBatchSize: 152.10 GB (163317627677) HDFS_SCAN_NODE (id=0):(Total: 14s883ms, non-child: 14s883ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:28/6.86 GB 0:24/5.82 GB 5:29/7.14 GB 1:16/3.92 GB 3:25/6.03 GB 7:30/7.36 GB 4:39/9.39 GB 6:34/8.29 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.69 GB, 16.18 GB, 17.13 GB, 17.69 GB, 18.59 GB, 19.59 GB, 19.96 GB, 20.36 GB, 20.54 GB, 20.67 GB, 20.97 GB, 21.14 GB, 21.19 GB, 21.40 GB, 21.64 GB, 21.70 GB, 21.82 GB, 22.01 GB, 22.29 GB, 22.32 GB, 22.72 GB, 23.18 GB, 23.61 GB, 23.91 GB, 24.09 GB, 25.00 GB, 26.80 GB, 29.26 GB, 32.62 GB, 34.12 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB, 34.36 GB - FooterProcessingTime: (Avg: 3s398ms ; Min: 16.811ms ; Max: 14s155ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.36 GB (36893519460) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 12m1s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.95 GB (14982357249) - PerReadThreadRawHdfsThroughput: 48.01 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 590.27M (590271306) - RowsReturned: 590.06M (590058314) - RowsReturnedRate: 39.64 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m48s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 31s907ms - TotalRawHdfsReadTime(*): 12m12s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 24.976ms, non-child: 24.976ms, % non-child: 100.00%) - CodegenTime: 331.394us - CompileTime: 4.451ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.555ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 13.483ms Instance 844a0200d7876345:20bb38b900000021 (host=vd1136.foo.com:22000):(Total: 260.074ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:29/7.06 GB 1:25/6.12 GB 3:33/8.01 GB 0:32/7.86 GB 5:33/7.84 GB 7:25/6.16 GB 6:29/7.05 GB 4:19/4.63 GB Fragment Instance Lifecycle Event Timeline: 1s662ms - Prepare Finished: 226.677ms (226.677ms) - Open Finished: 260.773ms (34.095ms) - First Batch Produced: 1s660ms (1s399ms) - First Batch Sent: 1s662ms (1.995ms) MemoryUsage(8m32s): 12.08 GB, 12.65 GB, 12.65 GB, 12.57 GB, 12.23 GB, 11.95 GB, 11.90 GB, 11.69 GB, 11.05 GB, 10.38 GB, 9.85 GB, 9.52 GB, 9.42 GB, 9.14 GB, 8.67 GB, 8.08 GB, 7.88 GB, 7.85 GB, 7.62 GB, 7.34 GB, 7.81 GB, 8.63 GB, 10.17 GB, 11.86 GB, 12.64 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.68 GB (13610463889) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.06 GB (186897707003) - RowsProduced: 585.93M (585929814) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h13m - TotalStorageWaitTime: 23m4s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 34.091ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 30.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 11s062ms - OpenTime: 34.093ms - ExecTreeOpenTime: 241.132us - PrepareTime: 225.912ms - ExecTreePrepareTime: 2.156ms KrpcDataStreamSender (dst_id=1):(Total: 11m31s, non-child: 11m31s, % non-child: 100.00%) - BytesSent: 76.97 GB (82649369528) - EosSent: 0 (0) - OverallThroughput: 113.91 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 585.93M (585928790) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m30s - UncompressedRowBatchSize: 151.04 GB (162176350971) HDFS_SCAN_NODE (id=0):(Total: 10s538ms, non-child: 10s538ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:29/7.06 GB 1:25/6.12 GB 3:33/8.01 GB 0:32/7.86 GB 5:33/7.84 GB 7:25/6.16 GB 6:29/7.05 GB 4:19/4.63 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.07 GB, 15.19 GB, 15.77 GB, 16.53 GB, 17.66 GB, 18.10 GB, 18.44 GB, 18.69 GB, 18.97 GB, 19.20 GB, 19.29 GB, 19.41 GB, 19.55 GB, 19.59 GB, 19.67 GB, 19.96 GB, 20.30 GB, 20.78 GB, 21.08 GB, 21.34 GB, 22.57 GB, 24.08 GB, 26.81 GB, 29.72 GB, 31.09 GB, 31.22 GB, 31.37 GB, 31.60 GB, 31.83 GB, 31.99 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.08 GB, 32.09 GB, 32.09 GB, 32.09 GB, 32.09 GB, 32.09 GB, 32.09 GB, 32.09 GB, 32.09 GB, 32.09 GB - FooterProcessingTime: (Avg: 2s095ms ; Min: 15.871ms ; Max: 8s881ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.09 GB (34454757102) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 11m19s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.67 GB (13607292828) - PerReadThreadRawHdfsThroughput: 54.65 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 586.13M (586134614) - RowsReturned: 585.93M (585929814) - RowsReturnedRate: 55.60 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m14s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 19s023ms - TotalRawHdfsReadTime(*): 10m1s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 33.482ms, non-child: 33.482ms, % non-child: 100.00%) - CodegenTime: 510.270us - CompileTime: 4.926ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.967ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 19.042ms Instance 844a0200d7876345:20bb38b900000029 (host=vc1533.foo.com:22000):(Total: 259.787ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:16/3.89 GB 3:25/6.12 GB 5:27/6.60 GB 1:28/6.93 GB 6:33/8.13 GB 4:31/7.61 GB 2:29/7.14 GB 0:32/7.89 GB Fragment Instance Lifecycle Event Timeline: 2s030ms - Prepare Finished: 235.660ms (235.660ms) - Open Finished: 260.228ms (24.568ms) - First Batch Produced: 2s028ms (1s768ms) - First Batch Sent: 2s030ms (1.930ms) MemoryUsage(8m32s): 13.15 GB, 13.99 GB, 13.98 GB, 13.90 GB, 13.70 GB, 13.44 GB, 13.34 GB, 13.11 GB, 12.71 GB, 12.04 GB, 11.39 GB, 10.96 GB, 10.61 GB, 10.41 GB, 10.10 GB, 9.72 GB, 9.41 GB, 8.90 GB, 8.59 GB, 8.43 GB, 8.14 GB, 8.19 GB, 8.51 GB, 9.09 GB, 10.14 GB, 11.30 GB, 12.67 GB, 13.81 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15019982253) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.40 GB (188336111681) - RowsProduced: 612.88M (612878529) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 31m33s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 24.547ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 23.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s375ms - OpenTime: 24.549ms - ExecTreeOpenTime: 208.209us - PrepareTime: 235.166ms - ExecTreePrepareTime: 2.008ms KrpcDataStreamSender (dst_id=1):(Total: 10m45s, non-child: 10m45s, % non-child: 100.00%) - BytesSent: 80.44 GB (86367184308) - EosSent: 0 (0) - OverallThroughput: 127.56 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 612.88M (612877505) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 10m44s - UncompressedRowBatchSize: 157.98 GB (169633749711) HDFS_SCAN_NODE (id=0):(Total: 7s914ms, non-child: 7s914ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:16/3.89 GB 3:25/6.12 GB 5:27/6.60 GB 1:28/6.93 GB 6:33/8.13 GB 4:31/7.61 GB 2:29/7.14 GB 0:32/7.89 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.24 GB, 16.75 GB, 17.40 GB, 18.20 GB, 19.32 GB, 19.98 GB, 20.29 GB, 20.58 GB, 20.75 GB, 20.97 GB, 21.20 GB, 21.30 GB, 21.41 GB, 21.54 GB, 21.61 GB, 21.67 GB, 21.99 GB, 22.14 GB, 22.40 GB, 22.60 GB, 22.73 GB, 23.49 GB, 24.48 GB, 25.82 GB, 27.71 GB, 29.74 GB, 32.15 GB, 34.19 GB, 34.55 GB, 34.68 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB, 34.78 GB - FooterProcessingTime: (Avg: 2s094ms ; Min: 35.218ms ; Max: 7s803ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.78 GB (37344565338) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m40s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.99 GB (15016915325) - PerReadThreadRawHdfsThroughput: 52.07 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 613.09M (613091521) - RowsReturned: 612.88M (612878529) - RowsReturnedRate: 77.43 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m32s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 27s237ms - TotalRawHdfsReadTime(*): 11m23s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 24.124ms, non-child: 24.124ms, % non-child: 100.00%) - CodegenTime: 332.025us - CompileTime: 4.012ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.218ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 13.449ms Instance 844a0200d7876345:20bb38b90000005f (host=vc1509.foo.com:22000):(Total: 259.781ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:22/5.41 GB 2:26/6.37 GB 4:27/6.67 GB 0:29/7.10 GB 1:21/5.16 GB 5:37/9.00 GB 6:32/7.85 GB 3:30/7.19 GB Fragment Instance Lifecycle Event Timeline: 1s738ms - Prepare Finished: 233.325ms (233.325ms) - Open Finished: 260.548ms (27.223ms) - First Batch Produced: 1s736ms (1s476ms) - First Batch Sent: 1s738ms (2.136ms) MemoryUsage(8m32s): 13.23 GB, 13.95 GB, 13.94 GB, 13.88 GB, 13.67 GB, 13.34 GB, 13.22 GB, 13.15 GB, 12.83 GB, 12.14 GB, 11.47 GB, 11.00 GB, 10.50 GB, 10.35 GB, 10.35 GB, 10.11 GB, 9.68 GB, 9.20 GB, 8.96 GB, 8.89 GB, 8.63 GB, 8.24 GB, 7.87 GB, 8.18 GB, 8.93 GB, 9.95 GB, 11.73 GB, 13.29 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB, 13.98 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15016399004) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.73 GB (172584902379) - RowsProduced: 600.01M (600012560) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 27m1s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 27.219ms - TotalThreadsSysTime: 3.999ms - TotalThreadsUserTime: 22.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s267ms - OpenTime: 27.220ms - ExecTreeOpenTime: 296.477us - PrepareTime: 232.484ms - ExecTreePrepareTime: 1.722ms KrpcDataStreamSender (dst_id=1):(Total: 11m7s, non-child: 11m7s, % non-child: 100.00%) - BytesSent: 78.73 GB (84537295566) - EosSent: 0 (0) - OverallThroughput: 120.74 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 600.01M (600011536) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m6s - UncompressedRowBatchSize: 154.67 GB (166072458302) HDFS_SCAN_NODE (id=0):(Total: 9s745ms, non-child: 9s745ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:22/5.41 GB 2:26/6.37 GB 4:27/6.67 GB 0:29/7.10 GB 1:21/5.16 GB 5:37/9.00 GB 6:32/7.85 GB 3:30/7.19 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.15 GB, 16.67 GB, 17.24 GB, 17.84 GB, 19.09 GB, 19.75 GB, 20.08 GB, 20.38 GB, 20.51 GB, 20.75 GB, 20.99 GB, 21.07 GB, 21.16 GB, 21.33 GB, 21.57 GB, 21.69 GB, 21.74 GB, 22.10 GB, 22.46 GB, 22.72 GB, 22.88 GB, 22.88 GB, 22.99 GB, 24.01 GB, 25.46 GB, 27.29 GB, 30.42 GB, 33.11 GB, 34.31 GB, 34.46 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB - FooterProcessingTime: (Avg: 2s129ms ; Min: 30.436ms ; Max: 8s611ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.55 GB (37094120684) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m41s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.98 GB (15013194233) - PerReadThreadRawHdfsThroughput: 54.92 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 600.23M (600225552) - RowsReturned: 600.01M (600012560) - RowsReturnedRate: 61.57 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m16s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 38s244ms - TotalRawHdfsReadTime(*): 10m44s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 26.650ms, non-child: 26.650ms, % non-child: 100.00%) - CodegenTime: 368.685us - CompileTime: 4.760ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.995ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.435ms Instance 844a0200d7876345:20bb38b90000001b (host=vd1139.foo.com:22000):(Total: 259.498ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:24/5.91 GB 4:27/6.65 GB 2:29/7.03 GB 7:20/4.88 GB 6:20/4.80 GB 3:37/8.66 GB 0:33/8.01 GB 1:36/8.84 GB Fragment Instance Lifecycle Event Timeline: 1s744ms - Prepare Finished: 221.493ms (221.493ms) - Open Finished: 260.001ms (38.508ms) - First Batch Produced: 1s742ms (1s482ms) - First Batch Sent: 1s744ms (2.088ms) MemoryUsage(8m32s): 12.03 GB, 12.63 GB, 12.59 GB, 12.36 GB, 12.01 GB, 11.88 GB, 11.65 GB, 11.15 GB, 10.35 GB, 9.82 GB, 9.46 GB, 9.27 GB, 9.18 GB, 8.88 GB, 8.29 GB, 8.21 GB, 8.13 GB, 7.89 GB, 7.81 GB, 8.24 GB, 8.99 GB, 10.08 GB, 11.74 GB, 12.49 GB, 12.51 GB, 12.51 GB, 12.50 GB, 12.51 GB, 12.55 GB, 12.60 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.64 GB (13576580266) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.01 GB (186843523729) - RowsProduced: 600.36M (600362128) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 22m12s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 38.497ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 35.995ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s545ms - OpenTime: 38.499ms - ExecTreeOpenTime: 140.128us - PrepareTime: 220.921ms - ExecTreePrepareTime: 2.090ms KrpcDataStreamSender (dst_id=1):(Total: 11m26s, non-child: 11m26s, % non-child: 100.00%) - BytesSent: 78.77 GB (84582215301) - EosSent: 0 (0) - OverallThroughput: 117.49 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 600.36M (600361104) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m25s - UncompressedRowBatchSize: 154.76 GB (166169780738) HDFS_SCAN_NODE (id=0):(Total: 10s047ms, non-child: 10s047ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:24/5.91 GB 4:27/6.65 GB 2:29/7.03 GB 7:20/4.88 GB 6:20/4.80 GB 3:37/8.66 GB 0:33/8.01 GB 1:36/8.84 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.56 GB, 15.39 GB, 16.03 GB, 17.04 GB, 17.86 GB, 18.22 GB, 18.48 GB, 18.64 GB, 18.90 GB, 19.04 GB, 19.27 GB, 19.41 GB, 19.73 GB, 19.93 GB, 20.15 GB, 20.80 GB, 21.15 GB, 21.40 GB, 21.97 GB, 23.19 GB, 24.65 GB, 26.71 GB, 29.59 GB, 30.93 GB, 31.15 GB, 31.33 GB, 31.50 GB, 31.73 GB, 32.00 GB, 32.35 GB, 32.57 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB, 32.58 GB - FooterProcessingTime: (Avg: 1s895ms ; Min: 43.732ms ; Max: 8s555ms ; Number of samples: 161) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.58 GB (34980494886) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m12s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 161 (161) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.64 GB (13573502814) - PerReadThreadRawHdfsThroughput: 48.30 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 600.57M (600566928) - RowsReturned: 600.36M (600362128) - RowsReturnedRate: 59.75 M/sec - ScanRangesComplete: 81 (81) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m41s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 21s061ms - TotalRawHdfsReadTime(*): 11m30s - TotalReadThroughput: 1.16 MB/sec CodeGen:(Total: 38.025ms, non-child: 38.025ms, % non-child: 100.00%) - CodegenTime: 458.191us - CompileTime: 4.549ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.849ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 25.042ms Instance 844a0200d7876345:20bb38b900000080 (host=vc1529.foo.com:22000):(Total: 258.287ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:26/6.40 GB 2:37/9.01 GB 3:29/7.10 GB 0:24/5.94 GB 1:29/7.10 GB 4:26/6.36 GB 6:24/5.78 GB 7:29/7.14 GB Fragment Instance Lifecycle Event Timeline: 1s840ms - Prepare Finished: 233.237ms (233.237ms) - Open Finished: 258.748ms (25.511ms) - First Batch Produced: 1s838ms (1s580ms) - First Batch Sent: 1s840ms (1.897ms) MemoryUsage(8m32s): 13.26 GB, 13.98 GB, 13.98 GB, 13.93 GB, 13.72 GB, 13.39 GB, 13.29 GB, 13.10 GB, 12.77 GB, 12.01 GB, 11.45 GB, 11.02 GB, 10.62 GB, 10.41 GB, 10.14 GB, 9.86 GB, 9.36 GB, 8.64 GB, 8.40 GB, 8.31 GB, 8.20 GB, 8.14 GB, 8.29 GB, 8.91 GB, 9.44 GB, 10.57 GB, 12.30 GB, 13.78 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15018540629) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.71 GB (172557724564) - RowsProduced: 606.54M (606536991) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 26m34s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 25.507ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 23.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s435ms - OpenTime: 25.508ms - ExecTreeOpenTime: 148.022us - PrepareTime: 232.708ms - ExecTreePrepareTime: 2.034ms KrpcDataStreamSender (dst_id=1):(Total: 11m9s, non-child: 11m9s, % non-child: 100.00%) - BytesSent: 79.62 GB (85493013549) - EosSent: 0 (0) - OverallThroughput: 121.85 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 606.54M (606535967) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m7s - UncompressedRowBatchSize: 156.35 GB (167879392220) HDFS_SCAN_NODE (id=0):(Total: 8s899ms, non-child: 8s899ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:26/6.40 GB 2:37/9.01 GB 3:29/7.10 GB 0:24/5.94 GB 1:29/7.10 GB 4:26/6.36 GB 6:24/5.78 GB 7:29/7.14 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.43 GB, 16.83 GB, 17.45 GB, 18.19 GB, 19.41 GB, 19.96 GB, 20.25 GB, 20.56 GB, 20.69 GB, 20.93 GB, 21.17 GB, 21.25 GB, 21.33 GB, 21.50 GB, 21.56 GB, 21.56 GB, 21.56 GB, 21.65 GB, 21.96 GB, 22.22 GB, 22.58 GB, 23.14 GB, 24.00 GB, 25.46 GB, 26.44 GB, 28.45 GB, 31.48 GB, 33.99 GB, 34.35 GB, 34.48 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB, 34.55 GB - FooterProcessingTime: (Avg: 2s176ms ; Min: 20.318ms ; Max: 11s117ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.55 GB (37093991017) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m10s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.98 GB (15015416523) - PerReadThreadRawHdfsThroughput: 53.94 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 606.75M (606749983) - RowsReturned: 606.54M (606536991) - RowsReturnedRate: 68.15 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m13s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 20s854ms - TotalRawHdfsReadTime(*): 10m55s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 25.072ms, non-child: 25.072ms, % non-child: 100.00%) - CodegenTime: 372.842us - CompileTime: 4.037ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.672ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 13.887ms Instance 844a0200d7876345:20bb38b900000014 (host=va1033.foo.com:22000):(Total: 257.959ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:22/5.36 GB 6:33/8.09 GB 7:16/3.89 GB 2:26/6.36 GB 3:33/8.12 GB 0:34/8.12 GB 4:34/8.29 GB 1:26/6.35 GB Fragment Instance Lifecycle Event Timeline: 1s745ms - Prepare Finished: 222.993ms (222.993ms) - Open Finished: 258.452ms (35.459ms) - First Batch Produced: 1s743ms (1s484ms) - First Batch Sent: 1s745ms (1.882ms) MemoryUsage(8m32s): 13.30 GB, 13.98 GB, 13.96 GB, 13.89 GB, 13.64 GB, 13.34 GB, 13.28 GB, 13.11 GB, 12.74 GB, 11.98 GB, 11.40 GB, 10.91 GB, 10.53 GB, 10.45 GB, 10.42 GB, 10.11 GB, 9.60 GB, 8.97 GB, 8.83 GB, 8.63 GB, 8.33 GB, 7.97 GB, 7.79 GB, 8.17 GB, 9.00 GB, 10.25 GB, 12.00 GB, 13.62 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15017872290) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.26 GB (188180250498) - RowsProduced: 604.46M (604456755) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h13m - TotalStorageWaitTime: 25m31s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 35.457ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 33.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 12s284ms - OpenTime: 35.458ms - ExecTreeOpenTime: 213.876us - PrepareTime: 222.420ms - ExecTreePrepareTime: 2.109ms KrpcDataStreamSender (dst_id=1):(Total: 11m3s, non-child: 11m3s, % non-child: 100.00%) - BytesSent: 79.35 GB (85203373951) - EosSent: 0 (0) - OverallThroughput: 122.47 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 604.46M (604455731) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m2s - UncompressedRowBatchSize: 155.81 GB (167303011457) HDFS_SCAN_NODE (id=0):(Total: 11s785ms, non-child: 11s785ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:22/5.36 GB 6:33/8.09 GB 7:16/3.89 GB 2:26/6.36 GB 3:33/8.12 GB 0:34/8.12 GB 4:34/8.29 GB 1:26/6.35 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.47 GB, 16.84 GB, 17.42 GB, 18.16 GB, 19.44 GB, 19.95 GB, 20.29 GB, 20.52 GB, 20.66 GB, 20.93 GB, 21.13 GB, 21.21 GB, 21.32 GB, 21.54 GB, 21.80 GB, 21.84 GB, 21.84 GB, 21.98 GB, 22.39 GB, 22.50 GB, 22.60 GB, 22.71 GB, 23.05 GB, 24.25 GB, 25.80 GB, 28.03 GB, 31.07 GB, 33.85 GB, 34.26 GB, 34.41 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB - FooterProcessingTime: (Avg: 2s296ms ; Min: 22.109ms ; Max: 8s410ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.49 GB (37033089266) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 12m35s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.98 GB (15014845605) - PerReadThreadRawHdfsThroughput: 51.15 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 604.67M (604669747) - RowsReturned: 604.46M (604456755) - RowsReturnedRate: 51.29 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m46s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 20s310ms - TotalRawHdfsReadTime(*): 11m30s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 34.953ms, non-child: 34.953ms, % non-child: 100.00%) - CodegenTime: 421.313us - CompileTime: 4.124ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.689ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.561ms Instance 844a0200d7876345:20bb38b90000000c (host=vd1116.foo.com:22000):(Total: 257.811ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:32/7.85 GB 4:27/6.60 GB 6:24/5.87 GB 0:40/9.79 GB 2:32/7.80 GB 5:23/5.58 GB 3:19/4.67 GB 7:27/6.68 GB Fragment Instance Lifecycle Event Timeline: 1s853ms - Prepare Finished: 225.506ms (225.506ms) - Open Finished: 258.254ms (32.748ms) - First Batch Produced: 1s851ms (1s593ms) - First Batch Sent: 1s853ms (1.728ms) MemoryUsage(8m32s): 13.26 GB, 13.95 GB, 13.93 GB, 13.81 GB, 13.57 GB, 13.26 GB, 13.19 GB, 13.06 GB, 12.71 GB, 11.99 GB, 11.35 GB, 10.89 GB, 10.45 GB, 10.32 GB, 10.27 GB, 10.09 GB, 9.78 GB, 9.23 GB, 8.95 GB, 8.98 GB, 8.71 GB, 8.37 GB, 8.22 GB, 8.30 GB, 9.00 GB, 10.07 GB, 11.61 GB, 13.51 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB, 14.00 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 14.00 GB (15029013380) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.39 GB (188326743535) - RowsProduced: 602.49M (602489733) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h12m - TotalStorageWaitTime: 25m18s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 32.744ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 29.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 15s368ms - OpenTime: 32.745ms - ExecTreeOpenTime: 192.835us - PrepareTime: 224.986ms - ExecTreePrepareTime: 1.985ms KrpcDataStreamSender (dst_id=1):(Total: 11m, non-child: 11m, % non-child: 100.00%) - BytesSent: 79.05 GB (84881120756) - EosSent: 0 (0) - OverallThroughput: 122.59 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 602.49M (602488709) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 10m59s - UncompressedRowBatchSize: 155.31 GB (166758956167) HDFS_SCAN_NODE (id=0):(Total: 14s908ms, non-child: 14s908ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:32/7.85 GB 4:27/6.60 GB 6:24/5.87 GB 0:40/9.79 GB 2:32/7.80 GB 5:23/5.58 GB 3:19/4.67 GB 7:27/6.68 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.45 GB, 16.81 GB, 17.36 GB, 18.04 GB, 19.26 GB, 19.81 GB, 20.12 GB, 20.40 GB, 20.52 GB, 20.75 GB, 21.00 GB, 21.06 GB, 21.15 GB, 21.35 GB, 21.59 GB, 21.80 GB, 21.99 GB, 22.21 GB, 22.55 GB, 22.93 GB, 23.01 GB, 23.14 GB, 23.48 GB, 24.24 GB, 25.59 GB, 27.50 GB, 30.20 GB, 33.50 GB, 34.36 GB, 34.49 GB, 34.58 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB - FooterProcessingTime: (Avg: 1s850ms ; Min: 33.550ms ; Max: 9s001ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.59 GB (37137871906) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 13m57s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.99 GB (15025881212) - PerReadThreadRawHdfsThroughput: 50.16 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 602.70M (602702725) - RowsReturned: 602.49M (602489733) - RowsReturnedRate: 40.41 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 10m51s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 24s339ms - TotalRawHdfsReadTime(*): 11m46s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 32.193ms, non-child: 32.193ms, % non-child: 100.00%) - CodegenTime: 469.113us - CompileTime: 4.318ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.986ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 19.260ms Instance 844a0200d7876345:20bb38b90000003d (host=vd1109.foo.com:22000):(Total: 257.412ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:35/8.56 GB 3:31/7.51 GB 6:24/5.94 GB 0:25/6.14 GB 4:23/5.57 GB 7:28/6.81 GB 2:32/7.85 GB 5:26/6.23 GB Fragment Instance Lifecycle Event Timeline: 1s987ms - Prepare Finished: 230.482ms (230.482ms) - Open Finished: 257.892ms (27.410ms) - First Batch Produced: 1s985ms (1s727ms) - First Batch Sent: 1s987ms (2.026ms) MemoryUsage(8m32s): 13.18 GB, 13.95 GB, 13.92 GB, 13.83 GB, 13.62 GB, 13.34 GB, 13.19 GB, 13.04 GB, 12.73 GB, 12.13 GB, 11.39 GB, 11.00 GB, 10.58 GB, 10.38 GB, 10.17 GB, 10.00 GB, 9.90 GB, 9.41 GB, 8.90 GB, 8.75 GB, 8.66 GB, 8.46 GB, 8.38 GB, 8.69 GB, 9.23 GB, 9.82 GB, 11.03 GB, 12.70 GB, 13.85 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.96 GB (14988365508) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.70 GB (172550786545) - RowsProduced: 600.37M (600365688) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h12m - TotalStorageWaitTime: 29m54s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 27.405ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 25.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 22s107ms - OpenTime: 27.406ms - ExecTreeOpenTime: 141.501us - PrepareTime: 229.911ms - ExecTreePrepareTime: 2.006ms KrpcDataStreamSender (dst_id=1):(Total: 11m9s, non-child: 11m9s, % non-child: 100.00%) - BytesSent: 78.79 GB (84599021746) - EosSent: 0 (0) - OverallThroughput: 120.54 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 600.36M (600364664) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m8s - UncompressedRowBatchSize: 154.76 GB (166170396590) HDFS_SCAN_NODE (id=0):(Total: 21s585ms, non-child: 21s585ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:35/8.56 GB 3:31/7.51 GB 6:24/5.94 GB 0:25/6.14 GB 4:23/5.57 GB 7:28/6.81 GB 2:32/7.85 GB 5:26/6.23 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.12 GB, 16.64 GB, 17.30 GB, 17.97 GB, 19.06 GB, 19.78 GB, 20.06 GB, 20.39 GB, 20.55 GB, 20.71 GB, 20.98 GB, 21.10 GB, 21.26 GB, 21.44 GB, 21.57 GB, 21.77 GB, 22.10 GB, 22.36 GB, 22.50 GB, 22.68 GB, 22.99 GB, 23.20 GB, 23.70 GB, 24.80 GB, 25.98 GB, 27.08 GB, 29.26 GB, 32.21 GB, 34.23 GB, 34.49 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB - FooterProcessingTime: (Avg: 2s153ms ; Min: 48.803ms ; Max: 7s972ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.56 GB (37104237559) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 13m43s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.96 GB (14985145159) - PerReadThreadRawHdfsThroughput: 47.03 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 600.58M (600578680) - RowsReturned: 600.37M (600365688) - RowsReturnedRate: 27.81 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m44s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 25s455ms - TotalRawHdfsReadTime(*): 12m32s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 27.000ms, non-child: 27.000ms, % non-child: 100.00%) - CodegenTime: 372.810us - CompileTime: 4.156ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.654ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 15.709ms Instance 844a0200d7876345:20bb38b900000041 (host=vc1518.foo.com:22000):(Total: 256.926ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:27/6.65 GB 2:31/7.57 GB 0:33/7.87 GB 1:28/6.81 GB 3:24/5.83 GB 4:30/7.12 GB 7:25/5.92 GB 6:28/6.81 GB Fragment Instance Lifecycle Event Timeline: 1s615ms - Prepare Finished: 231.478ms (231.478ms) - Open Finished: 257.430ms (25.952ms) - First Batch Produced: 1s613ms (1s356ms) - First Batch Sent: 1s615ms (1.945ms) MemoryUsage(8m32s): 13.12 GB, 13.94 GB, 13.92 GB, 13.84 GB, 13.62 GB, 13.33 GB, 13.19 GB, 13.02 GB, 12.75 GB, 12.09 GB, 11.42 GB, 11.01 GB, 10.51 GB, 10.29 GB, 10.15 GB, 9.89 GB, 9.78 GB, 9.31 GB, 8.91 GB, 8.86 GB, 8.69 GB, 8.55 GB, 8.61 GB, 8.82 GB, 9.18 GB, 9.94 GB, 11.20 GB, 12.85 GB, 13.85 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.95 GB (14976636513) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.62 GB (172468846769) - RowsProduced: 601.72M (601719662) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 29m20s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 25.976ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 24.996ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s133ms - OpenTime: 25.978ms - ExecTreeOpenTime: 156.984us - PrepareTime: 230.873ms - ExecTreePrepareTime: 2.054ms KrpcDataStreamSender (dst_id=1):(Total: 11m10s, non-child: 11m10s, % non-child: 100.00%) - BytesSent: 78.95 GB (84772367917) - EosSent: 0 (0) - OverallThroughput: 120.54 MB/sec - PeakMemoryUsage: 3.59 MB (3760800) - RowsReturned: 601.72M (601718638) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m9s - UncompressedRowBatchSize: 155.11 GB (166545811809) HDFS_SCAN_NODE (id=0):(Total: 9s634ms, non-child: 9s634ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:27/6.65 GB 2:31/7.57 GB 0:33/7.87 GB 1:28/6.81 GB 3:24/5.83 GB 4:30/7.12 GB 7:25/5.92 GB 6:28/6.81 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.23 GB, 16.71 GB, 17.31 GB, 17.97 GB, 19.06 GB, 19.77 GB, 20.07 GB, 20.40 GB, 20.54 GB, 20.72 GB, 20.99 GB, 21.09 GB, 21.15 GB, 21.32 GB, 21.55 GB, 21.56 GB, 21.94 GB, 22.27 GB, 22.58 GB, 22.88 GB, 23.04 GB, 23.38 GB, 24.10 GB, 25.04 GB, 25.92 GB, 27.25 GB, 29.52 GB, 32.51 GB, 34.30 GB, 34.45 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB - FooterProcessingTime: (Avg: 1s901ms ; Min: 23.433ms ; Max: 10s810ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.51 GB (37059631183) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m56s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.95 GB (14973520616) - PerReadThreadRawHdfsThroughput: 50.13 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 601.93M (601932654) - RowsReturned: 601.72M (601719662) - RowsReturnedRate: 62.46 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m38s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 27s782ms - TotalRawHdfsReadTime(*): 11m44s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 25.502ms, non-child: 25.502ms, % non-child: 100.00%) - CodegenTime: 460.708us - CompileTime: 4.056ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.341ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.533ms Instance 844a0200d7876345:20bb38b90000007d (host=vc1513.foo.com:22000):(Total: 256.452ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:29/7.14 GB 0:26/6.44 GB 4:31/7.53 GB 1:23/5.57 GB 3:20/4.69 GB 6:30/7.29 GB 7:38/9.13 GB 5:29/7.09 GB Fragment Instance Lifecycle Event Timeline: 2s621ms - Prepare Finished: 232.917ms (232.917ms) - Open Finished: 256.902ms (23.985ms) - First Batch Produced: 2s619ms (2s362ms) - First Batch Sent: 2s621ms (2.058ms) MemoryUsage(8m32s): 13.11 GB, 13.98 GB, 13.96 GB, 13.87 GB, 13.65 GB, 13.34 GB, 13.26 GB, 13.11 GB, 12.78 GB, 12.02 GB, 11.44 GB, 10.91 GB, 10.49 GB, 10.35 GB, 10.21 GB, 9.97 GB, 9.75 GB, 9.27 GB, 9.01 GB, 8.83 GB, 8.57 GB, 8.29 GB, 8.22 GB, 8.43 GB, 9.06 GB, 10.07 GB, 11.93 GB, 13.61 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB, 13.88 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.98 GB (15007337995) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.65 GB (172491275414) - RowsProduced: 605.91M (605913607) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 31m49s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 23.980ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 23.996ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 17s056ms - OpenTime: 23.982ms - ExecTreeOpenTime: 138.549us - PrepareTime: 232.394ms - ExecTreePrepareTime: 2.010ms KrpcDataStreamSender (dst_id=1):(Total: 10m57s, non-child: 10m57s, % non-child: 100.00%) - BytesSent: 79.53 GB (85393160190) - EosSent: 0 (0) - OverallThroughput: 123.88 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 605.91M (605912583) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 10m56s - UncompressedRowBatchSize: 156.19 GB (167706512881) HDFS_SCAN_NODE (id=0):(Total: 16s535ms, non-child: 16s535ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:29/7.14 GB 0:26/6.44 GB 4:31/7.53 GB 1:23/5.57 GB 3:20/4.69 GB 6:30/7.29 GB 7:38/9.13 GB 5:29/7.09 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.09 GB, 16.72 GB, 17.36 GB, 17.99 GB, 19.23 GB, 19.86 GB, 20.19 GB, 20.50 GB, 20.62 GB, 20.87 GB, 21.12 GB, 21.18 GB, 21.28 GB, 21.45 GB, 21.60 GB, 21.73 GB, 22.14 GB, 22.52 GB, 22.76 GB, 22.83 GB, 22.99 GB, 23.23 GB, 23.79 GB, 24.61 GB, 25.78 GB, 27.57 GB, 30.92 GB, 33.85 GB, 34.37 GB, 34.48 GB, 34.55 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB - FooterProcessingTime: (Avg: 2s506ms ; Min: 26.655ms ; Max: 15s504ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.56 GB (37104151824) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m3s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (15004241784) - PerReadThreadRawHdfsThroughput: 48.27 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 606.13M (606126599) - RowsReturned: 605.91M (605913607) - RowsReturnedRate: 36.64 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m31s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 26s322ms - TotalRawHdfsReadTime(*): 12m13s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 23.625ms, non-child: 23.625ms, % non-child: 100.00%) - CodegenTime: 339.124us - CompileTime: 4.064ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.219ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 12.743ms Instance 844a0200d7876345:20bb38b900000054 (host=vd1102.foo.com:22000):(Total: 256.445ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 3:32/7.84 GB 5:32/7.82 GB 2:39/9.54 GB 0:22/5.33 GB 7:23/5.50 GB 1:23/5.50 GB 4:29/6.91 GB 6:26/6.25 GB Fragment Instance Lifecycle Event Timeline: 2s063ms - Prepare Finished: 215.870ms (215.870ms) - Open Finished: 256.874ms (41.003ms) - First Batch Produced: 2s061ms (1s804ms) - First Batch Sent: 2s063ms (1.725ms) MemoryUsage(8m32s): 13.11 GB, 13.94 GB, 13.94 GB, 13.93 GB, 13.76 GB, 13.44 GB, 13.21 GB, 13.14 GB, 12.92 GB, 12.45 GB, 11.71 GB, 11.20 GB, 10.75 GB, 10.39 GB, 10.23 GB, 9.99 GB, 9.66 GB, 9.28 GB, 8.84 GB, 8.52 GB, 8.63 GB, 8.57 GB, 8.43 GB, 8.46 GB, 8.84 GB, 9.57 GB, 10.65 GB, 11.89 GB, 13.33 GB, 13.81 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.95 GB (14973342651) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.55 GB (172384177811) - RowsProduced: 593.94M (593938291) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 30m33s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 40.996ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 36.994ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s885ms - OpenTime: 40.997ms - ExecTreeOpenTime: 209.714us - PrepareTime: 215.376ms - ExecTreePrepareTime: 2.025ms KrpcDataStreamSender (dst_id=1):(Total: 11m22s, non-child: 11m22s, % non-child: 100.00%) - BytesSent: 77.97 GB (83723280228) - EosSent: 0 (0) - OverallThroughput: 116.98 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 593.94M (593937267) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m21s - UncompressedRowBatchSize: 153.10 GB (164392293576) HDFS_SCAN_NODE (id=0):(Total: 8s367ms, non-child: 8s367ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:32/7.84 GB 5:32/7.82 GB 2:39/9.54 GB 0:22/5.33 GB 7:23/5.50 GB 1:23/5.50 GB 4:29/6.91 GB 6:26/6.25 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.83 GB, 16.37 GB, 17.11 GB, 17.73 GB, 18.70 GB, 19.60 GB, 19.92 GB, 20.23 GB, 20.45 GB, 20.61 GB, 20.84 GB, 21.01 GB, 21.09 GB, 21.20 GB, 21.32 GB, 21.37 GB, 21.37 GB, 21.59 GB, 21.87 GB, 22.12 GB, 22.67 GB, 23.01 GB, 23.42 GB, 23.99 GB, 25.15 GB, 26.56 GB, 28.52 GB, 30.69 GB, 33.25 GB, 34.13 GB, 34.20 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB, 34.21 GB - FooterProcessingTime: (Avg: 2s337ms ; Min: 26.765ms ; Max: 9s961ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.21 GB (36728890686) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m29s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.94 GB (14970231500) - PerReadThreadRawHdfsThroughput: 46.62 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 594.15M (594151283) - RowsReturned: 593.94M (593938291) - RowsReturnedRate: 70.98 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m52s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 25s352ms - TotalRawHdfsReadTime(*): 12m31s - TotalReadThroughput: 1.21 MB/sec CodeGen:(Total: 40.395ms, non-child: 40.395ms, % non-child: 100.00%) - CodegenTime: 491.462us - CompileTime: 4.796ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 9.017ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 25.979ms Instance 844a0200d7876345:20bb38b900000038 (host=vd1108.foo.com:22000):(Total: 256.305ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:25/6.07 GB 6:37/9.01 GB 3:34/8.30 GB 1:26/6.28 GB 4:20/4.87 GB 5:33/8.04 GB 0:23/5.66 GB 2:25/6.11 GB Fragment Instance Lifecycle Event Timeline: 2s010ms - Prepare Finished: 225.087ms (225.087ms) - Open Finished: 256.727ms (31.639ms) - First Batch Produced: 2s008ms (1s752ms) - First Batch Sent: 2s010ms (1.924ms) MemoryUsage(8m32s): 13.22 GB, 13.97 GB, 13.96 GB, 13.91 GB, 13.75 GB, 13.42 GB, 13.27 GB, 13.16 GB, 12.93 GB, 12.33 GB, 11.59 GB, 11.16 GB, 10.62 GB, 10.40 GB, 10.35 GB, 10.15 GB, 9.72 GB, 9.09 GB, 8.82 GB, 8.79 GB, 8.68 GB, 8.42 GB, 8.23 GB, 8.23 GB, 8.53 GB, 9.46 GB, 10.77 GB, 12.84 GB, 13.91 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (15002805695) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.68 GB (172532104702) - RowsProduced: 599.16M (599158613) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 27m4s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 31.623ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 29.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 14s431ms - OpenTime: 31.624ms - ExecTreeOpenTime: 153.125us - PrepareTime: 224.595ms - ExecTreePrepareTime: 1.961ms KrpcDataStreamSender (dst_id=1):(Total: 11m8s, non-child: 11m8s, % non-child: 100.00%) - BytesSent: 78.60 GB (84397827263) - EosSent: 0 (0) - OverallThroughput: 120.35 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 599.16M (599157589) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m7s - UncompressedRowBatchSize: 154.45 GB (165836920304) HDFS_SCAN_NODE (id=0):(Total: 13s903ms, non-child: 13s903ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:25/6.07 GB 6:37/9.01 GB 3:34/8.30 GB 1:26/6.28 GB 4:20/4.87 GB 5:33/8.04 GB 0:23/5.66 GB 2:25/6.11 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.12 GB, 16.65 GB, 17.30 GB, 17.87 GB, 19.04 GB, 19.78 GB, 20.09 GB, 20.45 GB, 20.56 GB, 20.76 GB, 21.07 GB, 21.15 GB, 21.22 GB, 21.40 GB, 21.65 GB, 21.66 GB, 21.66 GB, 21.80 GB, 22.31 GB, 22.59 GB, 22.85 GB, 23.09 GB, 23.40 GB, 24.10 GB, 24.88 GB, 26.54 GB, 28.85 GB, 32.44 GB, 34.27 GB, 34.43 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB - FooterProcessingTime: (Avg: 2s094ms ; Min: 28.982ms ; Max: 14s882ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.49 GB (37036634671) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m29s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (14999739037) - PerReadThreadRawHdfsThroughput: 49.19 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 599.37M (599371605) - RowsReturned: 599.16M (599158613) - RowsReturnedRate: 43.10 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m59s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 23s819ms - TotalRawHdfsReadTime(*): 11m58s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 31.100ms, non-child: 31.100ms, % non-child: 100.00%) - CodegenTime: 479.920us - CompileTime: 4.316ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.682ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 18.556ms Instance 844a0200d7876345:20bb38b90000004b (host=ve1141.foo.com:22000):(Total: 254.997ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:26/6.33 GB 1:30/7.43 GB 6:24/5.80 GB 0:32/7.80 GB 2:32/7.77 GB 4:20/4.85 GB 5:28/6.82 GB 3:32/7.88 GB Fragment Instance Lifecycle Event Timeline: 1s541ms - Prepare Finished: 219.581ms (219.581ms) - Open Finished: 255.700ms (36.118ms) - First Batch Produced: 1s539ms (1s283ms) - First Batch Sent: 1s541ms (2.313ms) MemoryUsage(8m32s): 12.03 GB, 12.71 GB, 12.68 GB, 12.58 GB, 12.31 GB, 12.12 GB, 11.93 GB, 11.52 GB, 10.80 GB, 10.26 GB, 9.76 GB, 9.51 GB, 9.30 GB, 9.02 GB, 8.27 GB, 7.74 GB, 7.59 GB, 7.66 GB, 7.44 GB, 7.81 GB, 8.42 GB, 9.80 GB, 11.56 GB, 12.45 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.72 GB (13653614640) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.46 GB (171222448026) - RowsProduced: 597.75M (597745610) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 24m47s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 36.119ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s262ms - OpenTime: 36.120ms - ExecTreeOpenTime: 191.255us - PrepareTime: 218.789ms - ExecTreePrepareTime: 1.849ms KrpcDataStreamSender (dst_id=1):(Total: 11m29s, non-child: 11m29s, % non-child: 100.00%) - BytesSent: 78.43 GB (84211139971) - EosSent: 0 (0) - OverallThroughput: 116.50 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 597.74M (597744586) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m27s - UncompressedRowBatchSize: 154.08 GB (165445898516) HDFS_SCAN_NODE (id=0):(Total: 8s748ms, non-child: 8s748ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:26/6.33 GB 1:30/7.43 GB 6:24/5.80 GB 0:32/7.80 GB 2:32/7.77 GB 4:20/4.85 GB 5:28/6.82 GB 3:32/7.88 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.49 GB, 15.45 GB, 16.11 GB, 17.09 GB, 18.03 GB, 18.42 GB, 18.71 GB, 18.86 GB, 19.15 GB, 19.31 GB, 19.40 GB, 19.56 GB, 19.67 GB, 19.78 GB, 19.78 GB, 19.89 GB, 20.24 GB, 20.96 GB, 21.38 GB, 22.50 GB, 23.74 GB, 26.24 GB, 29.26 GB, 30.83 GB, 31.32 GB, 31.43 GB, 31.68 GB, 31.95 GB, 32.15 GB, 32.38 GB, 32.54 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB, 32.55 GB - FooterProcessingTime: (Avg: 2s085ms ; Min: 13.853ms ; Max: 13s260ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.55 GB (34955010700) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m46s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.71 GB (13650511068) - PerReadThreadRawHdfsThroughput: 47.74 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 597.95M (597950410) - RowsReturned: 597.75M (597745610) - RowsReturnedRate: 68.33 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m59s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 21s061ms - TotalRawHdfsReadTime(*): 11m38s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 35.623ms, non-child: 35.623ms, % non-child: 100.00%) - CodegenTime: 420.655us - CompileTime: 4.436ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.172ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.332ms Instance 844a0200d7876345:20bb38b900000075 (host=vd1119.foo.com:22000):(Total: 254.661ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:25/6.15 GB 2:21/5.16 GB 4:31/7.64 GB 0:28/6.68 GB 3:32/7.84 GB 7:35/8.52 GB 6:19/4.63 GB 1:34/8.26 GB Fragment Instance Lifecycle Event Timeline: 1s716ms - Prepare Finished: 222.966ms (222.966ms) - Open Finished: 255.361ms (32.394ms) - First Batch Produced: 1s714ms (1s459ms) - First Batch Sent: 1s716ms (1.823ms) MemoryUsage(8m32s): 13.05 GB, 13.93 GB, 13.92 GB, 13.82 GB, 13.62 GB, 13.30 GB, 13.05 GB, 12.91 GB, 12.66 GB, 12.25 GB, 11.51 GB, 11.05 GB, 10.56 GB, 10.16 GB, 9.93 GB, 9.74 GB, 9.71 GB, 9.59 GB, 9.09 GB, 9.03 GB, 9.25 GB, 9.25 GB, 9.14 GB, 9.15 GB, 9.28 GB, 9.40 GB, 10.07 GB, 11.53 GB, 13.14 GB, 13.83 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.95 GB (14984001646) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.72 GB (172571892017) - RowsProduced: 594.69M (594694493) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 32m39s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 32.388ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 28.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 6s928ms - OpenTime: 32.389ms - ExecTreeOpenTime: 131.165us - PrepareTime: 222.188ms - ExecTreePrepareTime: 2.122ms KrpcDataStreamSender (dst_id=1):(Total: 11m16s, non-child: 11m16s, % non-child: 100.00%) - BytesSent: 78.10 GB (83860983203) - EosSent: 0 (0) - OverallThroughput: 118.27 MB/sec - PeakMemoryUsage: 3.59 MB (3760800) - RowsReturned: 594.69M (594693469) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m14s - UncompressedRowBatchSize: 153.30 GB (164601772172) HDFS_SCAN_NODE (id=0):(Total: 6s428ms, non-child: 6s428ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:25/6.15 GB 2:21/5.16 GB 4:31/7.64 GB 0:28/6.68 GB 3:32/7.84 GB 7:35/8.52 GB 6:19/4.63 GB 1:34/8.26 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.95 GB, 16.41 GB, 17.18 GB, 17.81 GB, 18.63 GB, 19.54 GB, 19.85 GB, 20.18 GB, 20.40 GB, 20.50 GB, 20.76 GB, 20.93 GB, 20.96 GB, 21.05 GB, 21.19 GB, 21.34 GB, 21.65 GB, 22.10 GB, 22.37 GB, 22.99 GB, 23.62 GB, 23.98 GB, 24.34 GB, 25.01 GB, 25.71 GB, 26.15 GB, 27.44 GB, 30.03 GB, 32.89 GB, 34.17 GB, 34.45 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB - FooterProcessingTime: (Avg: 2s344ms ; Min: 30.178ms ; Max: 15s650ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.46 GB (36997649589) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m31s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.95 GB (14980967652) - PerReadThreadRawHdfsThroughput: 48.62 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 594.91M (594907485) - RowsReturned: 594.69M (594694493) - RowsReturnedRate: 92.51 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m37s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 24s595ms - TotalRawHdfsReadTime(*): 12m5s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 31.964ms, non-child: 31.964ms, % non-child: 100.00%) - CodegenTime: 408.948us - CompileTime: 3.980ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.458ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 21.061ms Instance 844a0200d7876345:20bb38b90000005a (host=vd1339.foo.com:22000):(Total: 254.400ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:19/4.70 GB 7:30/7.36 GB 1:34/8.39 GB 2:22/5.45 GB 0:40/9.79 GB 3:21/5.17 GB 6:32/7.63 GB 5:25/6.10 GB Fragment Instance Lifecycle Event Timeline: 1s608ms - Prepare Finished: 214.733ms (214.733ms) - Open Finished: 254.840ms (40.106ms) - First Batch Produced: 1s606ms (1s351ms) - First Batch Sent: 1s608ms (2.035ms) MemoryUsage(8m32s): 11.96 GB, 12.74 GB, 12.72 GB, 12.66 GB, 12.46 GB, 12.20 GB, 12.11 GB, 11.84 GB, 11.24 GB, 10.58 GB, 10.02 GB, 9.67 GB, 9.55 GB, 9.26 GB, 8.83 GB, 8.06 GB, 7.87 GB, 7.74 GB, 7.39 GB, 7.17 GB, 7.59 GB, 8.26 GB, 9.57 GB, 12.00 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.74 GB (13680348562) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.48 GB (171236853582) - RowsProduced: 589.57M (589574757) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 28m6s - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 40.103ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 38.994ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 12s883ms - OpenTime: 40.104ms - ExecTreeOpenTime: 263.587us - PrepareTime: 214.224ms - ExecTreePrepareTime: 2.114ms KrpcDataStreamSender (dst_id=1):(Total: 11m19s, non-child: 11m19s, % non-child: 100.00%) - BytesSent: 77.35 GB (83058356810) - EosSent: 0 (0) - OverallThroughput: 116.61 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 589.57M (589573733) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m17s - UncompressedRowBatchSize: 151.98 GB (163184176076) HDFS_SCAN_NODE (id=0):(Total: 12s327ms, non-child: 12s327ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:19/4.70 GB 7:30/7.36 GB 1:34/8.39 GB 2:22/5.45 GB 0:40/9.79 GB 3:21/5.17 GB 6:32/7.63 GB 5:25/6.10 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 11.93 GB, 15.29 GB, 15.94 GB, 16.68 GB, 17.91 GB, 18.34 GB, 18.71 GB, 18.88 GB, 19.10 GB, 19.39 GB, 19.46 GB, 19.58 GB, 19.76 GB, 19.78 GB, 19.82 GB, 19.98 GB, 20.29 GB, 20.59 GB, 20.75 GB, 21.17 GB, 22.39 GB, 23.61 GB, 25.92 GB, 30.13 GB, 31.21 GB, 31.34 GB, 31.46 GB, 31.63 GB, 31.90 GB, 32.20 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB - FooterProcessingTime: (Avg: 2s553ms ; Min: 17.308ms ; Max: 17s503ms ; Number of samples: 161) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.34 GB (34723297027) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m10s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 161 (161) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.74 GB (13677199889) - PerReadThreadRawHdfsThroughput: 44.74 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 589.78M (589779557) - RowsReturned: 589.57M (589574757) - RowsReturnedRate: 47.83 M/sec - ScanRangesComplete: 81 (81) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m40s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 20s879ms - TotalRawHdfsReadTime(*): 12m20s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 39.263ms, non-child: 39.263ms, % non-child: 100.00%) - CodegenTime: 744.391us - CompileTime: 5.198ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 9.444ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 23.732ms Instance 844a0200d7876345:20bb38b900000012 (host=vd1335.foo.com:22000):(Total: 254.166ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:16/3.93 GB 6:36/8.87 GB 3:29/7.06 GB 1:28/6.66 GB 2:25/5.95 GB 5:23/5.69 GB 0:34/8.30 GB 7:32/7.75 GB Fragment Instance Lifecycle Event Timeline: 1s569ms - Prepare Finished: 218.846ms (218.846ms) - Open Finished: 254.969ms (36.123ms) - First Batch Produced: 1s567ms (1s312ms) - First Batch Sent: 1s569ms (1.559ms) MemoryUsage(8m32s): 11.97 GB, 12.73 GB, 12.67 GB, 12.54 GB, 12.31 GB, 12.09 GB, 11.87 GB, 11.52 GB, 10.86 GB, 10.26 GB, 9.78 GB, 9.55 GB, 9.32 GB, 9.04 GB, 8.60 GB, 8.07 GB, 7.86 GB, 7.61 GB, 7.42 GB, 7.90 GB, 8.45 GB, 9.21 GB, 10.61 GB, 11.84 GB, 12.59 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.73 GB (13672275855) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.06 GB (186898093888) - RowsProduced: 594.37M (594372494) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h13m - TotalStorageWaitTime: 28m33s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 36.120ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 35.995ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 13s646ms - OpenTime: 36.121ms - ExecTreeOpenTime: 163.722us - PrepareTime: 217.953ms - ExecTreePrepareTime: 1.929ms KrpcDataStreamSender (dst_id=1):(Total: 11m29s, non-child: 11m29s, % non-child: 100.00%) - BytesSent: 78.05 GB (83810779005) - EosSent: 0 (0) - OverallThroughput: 115.91 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 594.37M (594371470) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m28s - UncompressedRowBatchSize: 153.21 GB (164512030918) HDFS_SCAN_NODE (id=0):(Total: 13s096ms, non-child: 13s096ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:16/3.93 GB 6:36/8.87 GB 3:29/7.06 GB 1:28/6.66 GB 2:25/5.95 GB 5:23/5.69 GB 0:34/8.30 GB 7:32/7.75 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.32 GB, 15.42 GB, 16.06 GB, 16.95 GB, 17.96 GB, 18.38 GB, 18.67 GB, 18.85 GB, 19.06 GB, 19.29 GB, 19.38 GB, 19.59 GB, 19.81 GB, 19.96 GB, 20.13 GB, 20.27 GB, 20.64 GB, 20.77 GB, 21.13 GB, 22.55 GB, 23.88 GB, 25.30 GB, 27.75 GB, 29.93 GB, 31.28 GB, 31.47 GB, 31.61 GB, 31.77 GB, 31.95 GB, 32.29 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB - FooterProcessingTime: (Avg: 2s313ms ; Min: 30.063ms ; Max: 10s140ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.50 GB (34897001619) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 11m46s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.73 GB (13669209072) - PerReadThreadRawHdfsThroughput: 46.52 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 594.58M (594577294) - RowsReturned: 594.37M (594372494) - RowsReturnedRate: 45.38 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m23s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 21s339ms - TotalRawHdfsReadTime(*): 11m55s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 35.652ms, non-child: 35.652ms, % non-child: 100.00%) - CodegenTime: 419.546us - CompileTime: 4.275ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.883ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.998ms Instance 844a0200d7876345:20bb38b90000002c (host=vd1327.foo.com:22000):(Total: 253.896ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 3:26/6.31 GB 7:27/6.65 GB 0:37/9.08 GB 2:28/6.53 GB 5:25/6.12 GB 1:26/6.36 GB 6:33/7.88 GB 4:25/5.96 GB Fragment Instance Lifecycle Event Timeline: 2s026ms - Prepare Finished: 225.332ms (225.332ms) - Open Finished: 254.677ms (29.345ms) - First Batch Produced: 2s024ms (1s769ms) - First Batch Sent: 2s026ms (2.037ms) MemoryUsage(8m32s): 12.03 GB, 12.62 GB, 12.60 GB, 12.51 GB, 12.23 GB, 12.08 GB, 11.89 GB, 11.39 GB, 10.62 GB, 10.13 GB, 9.63 GB, 9.46 GB, 9.25 GB, 8.79 GB, 8.12 GB, 7.88 GB, 7.89 GB, 7.68 GB, 7.56 GB, 7.89 GB, 8.82 GB, 10.27 GB, 12.20 GB, 12.63 GB, 12.63 GB, 12.63 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB, 12.62 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.65 GB (13580731498) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.03 GB (186867349859) - RowsProduced: 600.79M (600789374) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 23m18s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 29.341ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 25.996ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s422ms - OpenTime: 29.342ms - ExecTreeOpenTime: 173.718us - PrepareTime: 224.436ms - ExecTreePrepareTime: 1.910ms KrpcDataStreamSender (dst_id=1):(Total: 11m26s, non-child: 11m26s, % non-child: 100.00%) - BytesSent: 78.83 GB (84648334519) - EosSent: 0 (0) - OverallThroughput: 117.52 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 600.79M (600788350) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m25s - UncompressedRowBatchSize: 154.87 GB (166288436955) HDFS_SCAN_NODE (id=0):(Total: 8s903ms, non-child: 8s903ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:26/6.31 GB 7:27/6.65 GB 0:37/9.08 GB 2:28/6.53 GB 5:25/6.12 GB 1:26/6.36 GB 6:33/7.88 GB 4:25/5.96 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.58 GB, 15.41 GB, 16.07 GB, 17.28 GB, 18.12 GB, 18.51 GB, 18.74 GB, 18.91 GB, 19.25 GB, 19.35 GB, 19.46 GB, 19.64 GB, 19.67 GB, 19.67 GB, 19.90 GB, 20.28 GB, 20.77 GB, 21.19 GB, 21.71 GB, 22.74 GB, 24.39 GB, 27.00 GB, 30.37 GB, 31.16 GB, 31.27 GB, 31.43 GB, 31.64 GB, 31.94 GB, 32.13 GB, 32.33 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB - FooterProcessingTime: (Avg: 2s787ms ; Min: 28.942ms ; Max: 10s859ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.50 GB (34901146716) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m48s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.65 GB (13577631784) - PerReadThreadRawHdfsThroughput: 52.67 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 600.99M (600994174) - RowsReturned: 600.79M (600789374) - RowsReturnedRate: 67.48 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m24s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 23s018ms - TotalRawHdfsReadTime(*): 10m31s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 28.876ms, non-child: 28.876ms, % non-child: 100.00%) - CodegenTime: 409.547us - CompileTime: 4.573ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.042ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 16.734ms Instance 844a0200d7876345:20bb38b900000013 (host=vc1502.foo.com:22000):(Total: 253.628ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:27/6.59 GB 3:26/6.36 GB 6:23/5.62 GB 7:33/8.13 GB 4:27/6.64 GB 1:25/6.14 GB 5:28/6.89 GB 0:35/8.56 GB Fragment Instance Lifecycle Event Timeline: 1s805ms - Prepare Finished: 226.633ms (226.633ms) - Open Finished: 254.051ms (27.417ms) - First Batch Produced: 1s803ms (1s549ms) - First Batch Sent: 1s805ms (1.977ms) MemoryUsage(8m32s): 13.29 GB, 14.00 GB, 14.00 GB, 13.96 GB, 13.78 GB, 13.50 GB, 13.40 GB, 13.28 GB, 12.98 GB, 12.29 GB, 11.67 GB, 11.22 GB, 10.76 GB, 10.56 GB, 10.42 GB, 10.14 GB, 9.73 GB, 9.08 GB, 8.60 GB, 8.44 GB, 8.19 GB, 8.00 GB, 7.91 GB, 8.08 GB, 8.76 GB, 9.51 GB, 11.08 GB, 12.95 GB, 13.89 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 14.00 GB (15037322287) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.37 GB (188304511172) - RowsProduced: 600.66M (600656031) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 25m57s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 27.414ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 22.997ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 11s870ms - OpenTime: 27.415ms - ExecTreeOpenTime: 141.441us - PrepareTime: 226.126ms - ExecTreePrepareTime: 1.875ms KrpcDataStreamSender (dst_id=1):(Total: 11m8s, non-child: 11m8s, % non-child: 100.00%) - BytesSent: 78.83 GB (84643016917) - EosSent: 0 (0) - OverallThroughput: 120.75 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 600.66M (600655007) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m7s - UncompressedRowBatchSize: 154.83 GB (166250768365) HDFS_SCAN_NODE (id=0):(Total: 11s359ms, non-child: 11s359ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:27/6.59 GB 3:26/6.36 GB 6:23/5.62 GB 7:33/8.13 GB 4:27/6.64 GB 1:25/6.14 GB 5:28/6.89 GB 0:35/8.56 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.37 GB, 16.78 GB, 17.38 GB, 18.03 GB, 19.28 GB, 19.94 GB, 20.26 GB, 20.59 GB, 20.72 GB, 20.95 GB, 21.22 GB, 21.30 GB, 21.39 GB, 21.56 GB, 21.67 GB, 21.67 GB, 21.73 GB, 21.86 GB, 21.96 GB, 22.12 GB, 22.24 GB, 22.63 GB, 23.15 GB, 23.98 GB, 25.30 GB, 26.65 GB, 29.40 GB, 32.68 GB, 34.30 GB, 34.47 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB, 34.53 GB - FooterProcessingTime: (Avg: 2s438ms ; Min: 29.796ms ; Max: 11s678ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.53 GB (37079659490) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m25s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 14.00 GB (15034126468) - PerReadThreadRawHdfsThroughput: 52.43 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 600.87M (600869023) - RowsReturned: 600.66M (600656031) - RowsReturnedRate: 52.88 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m14s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 20s183ms - TotalRawHdfsReadTime(*): 11m14s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 27.009ms, non-child: 27.009ms, % non-child: 100.00%) - CodegenTime: 394.414us - CompileTime: 4.112ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.738ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 15.566ms Instance 844a0200d7876345:20bb38b90000006c (host=ve1130.foo.com:22000):(Total: 253.399ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:26/6.28 GB 0:33/8.05 GB 6:31/7.60 GB 2:28/6.81 GB 1:31/7.43 GB 5:27/6.61 GB 3:26/6.44 GB 7:23/5.54 GB Fragment Instance Lifecycle Event Timeline: 1s802ms - Prepare Finished: 215.839ms (215.839ms) - Open Finished: 254.108ms (38.268ms) - First Batch Produced: 1s799ms (1s545ms) - First Batch Sent: 1s802ms (2.632ms) MemoryUsage(8m32s): 12.05 GB, 12.66 GB, 12.64 GB, 12.53 GB, 12.22 GB, 11.92 GB, 11.86 GB, 11.64 GB, 11.07 GB, 10.28 GB, 9.78 GB, 9.35 GB, 9.29 GB, 9.15 GB, 8.91 GB, 8.42 GB, 8.39 GB, 8.24 GB, 7.85 GB, 7.59 GB, 7.64 GB, 8.32 GB, 9.25 GB, 11.28 GB, 12.65 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.69 GB (13630266530) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.45 GB (171210071191) - RowsProduced: 582.43M (582434661) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 22m17s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 38.256ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 35.994ms - TotalThreadsVoluntaryContextSwitches: 6 (6) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s187ms - OpenTime: 38.257ms - ExecTreeOpenTime: 264.697us - PrepareTime: 215.047ms - ExecTreePrepareTime: 2.007ms KrpcDataStreamSender (dst_id=1):(Total: 11m34s, non-child: 11m34s, % non-child: 100.00%) - BytesSent: 76.45 GB (82083334909) - EosSent: 0 (0) - OverallThroughput: 112.68 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 582.43M (582433637) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m33s - UncompressedRowBatchSize: 150.14 GB (161206634853) HDFS_SCAN_NODE (id=0):(Total: 8s700ms, non-child: 8s700ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:26/6.28 GB 0:33/8.05 GB 6:31/7.60 GB 2:28/6.81 GB 1:31/7.43 GB 5:27/6.61 GB 3:26/6.44 GB 7:23/5.54 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 11.94 GB, 15.13 GB, 15.69 GB, 16.34 GB, 17.53 GB, 17.97 GB, 18.32 GB, 18.49 GB, 18.65 GB, 18.94 GB, 19.02 GB, 19.15 GB, 19.43 GB, 19.57 GB, 19.76 GB, 20.19 GB, 20.91 GB, 21.06 GB, 21.07 GB, 21.33 GB, 22.14 GB, 23.42 GB, 25.11 GB, 28.66 GB, 31.02 GB, 31.25 GB, 31.42 GB, 31.55 GB, 31.73 GB, 31.94 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.00 GB, 32.01 GB, 32.01 GB, 32.01 GB, 32.01 GB, 32.01 GB, 32.01 GB, 32.01 GB, 32.01 GB - FooterProcessingTime: (Avg: 1s980ms ; Min: 29.718ms ; Max: 10s549ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.01 GB (34366450428) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m45s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.69 GB (13627085468) - PerReadThreadRawHdfsThroughput: 53.29 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 582.64M (582639461) - RowsReturned: 582.43M (582434661) - RowsReturnedRate: 66.94 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m53s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 20s318ms - TotalRawHdfsReadTime(*): 10m15s - TotalReadThroughput: 1.13 MB/sec CodeGen:(Total: 37.583ms, non-child: 37.583ms, % non-child: 100.00%) - CodegenTime: 589.773us - CompileTime: 4.564ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.790ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 24.580ms Instance 844a0200d7876345:20bb38b900000004 (host=vd1112.foo.com:22000):(Total: 253.382ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:26/6.36 GB 4:26/6.39 GB 6:24/5.83 GB 3:37/8.88 GB 7:20/4.88 GB 5:33/8.09 GB 1:26/6.37 GB 2:31/7.59 GB Fragment Instance Lifecycle Event Timeline: 2s360ms - Prepare Finished: 218.130ms (218.130ms) - Open Finished: 253.887ms (35.757ms) - First Batch Produced: 2s359ms (2s105ms) - First Batch Sent: 2s360ms (1.194ms) MemoryUsage(8m32s): 13.06 GB, 13.98 GB, 13.98 GB, 13.94 GB, 13.78 GB, 13.55 GB, 13.34 GB, 13.24 GB, 13.03 GB, 12.55 GB, 11.87 GB, 11.35 GB, 10.93 GB, 10.52 GB, 10.33 GB, 10.17 GB, 9.92 GB, 9.58 GB, 9.00 GB, 8.67 GB, 8.56 GB, 8.41 GB, 8.32 GB, 8.48 GB, 8.57 GB, 9.43 GB, 10.44 GB, 11.52 GB, 13.41 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.98 GB (15012964631) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.39 GB (188320595207) - RowsProduced: 595.50M (595498741) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 34m35s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 35.748ms - TotalThreadsSysTime: 999.000us - TotalThreadsUserTime: 33.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 11s420ms - OpenTime: 35.749ms - ExecTreeOpenTime: 280.241us - PrepareTime: 217.574ms - ExecTreePrepareTime: 1.838ms KrpcDataStreamSender (dst_id=1):(Total: 11m4s, non-child: 11m4s, % non-child: 100.00%) - BytesSent: 78.19 GB (83954461002) - EosSent: 0 (0) - OverallThroughput: 120.53 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 595.50M (595497717) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m3s - UncompressedRowBatchSize: 153.50 GB (164823283962) HDFS_SCAN_NODE (id=0):(Total: 10s888ms, non-child: 10s888ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:26/6.36 GB 4:26/6.39 GB 6:24/5.83 GB 3:37/8.88 GB 7:20/4.88 GB 5:33/8.09 GB 1:26/6.37 GB 2:31/7.59 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.78 GB, 16.33 GB, 17.14 GB, 17.81 GB, 18.71 GB, 19.68 GB, 20.03 GB, 20.33 GB, 20.57 GB, 20.73 GB, 20.95 GB, 21.17 GB, 21.23 GB, 21.33 GB, 21.47 GB, 21.60 GB, 21.72 GB, 21.93 GB, 22.00 GB, 22.33 GB, 22.49 GB, 22.77 GB, 23.28 GB, 24.08 GB, 24.79 GB, 26.39 GB, 28.17 GB, 30.14 GB, 33.46 GB, 34.43 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB - FooterProcessingTime: (Avg: 3s575ms ; Min: 35.770ms ; Max: 13s931ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.48 GB (37020983891) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m35s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.98 GB (15009881512) - PerReadThreadRawHdfsThroughput: 44.43 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 595.71M (595711733) - RowsReturned: 595.50M (595498741) - RowsReturnedRate: 54.69 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m37s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 26s626ms - TotalRawHdfsReadTime(*): 13m14s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 35.163ms, non-child: 35.163ms, % non-child: 100.00%) - CodegenTime: 428.210us - CompileTime: 4.760ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.521ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 21.276ms Instance 844a0200d7876345:20bb38b90000004f (host=vd1342.foo.com:22000):(Total: 253.239ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 6:19/4.63 GB 4:22/5.37 GB 5:28/6.81 GB 2:30/7.39 GB 1:29/7.02 GB 7:30/7.34 GB 0:33/8.02 GB 3:33/8.08 GB Fragment Instance Lifecycle Event Timeline: 1s467ms - Prepare Finished: 213.991ms (213.991ms) - Open Finished: 253.811ms (39.819ms) - First Batch Produced: 1s465ms (1s212ms) - First Batch Sent: 1s467ms (1.989ms) MemoryUsage(8m32s): 12.17 GB, 12.68 GB, 12.66 GB, 12.48 GB, 12.06 GB, 11.97 GB, 11.86 GB, 11.31 GB, 10.50 GB, 9.90 GB, 9.43 GB, 9.32 GB, 9.21 GB, 8.79 GB, 8.06 GB, 8.14 GB, 8.06 GB, 7.64 GB, 7.33 GB, 7.51 GB, 8.48 GB, 10.43 GB, 12.41 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.71 GB (13642351627) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.47 GB (171226604972) - RowsProduced: 597.55M (597552785) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 18m41s - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 39.816ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 36.994ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s335ms - OpenTime: 39.817ms - ExecTreeOpenTime: 208.241us - PrepareTime: 213.350ms - ExecTreePrepareTime: 1.902ms KrpcDataStreamSender (dst_id=1):(Total: 11m23s, non-child: 11m23s, % non-child: 100.00%) - BytesSent: 78.44 GB (84222667991) - EosSent: 0 (0) - OverallThroughput: 117.43 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 597.55M (597551761) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m22s - UncompressedRowBatchSize: 154.03 GB (165392642420) HDFS_SCAN_NODE (id=0):(Total: 7s769ms, non-child: 7s769ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:19/4.63 GB 4:22/5.37 GB 5:28/6.81 GB 2:30/7.39 GB 1:29/7.02 GB 7:30/7.34 GB 0:33/8.02 GB 3:33/8.08 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.50 GB, 15.41 GB, 15.95 GB, 17.12 GB, 17.91 GB, 18.33 GB, 18.53 GB, 18.69 GB, 19.02 GB, 19.09 GB, 19.21 GB, 19.40 GB, 19.48 GB, 19.57 GB, 19.86 GB, 20.56 GB, 20.88 GB, 21.01 GB, 21.34 GB, 22.01 GB, 23.78 GB, 27.22 GB, 30.55 GB, 31.10 GB, 31.26 GB, 31.45 GB, 31.68 GB, 31.91 GB, 32.02 GB, 32.17 GB, 32.33 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB, 32.34 GB - FooterProcessingTime: (Avg: 1s493ms ; Min: 14.427ms ; Max: 7s257ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.34 GB (34722529113) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m54s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.70 GB (13639276648) - PerReadThreadRawHdfsThroughput: 58.91 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 597.76M (597757585) - RowsReturned: 597.55M (597552785) - RowsReturnedRate: 76.91 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m52s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 19s381ms - TotalRawHdfsReadTime(*): 9m22s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 39.266ms, non-child: 39.266ms, % non-child: 100.00%) - CodegenTime: 427.236us - CompileTime: 4.716ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 9.455ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 24.471ms Instance 844a0200d7876345:20bb38b90000000a (host=vd1132.foo.com:22000):(Total: 253.109ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:36/8.82 GB 4:29/7.04 GB 6:23/5.66 GB 3:31/7.47 GB 1:34/8.38 GB 2:18/4.35 GB 7:26/6.36 GB 0:27/6.52 GB Fragment Instance Lifecycle Event Timeline: 1s755ms - Prepare Finished: 217.487ms (217.487ms) - Open Finished: 253.444ms (35.957ms) - First Batch Produced: 1s754ms (1s500ms) - First Batch Sent: 1s755ms (1.827ms) MemoryUsage(8m32s): 12.07 GB, 12.69 GB, 12.68 GB, 12.58 GB, 12.27 GB, 11.99 GB, 11.94 GB, 11.74 GB, 11.13 GB, 10.41 GB, 9.86 GB, 9.44 GB, 9.31 GB, 9.14 GB, 8.74 GB, 8.35 GB, 8.12 GB, 8.05 GB, 7.84 GB, 7.56 GB, 7.44 GB, 7.82 GB, 8.96 GB, 10.69 GB, 12.35 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.69 GB (13626925077) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.09 GB (186932769177) - RowsProduced: 579.03M (579032342) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h10m - TotalStorageWaitTime: 22m39s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 35.953ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 33.995ms - TotalThreadsVoluntaryContextSwitches: 5 (5) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 18s722ms - OpenTime: 35.955ms - ExecTreeOpenTime: 299.750us - PrepareTime: 217.085ms - ExecTreePrepareTime: 1.854ms KrpcDataStreamSender (dst_id=1):(Total: 11m21s, non-child: 11m21s, % non-child: 100.00%) - BytesSent: 76.05 GB (81655799237) - EosSent: 0 (0) - OverallThroughput: 114.20 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 579.03M (579031318) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m20s - UncompressedRowBatchSize: 149.26 GB (160265987997) HDFS_SCAN_NODE (id=0):(Total: 18s189ms, non-child: 18s189ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:36/8.82 GB 4:29/7.04 GB 6:23/5.66 GB 3:31/7.47 GB 1:34/8.38 GB 2:18/4.35 GB 7:26/6.36 GB 0:27/6.52 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 11.95 GB, 15.17 GB, 15.75 GB, 16.42 GB, 17.61 GB, 18.07 GB, 18.43 GB, 18.58 GB, 18.78 GB, 19.06 GB, 19.13 GB, 19.24 GB, 19.40 GB, 19.45 GB, 19.49 GB, 20.01 GB, 20.50 GB, 20.75 GB, 21.00 GB, 21.29 GB, 21.73 GB, 22.65 GB, 24.65 GB, 27.65 GB, 30.56 GB, 31.20 GB, 31.33 GB, 31.49 GB, 31.69 GB, 31.92 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB, 32.04 GB - FooterProcessingTime: (Avg: 1s764ms ; Min: 17.733ms ; Max: 9s620ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.04 GB (34399536175) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 16m32s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.69 GB (13623833739) - PerReadThreadRawHdfsThroughput: 53.13 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 579.24M (579237142) - RowsReturned: 579.03M (579032342) - RowsReturnedRate: 31.83 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 11m11s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 14s783ms - TotalRawHdfsReadTime(*): 10m17s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 35.391ms, non-child: 35.391ms, % non-child: 100.00%) - CodegenTime: 413.847us - CompileTime: 4.559ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.806ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.434ms Instance 844a0200d7876345:20bb38b900000007 (host=vd1105.foo.com:22000):(Total: 252.526ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:28/6.82 GB 2:25/6.10 GB 5:23/5.61 GB 3:40/9.80 GB 7:29/7.10 GB 6:26/6.39 GB 1:27/6.54 GB 4:27/6.17 GB Fragment Instance Lifecycle Event Timeline: 1s520ms - Prepare Finished: 217.595ms (217.595ms) - Open Finished: 253.143ms (35.547ms) - First Batch Produced: 1s518ms (1s265ms) - First Batch Sent: 1s520ms (1.675ms) MemoryUsage(8m32s): 13.24 GB, 13.97 GB, 13.96 GB, 13.89 GB, 13.71 GB, 13.41 GB, 13.24 GB, 13.16 GB, 12.88 GB, 12.31 GB, 11.61 GB, 11.13 GB, 10.63 GB, 10.38 GB, 10.31 GB, 10.25 GB, 9.98 GB, 9.53 GB, 9.11 GB, 8.90 GB, 8.75 GB, 8.41 GB, 8.09 GB, 8.08 GB, 8.59 GB, 9.37 GB, 10.64 GB, 12.39 GB, 13.47 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (15002331590) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.28 GB (188210601299) - RowsProduced: 597.15M (597148967) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 26m59s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 35.544ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 34.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 13s119ms - OpenTime: 35.545ms - ExecTreeOpenTime: 195.204us - PrepareTime: 216.881ms - ExecTreePrepareTime: 1.997ms KrpcDataStreamSender (dst_id=1):(Total: 11m11s, non-child: 11m11s, % non-child: 100.00%) - BytesSent: 78.40 GB (84176728956) - EosSent: 0 (0) - OverallThroughput: 119.51 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 597.15M (597147943) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m10s - UncompressedRowBatchSize: 153.93 GB (165281007747) HDFS_SCAN_NODE (id=0):(Total: 12s634ms, non-child: 12s634ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:28/6.82 GB 2:25/6.10 GB 5:23/5.61 GB 3:40/9.80 GB 7:29/7.10 GB 6:26/6.39 GB 1:27/6.54 GB 4:27/6.17 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.07 GB, 16.51 GB, 17.22 GB, 17.80 GB, 18.90 GB, 19.72 GB, 20.06 GB, 20.37 GB, 20.53 GB, 20.72 GB, 20.98 GB, 21.10 GB, 21.18 GB, 21.33 GB, 21.54 GB, 21.89 GB, 22.02 GB, 22.28 GB, 22.45 GB, 22.67 GB, 22.91 GB, 22.94 GB, 23.07 GB, 23.67 GB, 24.89 GB, 26.40 GB, 28.73 GB, 31.75 GB, 33.63 GB, 34.37 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB, 34.42 GB - FooterProcessingTime: (Avg: 2s086ms ; Min: 28.548ms ; Max: 11s287ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.42 GB (36956592055) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m46s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (14999043659) - PerReadThreadRawHdfsThroughput: 53.99 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 597.36M (597361959) - RowsReturned: 597.15M (597148967) - RowsReturnedRate: 47.26 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m1s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 25s478ms - TotalRawHdfsReadTime(*): 10m52s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 35.064ms, non-child: 35.064ms, % non-child: 100.00%) - CodegenTime: 366.001us - CompileTime: 4.183ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.719ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.650ms Instance 844a0200d7876345:20bb38b900000065 (host=ve1126.foo.com:22000):(Total: 252.177ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:28/6.72 GB 4:20/4.83 GB 6:38/9.18 GB 3:21/5.16 GB 0:28/6.85 GB 1:32/7.77 GB 7:34/8.33 GB 5:24/5.84 GB Fragment Instance Lifecycle Event Timeline: 1s541ms - Prepare Finished: 221.167ms (221.167ms) - Open Finished: 252.659ms (31.491ms) - First Batch Produced: 1s538ms (1s286ms) - First Batch Sent: 1s541ms (2.350ms) MemoryUsage(8m32s): 12.06 GB, 12.61 GB, 12.56 GB, 12.37 GB, 12.03 GB, 11.95 GB, 11.92 GB, 11.35 GB, 10.69 GB, 10.06 GB, 9.68 GB, 9.54 GB, 9.40 GB, 8.86 GB, 8.32 GB, 8.08 GB, 7.82 GB, 7.56 GB, 7.30 GB, 7.78 GB, 9.14 GB, 11.03 GB, 12.05 GB, 12.63 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.65 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.67 GB (13606935211) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.41 GB (171160891997) - RowsProduced: 602.67M (602672944) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 21m34s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 31.488ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 28.995ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 6s613ms - OpenTime: 31.490ms - ExecTreeOpenTime: 185.063us - PrepareTime: 220.616ms - ExecTreePrepareTime: 2.013ms KrpcDataStreamSender (dst_id=1):(Total: 11m39s, non-child: 11m39s, % non-child: 100.00%) - BytesSent: 79.12 GB (84959540877) - EosSent: 0 (0) - OverallThroughput: 115.76 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 602.67M (602671920) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m38s - UncompressedRowBatchSize: 155.35 GB (166809572157) HDFS_SCAN_NODE (id=0):(Total: 6s064ms, non-child: 6s064ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:28/6.72 GB 4:20/4.83 GB 6:38/9.18 GB 3:21/5.16 GB 0:28/6.85 GB 1:32/7.77 GB 7:34/8.33 GB 5:24/5.84 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.61 GB, 15.39 GB, 15.94 GB, 17.07 GB, 17.85 GB, 18.36 GB, 18.75 GB, 18.98 GB, 19.25 GB, 19.36 GB, 19.51 GB, 19.68 GB, 19.86 GB, 19.94 GB, 20.19 GB, 20.46 GB, 20.74 GB, 21.09 GB, 21.36 GB, 22.49 GB, 25.00 GB, 28.36 GB, 30.10 GB, 31.16 GB, 31.33 GB, 31.52 GB, 31.76 GB, 31.92 GB, 32.05 GB, 32.29 GB, 32.59 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB, 32.60 GB - FooterProcessingTime: (Avg: 1s833ms ; Min: 35.468ms ; Max: 7s800ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.60 GB (35000852155) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 6m17s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.67 GB (13603835703) - PerReadThreadRawHdfsThroughput: 54.36 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 602.88M (602877744) - RowsReturned: 602.67M (602672944) - RowsReturnedRate: 99.37 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m27s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 19s798ms - TotalRawHdfsReadTime(*): 10m14s - TotalReadThroughput: 1.16 MB/sec CodeGen:(Total: 30.997ms, non-child: 30.997ms, % non-child: 100.00%) - CodegenTime: 416.573us - CompileTime: 4.377ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.971ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 19.140ms Instance 844a0200d7876345:20bb38b900000037 (host=vd1127.foo.com:22000):(Total: 252.165ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:30/7.31 GB 1:25/6.06 GB 7:29/7.06 GB 5:27/6.54 GB 3:28/6.84 GB 2:28/6.79 GB 4:25/6.08 GB 6:33/8.02 GB Fragment Instance Lifecycle Event Timeline: 1s874ms - Prepare Finished: 219.564ms (219.564ms) - Open Finished: 252.901ms (33.336ms) - First Batch Produced: 1s873ms (1s620ms) - First Batch Sent: 1s874ms (1.207ms) MemoryUsage(8m32s): 12.02 GB, 12.69 GB, 12.67 GB, 12.56 GB, 12.23 GB, 12.00 GB, 11.83 GB, 11.50 GB, 10.77 GB, 10.18 GB, 9.66 GB, 9.36 GB, 9.11 GB, 8.83 GB, 8.39 GB, 8.06 GB, 7.99 GB, 7.85 GB, 7.81 GB, 8.00 GB, 8.45 GB, 9.20 GB, 10.47 GB, 12.07 GB, 12.67 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.69 GB (13626259969) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.44 GB (171199917754) - RowsProduced: 592.38M (592382394) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 25m55s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 33.327ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 31.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7s492ms - OpenTime: 33.328ms - ExecTreeOpenTime: 165.839us - PrepareTime: 218.755ms - ExecTreePrepareTime: 2.013ms KrpcDataStreamSender (dst_id=1):(Total: 11m30s, non-child: 11m30s, % non-child: 100.00%) - BytesSent: 77.70 GB (83432090670) - EosSent: 0 (0) - OverallThroughput: 115.19 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 592.38M (592381370) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m29s - UncompressedRowBatchSize: 152.70 GB (163960921321) HDFS_SCAN_NODE (id=0):(Total: 6s931ms, non-child: 6s931ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:30/7.31 GB 1:25/6.06 GB 7:29/7.06 GB 5:27/6.54 GB 3:28/6.84 GB 2:28/6.79 GB 4:25/6.08 GB 6:33/8.02 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.32 GB, 15.36 GB, 16.04 GB, 16.84 GB, 17.87 GB, 18.25 GB, 18.58 GB, 18.73 GB, 18.96 GB, 19.17 GB, 19.24 GB, 19.37 GB, 19.48 GB, 19.53 GB, 19.68 GB, 20.25 GB, 20.75 GB, 21.01 GB, 21.58 GB, 22.57 GB, 23.70 GB, 25.07 GB, 27.31 GB, 30.08 GB, 31.23 GB, 31.42 GB, 31.56 GB, 31.75 GB, 32.01 GB, 32.30 GB, 32.46 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB, 32.47 GB - FooterProcessingTime: (Avg: 2s374ms ; Min: 29.363ms ; Max: 14s689ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.47 GB (34861767456) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m45s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.69 GB (13623135795) - PerReadThreadRawHdfsThroughput: 50.37 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 592.59M (592587194) - RowsReturned: 592.38M (592382394) - RowsReturnedRate: 85.47 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m48s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 27s488ms - TotalRawHdfsReadTime(*): 11m - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 32.831ms, non-child: 32.831ms, % non-child: 100.00%) - CodegenTime: 446.089us - CompileTime: 4.402ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.503ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 20.338ms Instance 844a0200d7876345:20bb38b900000064 (host=va1036.foo.com:22000):(Total: 252.091ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:21/5.13 GB 2:29/7.06 GB 1:28/6.83 GB 7:28/6.59 GB 0:24/5.90 GB 6:21/5.16 GB 3:31/7.50 GB 5:41/9.95 GB Fragment Instance Lifecycle Event Timeline: 1s621ms - Prepare Finished: 223.479ms (223.479ms) - Open Finished: 252.502ms (29.023ms) - First Batch Produced: 1s618ms (1s365ms) - First Batch Sent: 1s621ms (2.887ms) MemoryUsage(8m32s): 13.19 GB, 13.96 GB, 13.94 GB, 13.87 GB, 13.66 GB, 13.34 GB, 13.19 GB, 13.04 GB, 12.69 GB, 12.06 GB, 11.37 GB, 10.95 GB, 10.50 GB, 10.28 GB, 10.31 GB, 9.96 GB, 9.50 GB, 8.93 GB, 8.71 GB, 8.59 GB, 8.53 GB, 8.49 GB, 8.42 GB, 8.64 GB, 9.59 GB, 10.52 GB, 11.82 GB, 13.55 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (15002094186) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.73 GB (172578984338) - RowsProduced: 606.34M (606342213) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 29m4s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 29.021ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 25.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s777ms - OpenTime: 29.022ms - ExecTreeOpenTime: 139.007us - PrepareTime: 222.996ms - ExecTreePrepareTime: 1.868ms KrpcDataStreamSender (dst_id=1):(Total: 11m23s, non-child: 11m23s, % non-child: 100.00%) - BytesSent: 79.61 GB (85476021512) - EosSent: 0 (0) - OverallThroughput: 119.25 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 606.34M (606341189) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m22s - UncompressedRowBatchSize: 156.30 GB (167825099604) HDFS_SCAN_NODE (id=0):(Total: 8s261ms, non-child: 8s261ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:21/5.13 GB 2:29/7.06 GB 1:28/6.83 GB 7:28/6.59 GB 0:24/5.90 GB 6:21/5.16 GB 3:31/7.50 GB 5:41/9.95 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.27 GB, 16.75 GB, 17.41 GB, 18.09 GB, 19.26 GB, 19.87 GB, 20.17 GB, 20.48 GB, 20.61 GB, 20.83 GB, 21.09 GB, 21.16 GB, 21.25 GB, 21.44 GB, 21.77 GB, 21.77 GB, 21.77 GB, 21.94 GB, 22.42 GB, 22.59 GB, 23.07 GB, 23.63 GB, 24.04 GB, 24.93 GB, 26.69 GB, 28.38 GB, 30.71 GB, 33.65 GB, 34.37 GB, 34.54 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB - FooterProcessingTime: (Avg: 2s702ms ; Min: 25.152ms ; Max: 14s360ms ; Number of samples: 177) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.63 GB (37180780255) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m54s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 177 (177) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (14998970081) - PerReadThreadRawHdfsThroughput: 50.64 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 606.56M (606555205) - RowsReturned: 606.34M (606342213) - RowsReturnedRate: 73.39 M/sec - ScanRangesComplete: 89 (89) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m52s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 22s185ms - TotalRawHdfsReadTime(*): 11m40s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 28.566ms, non-child: 28.566ms, % non-child: 100.00%) - CodegenTime: 488.035us - CompileTime: 4.285ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.204ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 16.464ms Instance 844a0200d7876345:20bb38b900000049 (host=vc1503.foo.com:22000):(Total: 251.787ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 6:26/6.12 GB 0:26/6.40 GB 7:26/6.39 GB 2:26/6.40 GB 4:27/6.65 GB 5:27/6.57 GB 3:37/8.93 GB 1:29/7.10 GB Fragment Instance Lifecycle Event Timeline: 1s759ms - Prepare Finished: 225.703ms (225.703ms) - Open Finished: 252.522ms (26.819ms) - First Batch Produced: 1s757ms (1s505ms) - First Batch Sent: 1s759ms (1.931ms) MemoryUsage(8m32s): 13.21 GB, 13.99 GB, 13.97 GB, 13.90 GB, 13.73 GB, 13.43 GB, 13.30 GB, 13.13 GB, 12.78 GB, 12.02 GB, 11.50 GB, 11.05 GB, 10.69 GB, 10.51 GB, 10.32 GB, 10.06 GB, 9.41 GB, 8.75 GB, 8.48 GB, 8.51 GB, 8.38 GB, 8.26 GB, 8.19 GB, 8.53 GB, 8.92 GB, 10.26 GB, 12.15 GB, 13.47 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15021296889) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.70 GB (172553938113) - RowsProduced: 606.76M (606755039) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h13m - TotalStorageWaitTime: 29m43s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 26.815ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 23.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 11s392ms - OpenTime: 26.816ms - ExecTreeOpenTime: 154.534us - PrepareTime: 224.889ms - ExecTreePrepareTime: 1.768ms KrpcDataStreamSender (dst_id=1):(Total: 11m10s, non-child: 11m10s, % non-child: 100.00%) - BytesSent: 79.60 GB (85473516852) - EosSent: 0 (0) - OverallThroughput: 121.57 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 606.75M (606754015) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m9s - UncompressedRowBatchSize: 156.41 GB (167938932194) HDFS_SCAN_NODE (id=0):(Total: 10s880ms, non-child: 10s880ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:26/6.12 GB 0:26/6.40 GB 7:26/6.39 GB 2:26/6.40 GB 4:27/6.65 GB 5:27/6.57 GB 3:37/8.93 GB 1:29/7.10 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.44 GB, 16.89 GB, 17.56 GB, 18.22 GB, 19.48 GB, 20.01 GB, 20.36 GB, 20.63 GB, 20.74 GB, 21.04 GB, 21.26 GB, 21.38 GB, 21.57 GB, 21.74 GB, 21.80 GB, 21.80 GB, 21.81 GB, 21.91 GB, 22.05 GB, 22.53 GB, 22.89 GB, 23.43 GB, 23.92 GB, 24.89 GB, 25.66 GB, 28.10 GB, 31.35 GB, 33.60 GB, 34.45 GB, 34.53 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB - FooterProcessingTime: (Avg: 2s622ms ; Min: 29.558ms ; Max: 15s086ms ; Number of samples: 177) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.59 GB (37143381792) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 11m39s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 177 (177) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.99 GB (15018180907) - PerReadThreadRawHdfsThroughput: 49.43 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 606.97M (606968031) - RowsReturned: 606.76M (606755039) - RowsReturnedRate: 55.76 M/sec - ScanRangesComplete: 89 (89) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m36s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 22s218ms - TotalRawHdfsReadTime(*): 11m56s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 26.378ms, non-child: 26.378ms, % non-child: 100.00%) - CodegenTime: 400.606us - CompileTime: 4.187ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.329ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.317ms Instance 844a0200d7876345:20bb38b90000005c (host=vd1338.foo.com:22000):(Total: 251.398ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:25/6.04 GB 4:36/8.75 GB 3:27/6.54 GB 7:35/8.50 GB 6:26/6.22 GB 5:25/6.15 GB 0:24/5.86 GB 1:27/6.58 GB Fragment Instance Lifecycle Event Timeline: 1s607ms - Prepare Finished: 217.794ms (217.794ms) - Open Finished: 251.852ms (34.058ms) - First Batch Produced: 1s605ms (1s353ms) - First Batch Sent: 1s607ms (2.177ms) MemoryUsage(8m32s): 11.82 GB, 12.69 GB, 12.60 GB, 12.41 GB, 12.20 GB, 11.95 GB, 11.68 GB, 11.25 GB, 10.69 GB, 10.07 GB, 9.67 GB, 9.36 GB, 9.12 GB, 8.97 GB, 8.81 GB, 8.25 GB, 7.89 GB, 7.93 GB, 8.32 GB, 8.33 GB, 8.94 GB, 9.43 GB, 10.28 GB, 11.52 GB, 12.53 GB, 12.60 GB, 12.60 GB, 12.60 GB, 12.60 GB, 12.60 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB, 12.59 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.70 GB (13638428883) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.35 GB (171106113336) - RowsProduced: 597.68M (597684794) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 32m20s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 34.055ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 11s186ms - OpenTime: 34.056ms - ExecTreeOpenTime: 159.968us - PrepareTime: 217.265ms - ExecTreePrepareTime: 1.780ms KrpcDataStreamSender (dst_id=1):(Total: 11m37s, non-child: 11m37s, % non-child: 100.00%) - BytesSent: 78.48 GB (84269935522) - EosSent: 0 (0) - OverallThroughput: 115.25 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 597.68M (597683770) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m35s - UncompressedRowBatchSize: 154.07 GB (165429277081) HDFS_SCAN_NODE (id=0):(Total: 10s672ms, non-child: 10s672ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:25/6.04 GB 4:36/8.75 GB 3:27/6.54 GB 7:35/8.50 GB 6:26/6.22 GB 5:25/6.15 GB 0:24/5.86 GB 1:27/6.58 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.28 GB, 15.44 GB, 16.17 GB, 16.93 GB, 17.89 GB, 18.29 GB, 18.61 GB, 18.80 GB, 18.97 GB, 19.20 GB, 19.30 GB, 19.51 GB, 19.69 GB, 20.04 GB, 20.36 GB, 20.53 GB, 20.75 GB, 21.30 GB, 22.45 GB, 23.02 GB, 24.56 GB, 25.60 GB, 27.18 GB, 29.47 GB, 31.24 GB, 31.45 GB, 31.58 GB, 31.80 GB, 32.16 GB, 32.45 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB, 32.63 GB - FooterProcessingTime: (Avg: 2s518ms ; Min: 26.687ms ; Max: 23s990ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.63 GB (35035700042) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m31s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.70 GB (13635312976) - PerReadThreadRawHdfsThroughput: 41.19 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 597.89M (597889594) - RowsReturned: 597.68M (597684794) - RowsReturnedRate: 56.00 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m58s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 22s815ms - TotalRawHdfsReadTime(*): 13m31s - TotalReadThroughput: 1.16 MB/sec CodeGen:(Total: 33.615ms, non-child: 33.615ms, % non-child: 100.00%) - CodegenTime: 396.995us - CompileTime: 4.170ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.204ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 21.626ms Instance 844a0200d7876345:20bb38b900000033 (host=vd1125.foo.com:22000):(Total: 251.345ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:24/5.69 GB 4:37/9.01 GB 5:17/4.21 GB 6:22/5.37 GB 7:36/8.84 GB 2:31/7.55 GB 0:32/7.85 GB 3:25/6.19 GB Fragment Instance Lifecycle Event Timeline: 1s514ms - Prepare Finished: 216.816ms (216.816ms) - Open Finished: 251.820ms (35.004ms) - First Batch Produced: 1s512ms (1s261ms) - First Batch Sent: 1s514ms (2.064ms) MemoryUsage(8m32s): 13.25 GB, 13.97 GB, 13.96 GB, 13.93 GB, 13.80 GB, 13.51 GB, 13.27 GB, 13.22 GB, 13.03 GB, 12.63 GB, 11.83 GB, 11.39 GB, 10.88 GB, 10.53 GB, 10.46 GB, 10.29 GB, 9.97 GB, 9.60 GB, 8.94 GB, 8.61 GB, 8.62 GB, 8.50 GB, 8.23 GB, 8.08 GB, 8.31 GB, 9.01 GB, 9.90 GB, 11.52 GB, 13.54 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.98 GB (15009822272) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.67 GB (172519742292) - RowsProduced: 593.17M (593172598) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 26m31s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 34.993ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 34.995ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s880ms - OpenTime: 34.994ms - ExecTreeOpenTime: 165.482us - PrepareTime: 216.273ms - ExecTreePrepareTime: 2.048ms KrpcDataStreamSender (dst_id=1):(Total: 11m26s, non-child: 11m26s, % non-child: 100.00%) - BytesSent: 77.87 GB (83616629352) - EosSent: 0 (0) - OverallThroughput: 116.20 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 593.17M (593171574) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m24s - UncompressedRowBatchSize: 152.90 GB (164179250759) HDFS_SCAN_NODE (id=0):(Total: 10s306ms, non-child: 10s306ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:24/5.69 GB 4:37/9.01 GB 5:17/4.21 GB 6:22/5.37 GB 7:36/8.84 GB 2:31/7.55 GB 0:32/7.85 GB 3:25/6.19 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.92 GB, 16.36 GB, 17.13 GB, 17.73 GB, 18.72 GB, 19.67 GB, 19.99 GB, 20.35 GB, 20.53 GB, 20.67 GB, 20.94 GB, 21.13 GB, 21.20 GB, 21.38 GB, 21.61 GB, 21.67 GB, 21.67 GB, 21.79 GB, 22.00 GB, 22.16 GB, 22.47 GB, 22.80 GB, 22.98 GB, 23.43 GB, 24.39 GB, 25.73 GB, 27.30 GB, 30.19 GB, 33.63 GB, 34.31 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB, 34.35 GB - FooterProcessingTime: (Avg: 2s114ms ; Min: 26.511ms ; Max: 14s102ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.35 GB (36885640286) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m41s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.98 GB (15006757825) - PerReadThreadRawHdfsThroughput: 52.26 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 593.39M (593385590) - RowsReturned: 593.17M (593172598) - RowsReturnedRate: 57.55 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m48s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 28s991ms - TotalRawHdfsReadTime(*): 11m13s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 34.485ms, non-child: 34.485ms, % non-child: 100.00%) - CodegenTime: 446.872us - CompileTime: 4.506ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.581ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 21.851ms Instance 844a0200d7876345:20bb38b900000046 (host=va1034.foo.com:22000):(Total: 250.910ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 3:26/6.28 GB 6:30/7.34 GB 2:28/6.83 GB 7:26/6.29 GB 1:26/6.30 GB 0:29/7.15 GB 4:29/7.13 GB 5:29/7.10 GB Fragment Instance Lifecycle Event Timeline: 1s633ms - Prepare Finished: 211.956ms (211.956ms) - Open Finished: 251.386ms (39.430ms) - First Batch Produced: 1s631ms (1s379ms) - First Batch Sent: 1s633ms (1.725ms) MemoryUsage(8m32s): 13.17 GB, 13.94 GB, 13.94 GB, 13.88 GB, 13.62 GB, 13.24 GB, 13.13 GB, 13.01 GB, 12.71 GB, 11.95 GB, 11.33 GB, 10.83 GB, 10.37 GB, 10.21 GB, 10.01 GB, 9.74 GB, 9.34 GB, 8.81 GB, 8.63 GB, 8.72 GB, 8.76 GB, 8.56 GB, 8.53 GB, 8.92 GB, 9.49 GB, 10.71 GB, 12.69 GB, 13.90 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.96 GB (14987799433) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.71 GB (172565125624) - RowsProduced: 605.72M (605717019) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h18m - TotalStorageWaitTime: 28m47s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 39.412ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 38.994ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s634ms - OpenTime: 39.413ms - ExecTreeOpenTime: 294.214us - PrepareTime: 211.430ms - ExecTreePrepareTime: 2.252ms KrpcDataStreamSender (dst_id=1):(Total: 11m8s, non-child: 11m8s, % non-child: 100.00%) - BytesSent: 79.52 GB (85385902537) - EosSent: 0 (0) - OverallThroughput: 121.87 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 605.72M (605715995) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m6s - UncompressedRowBatchSize: 156.14 GB (167652243867) HDFS_SCAN_NODE (id=0):(Total: 8s148ms, non-child: 8s148ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:26/6.28 GB 6:30/7.34 GB 2:28/6.83 GB 7:26/6.29 GB 1:26/6.30 GB 0:29/7.15 GB 4:29/7.13 GB 5:29/7.10 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.06 GB, 16.67 GB, 17.30 GB, 17.91 GB, 19.14 GB, 19.74 GB, 20.05 GB, 20.38 GB, 20.48 GB, 20.72 GB, 20.97 GB, 21.03 GB, 21.11 GB, 21.28 GB, 21.35 GB, 21.35 GB, 21.55 GB, 21.96 GB, 22.30 GB, 22.78 GB, 23.35 GB, 23.67 GB, 24.29 GB, 25.30 GB, 26.31 GB, 28.51 GB, 31.99 GB, 34.09 GB, 34.29 GB, 34.42 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB - FooterProcessingTime: (Avg: 2s434ms ; Min: 29.295ms ; Max: 10s773ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.48 GB (37026430347) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 6m14s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.96 GB (14984762412) - PerReadThreadRawHdfsThroughput: 53.70 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 605.93M (605930011) - RowsReturned: 605.72M (605717019) - RowsReturnedRate: 74.33 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m21s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 25s322ms - TotalRawHdfsReadTime(*): 10m57s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 38.816ms, non-child: 38.816ms, % non-child: 100.00%) - CodegenTime: 391.084us - CompileTime: 5.562ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.547ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 24.195ms Instance 844a0200d7876345:20bb38b90000007c (host=vc1539.foo.com:22000):(Total: 250.167ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:25/6.12 GB 2:31/7.67 GB 6:18/4.46 GB 4:32/7.76 GB 3:27/6.61 GB 1:38/9.23 GB 7:26/6.32 GB 0:26/6.32 GB Fragment Instance Lifecycle Event Timeline: 1s648ms - Prepare Finished: 224.521ms (224.521ms) - Open Finished: 250.655ms (26.133ms) - First Batch Produced: 1s646ms (1s395ms) - First Batch Sent: 1s648ms (2.115ms) MemoryUsage(8m32s): 13.15 GB, 13.98 GB, 13.98 GB, 13.95 GB, 13.83 GB, 13.58 GB, 13.34 GB, 13.26 GB, 13.08 GB, 12.72 GB, 11.97 GB, 11.47 GB, 11.01 GB, 10.57 GB, 10.38 GB, 10.29 GB, 10.08 GB, 9.59 GB, 8.95 GB, 8.48 GB, 8.57 GB, 8.55 GB, 8.41 GB, 8.26 GB, 8.39 GB, 8.86 GB, 9.76 GB, 11.40 GB, 13.18 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.98 GB (15015035019) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.68 GB (172533560950) - RowsProduced: 592.53M (592527383) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h17m - TotalStorageWaitTime: 29m47s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 26.125ms - TotalThreadsSysTime: 3.999ms - TotalThreadsUserTime: 20.997ms - TotalThreadsVoluntaryContextSwitches: 9 (9) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s327ms - OpenTime: 26.127ms - ExecTreeOpenTime: 187.462us - PrepareTime: 223.957ms - ExecTreePrepareTime: 1.949ms KrpcDataStreamSender (dst_id=1):(Total: 11m15s, non-child: 11m15s, % non-child: 100.00%) - BytesSent: 77.73 GB (83466936652) - EosSent: 0 (0) - OverallThroughput: 117.88 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 592.53M (592526359) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m13s - UncompressedRowBatchSize: 152.74 GB (164001063651) HDFS_SCAN_NODE (id=0):(Total: 7s820ms, non-child: 7s820ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:25/6.12 GB 2:31/7.67 GB 6:18/4.46 GB 4:32/7.76 GB 3:27/6.61 GB 1:38/9.23 GB 7:26/6.32 GB 0:26/6.32 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.82 GB, 16.37 GB, 17.13 GB, 17.74 GB, 18.62 GB, 19.66 GB, 19.98 GB, 20.33 GB, 20.55 GB, 20.67 GB, 20.94 GB, 21.16 GB, 21.21 GB, 21.31 GB, 21.47 GB, 21.66 GB, 21.69 GB, 21.69 GB, 21.82 GB, 21.91 GB, 22.40 GB, 22.82 GB, 23.20 GB, 23.63 GB, 24.41 GB, 25.43 GB, 27.04 GB, 29.97 GB, 33.04 GB, 34.34 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB - FooterProcessingTime: (Avg: 2s133ms ; Min: 10.394ms ; Max: 13s934ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.37 GB (36907814397) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m23s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.98 GB (15012005152) - PerReadThreadRawHdfsThroughput: 49.09 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 592.74M (592740375) - RowsReturned: 592.53M (592527383) - RowsReturnedRate: 75.77 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m34s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 23s335ms - TotalRawHdfsReadTime(*): 11m56s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 25.657ms, non-child: 25.657ms, % non-child: 100.00%) - CodegenTime: 424.188us - CompileTime: 4.256ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.487ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.433ms Instance 844a0200d7876345:20bb38b900000076 (host=vd1134.foo.com:22000):(Total: 249.875ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 3:27/6.68 GB 7:25/6.11 GB 1:28/6.75 GB 6:27/6.48 GB 4:24/5.90 GB 5:25/6.11 GB 0:33/7.91 GB 2:36/8.84 GB Fragment Instance Lifecycle Event Timeline: 1s935ms - Prepare Finished: 212.098ms (212.098ms) - Open Finished: 250.563ms (38.464ms) - First Batch Produced: 1s933ms (1s682ms) - First Batch Sent: 1s935ms (2.149ms) MemoryUsage(8m32s): 12.00 GB, 12.63 GB, 12.60 GB, 12.52 GB, 12.22 GB, 12.05 GB, 11.99 GB, 11.77 GB, 11.12 GB, 10.44 GB, 9.90 GB, 9.53 GB, 9.50 GB, 9.31 GB, 8.89 GB, 8.28 GB, 8.08 GB, 8.00 GB, 7.73 GB, 7.43 GB, 7.50 GB, 8.43 GB, 9.65 GB, 11.78 GB, 12.69 GB, 12.71 GB, 12.71 GB, 12.70 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.71 GB (13645426881) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.44 GB (171193939379) - RowsProduced: 586.12M (586119918) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 25m14s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 38.457ms - TotalThreadsSysTime: 4.000ms - TotalThreadsUserTime: 33.994ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7s415ms - OpenTime: 38.458ms - ExecTreeOpenTime: 238.482us - PrepareTime: 211.346ms - ExecTreePrepareTime: 1.920ms KrpcDataStreamSender (dst_id=1):(Total: 11m37s, non-child: 11m37s, % non-child: 100.00%) - BytesSent: 76.91 GB (82583604967) - EosSent: 0 (0) - OverallThroughput: 112.94 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 586.12M (586118894) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m36s - UncompressedRowBatchSize: 151.09 GB (162228165438) HDFS_SCAN_NODE (id=0):(Total: 6s860ms, non-child: 6s860ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:27/6.68 GB 7:25/6.11 GB 1:28/6.75 GB 6:27/6.48 GB 4:24/5.90 GB 5:25/6.11 GB 0:33/7.91 GB 2:36/8.84 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 11.90 GB, 15.13 GB, 15.75 GB, 16.46 GB, 17.65 GB, 18.16 GB, 18.55 GB, 18.68 GB, 18.91 GB, 19.17 GB, 19.25 GB, 19.43 GB, 19.70 GB, 19.73 GB, 19.89 GB, 20.19 GB, 20.46 GB, 20.79 GB, 21.12 GB, 21.35 GB, 22.01 GB, 23.72 GB, 25.89 GB, 29.60 GB, 31.16 GB, 31.27 GB, 31.41 GB, 31.58 GB, 31.86 GB, 32.05 GB, 32.10 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB - FooterProcessingTime: (Avg: 2s483ms ; Min: 23.699ms ; Max: 11s343ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.11 GB (34472828290) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 6m10s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.71 GB (13642321918) - PerReadThreadRawHdfsThroughput: 46.62 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 586.32M (586324718) - RowsReturned: 586.12M (586119918) - RowsReturnedRate: 85.44 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m30s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 31s567ms - TotalRawHdfsReadTime(*): 11m45s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 37.943ms, non-child: 37.943ms, % non-child: 100.00%) - CodegenTime: 400.520us - CompileTime: 4.236ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.700ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 24.433ms Instance 844a0200d7876345:20bb38b900000032 (host=ve1135.foo.com:22000):(Total: 249.873ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:27/6.59 GB 1:27/6.53 GB 6:40/9.82 GB 5:19/4.57 GB 3:29/7.06 GB 2:32/7.54 GB 7:26/6.40 GB 0:25/6.15 GB Fragment Instance Lifecycle Event Timeline: 2s099ms - Prepare Finished: 217.145ms (217.145ms) - Open Finished: 250.570ms (33.424ms) - First Batch Produced: 2s097ms (1s846ms) - First Batch Sent: 2s099ms (1.972ms) MemoryUsage(8m32s): 12.05 GB, 12.71 GB, 12.66 GB, 12.47 GB, 12.12 GB, 11.98 GB, 11.73 GB, 11.20 GB, 10.45 GB, 9.89 GB, 9.41 GB, 9.27 GB, 9.12 GB, 8.74 GB, 8.00 GB, 7.97 GB, 8.06 GB, 7.74 GB, 7.74 GB, 8.11 GB, 9.04 GB, 10.66 GB, 12.37 GB, 12.65 GB, 12.65 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.72 GB (13654061616) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.39 GB (171139269720) - RowsProduced: 605.68M (605675113) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 25m14s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 33.415ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 29.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s759ms - OpenTime: 33.417ms - ExecTreeOpenTime: 144.373us - PrepareTime: 216.371ms - ExecTreePrepareTime: 1.834ms KrpcDataStreamSender (dst_id=1):(Total: 11m33s, non-child: 11m33s, % non-child: 100.00%) - BytesSent: 79.52 GB (85387994721) - EosSent: 0 (0) - OverallThroughput: 117.47 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 605.67M (605674089) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m31s - UncompressedRowBatchSize: 156.13 GB (167640427366) HDFS_SCAN_NODE (id=0):(Total: 8s223ms, non-child: 8s223ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:27/6.59 GB 1:27/6.53 GB 6:40/9.82 GB 5:19/4.57 GB 3:29/7.06 GB 2:32/7.54 GB 7:26/6.40 GB 0:25/6.15 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.60 GB, 15.56 GB, 16.21 GB, 17.28 GB, 18.06 GB, 18.45 GB, 18.68 GB, 18.86 GB, 19.15 GB, 19.25 GB, 19.36 GB, 19.52 GB, 19.76 GB, 19.97 GB, 20.08 GB, 20.62 GB, 21.26 GB, 21.44 GB, 22.19 GB, 23.17 GB, 24.88 GB, 27.81 GB, 30.72 GB, 31.26 GB, 31.39 GB, 31.62 GB, 31.86 GB, 32.10 GB, 32.29 GB, 32.54 GB, 32.76 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB, 32.77 GB - FooterProcessingTime: (Avg: 2s470ms ; Min: 31.609ms ; Max: 11s066ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.77 GB (35191876749) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m23s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.71 GB (13650888383) - PerReadThreadRawHdfsThroughput: 46.36 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 605.88M (605879913) - RowsReturned: 605.68M (605675113) - RowsReturnedRate: 73.65 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m20s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 19s702ms - TotalRawHdfsReadTime(*): 12m3s - TotalReadThroughput: 1.16 MB/sec CodeGen:(Total: 32.972ms, non-child: 32.972ms, % non-child: 100.00%) - CodegenTime: 466.568us - CompileTime: 4.128ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.109ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 21.205ms Instance 844a0200d7876345:20bb38b900000011 (host=vc1532.foo.com:22000):(Total: 249.298ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:27/6.64 GB 0:20/4.87 GB 2:31/7.55 GB 5:29/7.14 GB 1:25/6.19 GB 3:29/6.79 GB 4:36/8.61 GB 6:27/6.61 GB Fragment Instance Lifecycle Event Timeline: 1s549ms - Prepare Finished: 222.258ms (222.258ms) - Open Finished: 250.045ms (27.786ms) - First Batch Produced: 1s547ms (1s297ms) - First Batch Sent: 1s549ms (1.952ms) MemoryUsage(8m32s): 13.37 GB, 13.99 GB, 13.99 GB, 13.95 GB, 13.78 GB, 13.45 GB, 13.35 GB, 13.26 GB, 13.00 GB, 12.36 GB, 11.67 GB, 11.23 GB, 10.70 GB, 10.51 GB, 10.39 GB, 10.14 GB, 9.90 GB, 9.12 GB, 8.52 GB, 8.58 GB, 8.40 GB, 8.10 GB, 8.08 GB, 7.90 GB, 8.66 GB, 9.31 GB, 10.91 GB, 13.35 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15025920900) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.34 GB (188264764604) - RowsProduced: 599.28M (599278977) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 23m17s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 27.784ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 25.996ms - TotalThreadsVoluntaryContextSwitches: 7 (7) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s548ms - OpenTime: 27.785ms - ExecTreeOpenTime: 144.842us - PrepareTime: 221.429ms - ExecTreePrepareTime: 1.817ms KrpcDataStreamSender (dst_id=1):(Total: 11m12s, non-child: 11m12s, % non-child: 100.00%) - BytesSent: 78.63 GB (84432316104) - EosSent: 0 (0) - OverallThroughput: 119.77 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 599.28M (599277953) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m11s - UncompressedRowBatchSize: 154.48 GB (165870584082) HDFS_SCAN_NODE (id=0):(Total: 10s055ms, non-child: 10s055ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:27/6.64 GB 0:20/4.87 GB 2:31/7.55 GB 5:29/7.14 GB 1:25/6.19 GB 3:29/6.79 GB 4:36/8.61 GB 6:27/6.61 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.41 GB, 16.76 GB, 17.33 GB, 17.90 GB, 19.16 GB, 19.86 GB, 20.15 GB, 20.52 GB, 20.62 GB, 20.82 GB, 21.15 GB, 21.21 GB, 21.29 GB, 21.47 GB, 21.59 GB, 21.59 GB, 21.81 GB, 21.88 GB, 21.88 GB, 22.21 GB, 22.45 GB, 22.64 GB, 23.30 GB, 23.67 GB, 25.11 GB, 26.35 GB, 29.22 GB, 33.39 GB, 34.34 GB, 34.37 GB, 34.45 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB - FooterProcessingTime: (Avg: 1s981ms ; Min: 33.682ms ; Max: 8s399ms ; Number of samples: 177) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.46 GB (36997722033) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m21s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 177 (177) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.99 GB (15022845964) - PerReadThreadRawHdfsThroughput: 54.93 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 599.49M (599491969) - RowsReturned: 599.28M (599278977) - RowsReturnedRate: 59.60 M/sec - ScanRangesComplete: 89 (89) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m25s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 28s618ms - TotalRawHdfsReadTime(*): 10m42s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 27.305ms, non-child: 27.305ms, % non-child: 100.00%) - CodegenTime: 439.363us - CompileTime: 4.103ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.017ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 15.642ms Instance 844a0200d7876345:20bb38b90000005e (host=vd1329.foo.com:22000):(Total: 249.265ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:18/4.42 GB 2:17/4.14 GB 6:32/7.86 GB 5:32/7.81 GB 0:36/8.88 GB 3:28/6.85 GB 1:26/6.37 GB 4:34/8.34 GB Fragment Instance Lifecycle Event Timeline: 1s813ms - Prepare Finished: 214.817ms (214.817ms) - Open Finished: 249.765ms (34.947ms) - First Batch Produced: 1s811ms (1s561ms) - First Batch Sent: 1s813ms (2.065ms) MemoryUsage(8m32s): 12.01 GB, 12.72 GB, 12.68 GB, 12.58 GB, 12.36 GB, 12.12 GB, 11.96 GB, 11.64 GB, 11.06 GB, 10.37 GB, 9.92 GB, 9.53 GB, 9.40 GB, 9.10 GB, 8.74 GB, 8.18 GB, 7.91 GB, 7.76 GB, 7.58 GB, 7.69 GB, 8.20 GB, 8.86 GB, 10.05 GB, 11.50 GB, 12.65 GB, 12.69 GB, 12.69 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.73 GB (13664163487) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.46 GB (171221639999) - RowsProduced: 591.68M (591678568) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 27m18s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 34.941ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 31.995ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s944ms - OpenTime: 34.942ms - ExecTreeOpenTime: 175.107us - PrepareTime: 214.245ms - ExecTreePrepareTime: 1.998ms KrpcDataStreamSender (dst_id=1):(Total: 11m36s, non-child: 11m36s, % non-child: 100.00%) - BytesSent: 77.68 GB (83406507681) - EosSent: 0 (0) - OverallThroughput: 114.15 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 591.68M (591677544) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m35s - UncompressedRowBatchSize: 152.52 GB (163766828717) HDFS_SCAN_NODE (id=0):(Total: 8s413ms, non-child: 8s413ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:18/4.42 GB 2:17/4.14 GB 6:32/7.86 GB 5:32/7.81 GB 0:36/8.88 GB 3:28/6.85 GB 1:26/6.37 GB 4:34/8.34 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.15 GB, 15.32 GB, 15.99 GB, 16.76 GB, 17.86 GB, 18.31 GB, 18.62 GB, 18.82 GB, 19.00 GB, 19.26 GB, 19.36 GB, 19.47 GB, 19.75 GB, 19.82 GB, 19.92 GB, 20.15 GB, 20.52 GB, 20.77 GB, 21.13 GB, 21.95 GB, 23.34 GB, 24.56 GB, 26.67 GB, 29.24 GB, 31.27 GB, 31.48 GB, 31.60 GB, 31.77 GB, 31.97 GB, 32.25 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB, 32.50 GB - FooterProcessingTime: (Avg: 2s431ms ; Min: 26.471ms ; Max: 14s148ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.50 GB (34896139135) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 6m38s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.72 GB (13661064007) - PerReadThreadRawHdfsThroughput: 49.46 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 591.88M (591883368) - RowsReturned: 591.68M (591678568) - RowsReturnedRate: 70.32 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m43s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 23s828ms - TotalRawHdfsReadTime(*): 11m12s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 34.316ms, non-child: 34.316ms, % non-child: 100.00%) - CodegenTime: 724.981us - CompileTime: 4.087ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.900ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.535ms Instance 844a0200d7876345:20bb38b900000001 (host=vd1113.foo.com:22000):(Total: 248.885ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:30/7.26 GB 4:26/6.25 GB 7:29/7.04 GB 3:26/6.36 GB 0:27/6.57 GB 6:22/5.36 GB 2:27/6.30 GB 1:40/9.71 GB Fragment Instance Lifecycle Event Timeline: 1s737ms - Prepare Finished: 212.479ms (212.479ms) - Open Finished: 249.503ms (37.023ms) - First Batch Produced: 1s735ms (1s486ms) - First Batch Sent: 1s737ms (2.022ms) MemoryUsage(8m32s): 13.25 GB, 13.95 GB, 13.94 GB, 13.88 GB, 13.68 GB, 13.37 GB, 13.26 GB, 13.10 GB, 12.76 GB, 12.04 GB, 11.56 GB, 11.10 GB, 10.65 GB, 10.50 GB, 10.32 GB, 10.00 GB, 9.42 GB, 8.82 GB, 8.60 GB, 8.61 GB, 8.49 GB, 8.23 GB, 8.24 GB, 8.62 GB, 9.28 GB, 10.59 GB, 12.29 GB, 13.66 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (15002909907) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.24 GB (188161354886) - RowsProduced: 608.41M (608413219) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 28m2s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 37.012ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 12s035ms - OpenTime: 37.013ms - ExecTreeOpenTime: 146.104us - PrepareTime: 211.780ms - ExecTreePrepareTime: 1.790ms KrpcDataStreamSender (dst_id=1):(Total: 11m9s, non-child: 11m9s, % non-child: 100.00%) - BytesSent: 79.83 GB (85719390631) - EosSent: 0 (0) - OverallThroughput: 122.04 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 608.41M (608412195) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m8s - UncompressedRowBatchSize: 156.83 GB (168398304197) HDFS_SCAN_NODE (id=0):(Total: 11s509ms, non-child: 11s509ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:30/7.26 GB 4:26/6.25 GB 7:29/7.04 GB 3:26/6.36 GB 0:27/6.57 GB 6:22/5.36 GB 2:27/6.30 GB 1:40/9.71 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.43 GB, 16.82 GB, 17.46 GB, 18.11 GB, 19.36 GB, 19.91 GB, 20.25 GB, 20.53 GB, 20.64 GB, 20.91 GB, 21.25 GB, 21.35 GB, 21.45 GB, 21.63 GB, 21.70 GB, 21.70 GB, 21.72 GB, 21.89 GB, 22.20 GB, 22.61 GB, 23.02 GB, 23.29 GB, 23.93 GB, 24.97 GB, 26.19 GB, 28.58 GB, 31.57 GB, 33.95 GB, 34.27 GB, 34.41 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB - FooterProcessingTime: (Avg: 3s000ms ; Min: 31.448ms ; Max: 13s185ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.48 GB (37025741798) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m39s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (14999753033) - PerReadThreadRawHdfsThroughput: 51.22 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 608.63M (608626211) - RowsReturned: 608.41M (608413219) - RowsReturnedRate: 52.86 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m6s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 22s116ms - TotalRawHdfsReadTime(*): 11m29s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 36.566ms, non-child: 36.566ms, % non-child: 100.00%) - CodegenTime: 410.002us - CompileTime: 4.321ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.750ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.991ms Instance 844a0200d7876345:20bb38b900000008 (host=vd1110.foo.com:22000):(Total: 248.785ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:31/7.60 GB 3:29/7.06 GB 6:22/5.34 GB 5:34/8.30 GB 0:25/6.07 GB 4:28/6.75 GB 1:32/7.35 GB 2:26/6.24 GB Fragment Instance Lifecycle Event Timeline: 1s305ms - Prepare Finished: 215.141ms (215.141ms) - Open Finished: 249.544ms (34.403ms) - First Batch Produced: 1s303ms (1s053ms) - First Batch Sent: 1s305ms (2.114ms) MemoryUsage(8m32s): 13.24 GB, 13.95 GB, 13.94 GB, 13.88 GB, 13.63 GB, 13.33 GB, 13.21 GB, 13.08 GB, 12.69 GB, 12.00 GB, 11.42 GB, 10.99 GB, 10.61 GB, 10.44 GB, 10.28 GB, 9.90 GB, 9.36 GB, 8.94 GB, 8.72 GB, 8.64 GB, 8.50 GB, 8.37 GB, 8.20 GB, 8.41 GB, 9.31 GB, 10.38 GB, 12.04 GB, 13.22 GB, 13.77 GB, 13.82 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.83 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (14997660879) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.25 GB (188170087602) - RowsProduced: 605.00M (604999597) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h13m - TotalStorageWaitTime: 26m51s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 34.391ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 33.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 13s717ms - OpenTime: 34.393ms - ExecTreeOpenTime: 158.025us - PrepareTime: 214.305ms - ExecTreePrepareTime: 1.895ms KrpcDataStreamSender (dst_id=1):(Total: 11m9s, non-child: 11m9s, % non-child: 100.00%) - BytesSent: 79.43 GB (85287805161) - EosSent: 0 (0) - OverallThroughput: 121.58 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 605.00M (604998573) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m7s - UncompressedRowBatchSize: 155.95 GB (167453146997) HDFS_SCAN_NODE (id=0):(Total: 13s209ms, non-child: 13s209ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:31/7.60 GB 3:29/7.06 GB 6:22/5.34 GB 5:34/8.30 GB 0:25/6.07 GB 4:28/6.75 GB 1:32/7.35 GB 2:26/6.24 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.43 GB, 16.78 GB, 17.44 GB, 18.13 GB, 19.33 GB, 19.88 GB, 20.24 GB, 20.49 GB, 20.64 GB, 20.90 GB, 21.11 GB, 21.26 GB, 21.43 GB, 21.59 GB, 21.66 GB, 21.67 GB, 21.71 GB, 22.05 GB, 22.34 GB, 22.63 GB, 23.03 GB, 23.40 GB, 23.78 GB, 24.52 GB, 26.23 GB, 28.12 GB, 31.06 GB, 33.08 GB, 34.13 GB, 34.32 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB, 34.43 GB - FooterProcessingTime: (Avg: 2s184ms ; Min: 35.821ms ; Max: 11s372ms ; Number of samples: 177) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.43 GB (36964954444) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 11m23s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 177 (177) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.96 GB (14994518562) - PerReadThreadRawHdfsThroughput: 51.67 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 605.21M (605212589) - RowsReturned: 605.00M (604999597) - RowsReturnedRate: 45.80 M/sec - ScanRangesComplete: 89 (89) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m51s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 18s473ms - TotalRawHdfsReadTime(*): 11m22s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 33.872ms, non-child: 33.872ms, % non-child: 100.00%) - CodegenTime: 441.101us - CompileTime: 4.210ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.575ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 20.475ms Instance 844a0200d7876345:20bb38b900000053 (host=ve1139.foo.com:22000):(Total: 248.285ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:26/6.35 GB 6:34/8.27 GB 7:26/6.40 GB 5:32/7.85 GB 2:30/7.36 GB 3:27/6.56 GB 1:22/5.41 GB 4:27/6.60 GB Fragment Instance Lifecycle Event Timeline: 1s732ms - Prepare Finished: 216.751ms (216.751ms) - Open Finished: 248.635ms (31.884ms) - First Batch Produced: 1s730ms (1s481ms) - First Batch Sent: 1s732ms (2.163ms) MemoryUsage(8m32s): 12.02 GB, 12.71 GB, 12.67 GB, 12.54 GB, 12.26 GB, 12.09 GB, 11.87 GB, 11.40 GB, 10.67 GB, 10.16 GB, 9.76 GB, 9.55 GB, 9.32 GB, 8.86 GB, 8.22 GB, 7.87 GB, 7.66 GB, 7.48 GB, 7.54 GB, 7.97 GB, 8.91 GB, 10.50 GB, 12.05 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.66 GB, 12.66 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB, 12.65 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.72 GB (13660550530) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.40 GB (171149354521) - RowsProduced: 604.94M (604944289) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h17m - TotalStorageWaitTime: 27m2s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 31.882ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 30.995ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7s347ms - OpenTime: 31.883ms - ExecTreeOpenTime: 279.716us - PrepareTime: 216.354ms - ExecTreePrepareTime: 1.786ms KrpcDataStreamSender (dst_id=1):(Total: 11m5s, non-child: 11m5s, % non-child: 100.00%) - BytesSent: 79.37 GB (85226052138) - EosSent: 0 (0) - OverallThroughput: 122.14 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 604.94M (604943265) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m4s - UncompressedRowBatchSize: 155.94 GB (167438286004) HDFS_SCAN_NODE (id=0):(Total: 6s860ms, non-child: 6s860ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:26/6.35 GB 6:34/8.27 GB 7:26/6.40 GB 5:32/7.85 GB 2:30/7.36 GB 3:27/6.56 GB 1:22/5.41 GB 4:27/6.60 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.36 GB, 15.44 GB, 16.12 GB, 17.15 GB, 18.08 GB, 18.47 GB, 18.74 GB, 18.91 GB, 19.19 GB, 19.37 GB, 19.58 GB, 19.73 GB, 19.79 GB, 19.80 GB, 19.98 GB, 20.31 GB, 20.53 GB, 20.96 GB, 21.80 GB, 22.97 GB, 24.70 GB, 27.48 GB, 30.19 GB, 31.31 GB, 31.45 GB, 31.58 GB, 31.78 GB, 32.03 GB, 32.29 GB, 32.57 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB, 32.76 GB - FooterProcessingTime: (Avg: 2s504ms ; Min: 28.075ms ; Max: 11s510ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.76 GB (35178908783) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m13s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.72 GB (13657551014) - PerReadThreadRawHdfsThroughput: 48.54 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 605.15M (605149089) - RowsReturned: 604.94M (604944289) - RowsReturnedRate: 88.17 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m48s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 18s765ms - TotalRawHdfsReadTime(*): 11m31s - TotalReadThroughput: 1.16 MB/sec CodeGen:(Total: 31.322ms, non-child: 31.322ms, % non-child: 100.00%) - CodegenTime: 366.500us - CompileTime: 4.733ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.445ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 19.581ms Instance 844a0200d7876345:20bb38b90000005d (host=vd1336.foo.com:22000):(Total: 248.194ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:28/6.86 GB 4:25/6.19 GB 7:24/5.91 GB 3:37/9.12 GB 5:28/6.86 GB 6:28/6.77 GB 2:32/7.92 GB 0:21/5.01 GB Fragment Instance Lifecycle Event Timeline: 2s010ms - Prepare Finished: 207.192ms (207.192ms) - Open Finished: 248.650ms (41.457ms) - First Batch Produced: 2s008ms (1s759ms) - First Batch Sent: 2s010ms (1.611ms) MemoryUsage(8m32s): 12.04 GB, 12.63 GB, 12.67 GB, 12.67 GB, 12.41 GB, 12.20 GB, 12.14 GB, 11.85 GB, 11.14 GB, 10.51 GB, 9.97 GB, 9.70 GB, 9.57 GB, 9.25 GB, 8.72 GB, 7.91 GB, 7.65 GB, 7.54 GB, 7.20 GB, 7.09 GB, 7.81 GB, 8.67 GB, 10.45 GB, 12.46 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.72 GB (13659992791) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.42 GB (171171240595) - RowsProduced: 590.00M (589999747) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 22m34s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 41.448ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 37.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s719ms - OpenTime: 41.449ms - ExecTreeOpenTime: 257.046us - PrepareTime: 206.680ms - ExecTreePrepareTime: 1.995ms KrpcDataStreamSender (dst_id=1):(Total: 11m23s, non-child: 11m23s, % non-child: 100.00%) - BytesSent: 77.43 GB (83138577020) - EosSent: 0 (0) - OverallThroughput: 115.93 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 590.00M (589998723) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m22s - UncompressedRowBatchSize: 152.09 GB (163301665968) HDFS_SCAN_NODE (id=0):(Total: 8s214ms, non-child: 8s214ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:28/6.86 GB 4:25/6.19 GB 7:24/5.91 GB 3:37/9.12 GB 5:28/6.86 GB 6:28/6.77 GB 2:32/7.92 GB 0:21/5.01 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 11.95 GB, 15.17 GB, 15.83 GB, 16.72 GB, 17.90 GB, 18.30 GB, 18.67 GB, 18.84 GB, 19.09 GB, 19.34 GB, 19.43 GB, 19.57 GB, 19.73 GB, 19.74 GB, 19.75 GB, 19.84 GB, 19.95 GB, 20.37 GB, 20.51 GB, 21.14 GB, 22.68 GB, 24.20 GB, 27.38 GB, 30.78 GB, 31.17 GB, 31.27 GB, 31.39 GB, 31.62 GB, 31.86 GB, 32.02 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB - FooterProcessingTime: (Avg: 2s058ms ; Min: 23.192ms ; Max: 12s330ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.16 GB (34529220347) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m18s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.72 GB (13656836191) - PerReadThreadRawHdfsThroughput: 48.71 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 590.20M (590204547) - RowsReturned: 590.00M (589999747) - RowsReturnedRate: 71.82 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m57s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 17s339ms - TotalRawHdfsReadTime(*): 11m15s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 40.862ms, non-child: 40.862ms, % non-child: 100.00%) - CodegenTime: 437.068us - CompileTime: 4.650ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 9.107ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 26.250ms Instance 844a0200d7876345:20bb38b90000006b (host=vc1510.foo.com:22000):(Total: 247.960ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 3:27/6.38 GB 1:14/3.39 GB 6:29/7.14 GB 5:22/5.40 GB 2:28/6.72 GB 4:41/9.91 GB 7:31/7.33 GB 0:34/8.38 GB Fragment Instance Lifecycle Event Timeline: 1s764ms - Prepare Finished: 225.000ms (225.000ms) - Open Finished: 248.318ms (23.318ms) - First Batch Produced: 1s762ms (1s513ms) - First Batch Sent: 1s764ms (1.914ms) MemoryUsage(8m32s): 13.28 GB, 13.96 GB, 13.96 GB, 13.93 GB, 13.77 GB, 13.39 GB, 13.21 GB, 13.17 GB, 12.97 GB, 12.38 GB, 11.66 GB, 11.16 GB, 10.64 GB, 10.38 GB, 10.27 GB, 10.09 GB, 9.68 GB, 9.18 GB, 8.84 GB, 8.85 GB, 8.65 GB, 8.48 GB, 8.08 GB, 7.99 GB, 8.24 GB, 9.17 GB, 10.85 GB, 12.53 GB, 13.60 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB, 13.81 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.96 GB (14990616945) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.57 GB (172408807361) - RowsProduced: 595.38M (595378670) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h17m - TotalStorageWaitTime: 25m26s - TotalThreadsInvoluntaryContextSwitches: 2 (2) - TotalThreadsTotalWallClockTime: 23.315ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 21.997ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s391ms - OpenTime: 23.316ms - ExecTreeOpenTime: 220.124us - PrepareTime: 224.559ms - ExecTreePrepareTime: 1.618ms KrpcDataStreamSender (dst_id=1):(Total: 11m19s, non-child: 11m19s, % non-child: 100.00%) - BytesSent: 78.20 GB (83969717224) - EosSent: 0 (0) - OverallThroughput: 117.78 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 595.38M (595377646) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m18s - UncompressedRowBatchSize: 153.47 GB (164790924816) HDFS_SCAN_NODE (id=0):(Total: 7s870ms, non-child: 7s870ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:27/6.38 GB 1:14/3.39 GB 6:29/7.14 GB 5:22/5.40 GB 2:28/6.72 GB 4:41/9.91 GB 7:31/7.33 GB 0:34/8.38 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.98 GB, 16.48 GB, 17.14 GB, 17.67 GB, 18.82 GB, 19.65 GB, 19.97 GB, 20.32 GB, 20.45 GB, 20.65 GB, 20.93 GB, 21.04 GB, 21.11 GB, 21.27 GB, 21.39 GB, 21.42 GB, 21.42 GB, 21.68 GB, 22.05 GB, 22.55 GB, 22.66 GB, 23.00 GB, 23.10 GB, 23.59 GB, 24.31 GB, 26.00 GB, 28.96 GB, 31.90 GB, 33.74 GB, 34.16 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB - FooterProcessingTime: (Avg: 1s930ms ; Min: 18.660ms ; Max: 11s111ms ; Number of samples: 177) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.28 GB (36807089490) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m19s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 177 (177) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.96 GB (14987574747) - PerReadThreadRawHdfsThroughput: 52.80 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 595.59M (595591662) - RowsReturned: 595.38M (595378670) - RowsReturnedRate: 75.64 M/sec - ScanRangesComplete: 89 (89) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m43s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 22s941ms - TotalRawHdfsReadTime(*): 11m4s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 22.849ms, non-child: 22.849ms, % non-child: 100.00%) - CodegenTime: 353.513us - CompileTime: 3.723ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 5.475ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 13.182ms Instance 844a0200d7876345:20bb38b90000000b (host=vc1511.foo.com:22000):(Total: 247.921ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:23/5.45 GB 5:37/9.08 GB 7:20/4.92 GB 6:26/6.33 GB 4:33/7.89 GB 2:21/5.12 GB 3:33/8.02 GB 0:34/8.14 GB Fragment Instance Lifecycle Event Timeline: 1s710ms - Prepare Finished: 223.147ms (223.147ms) - Open Finished: 248.505ms (25.358ms) - First Batch Produced: 1s708ms (1s460ms) - First Batch Sent: 1s710ms (1.877ms) MemoryUsage(8m32s): 13.15 GB, 13.86 GB, 13.85 GB, 13.81 GB, 13.62 GB, 13.31 GB, 13.18 GB, 13.12 GB, 12.91 GB, 12.45 GB, 11.68 GB, 11.21 GB, 10.73 GB, 10.41 GB, 10.34 GB, 10.11 GB, 9.87 GB, 9.42 GB, 8.98 GB, 8.95 GB, 8.80 GB, 8.75 GB, 8.54 GB, 8.48 GB, 8.86 GB, 9.35 GB, 10.39 GB, 11.77 GB, 13.36 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.92 GB (14946736078) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.34 GB (188267575484) - RowsProduced: 592.09M (592093934) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 28m51s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 25.354ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 22.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s865ms - OpenTime: 25.356ms - ExecTreeOpenTime: 248.475us - PrepareTime: 222.490ms - ExecTreePrepareTime: 1.983ms KrpcDataStreamSender (dst_id=1):(Total: 11m6s, non-child: 11m6s, % non-child: 100.00%) - BytesSent: 77.71 GB (83439844388) - EosSent: 0 (0) - OverallThroughput: 119.38 MB/sec - PeakMemoryUsage: 3.59 MB (3760800) - RowsReturned: 592.09M (592092910) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m5s - UncompressedRowBatchSize: 152.63 GB (163881098799) HDFS_SCAN_NODE (id=0):(Total: 9s366ms, non-child: 9s366ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:23/5.45 GB 5:37/9.08 GB 7:20/4.92 GB 6:26/6.33 GB 4:33/7.89 GB 2:21/5.12 GB 3:33/8.02 GB 0:34/8.14 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.85 GB, 16.28 GB, 17.00 GB, 17.58 GB, 18.55 GB, 19.48 GB, 19.92 GB, 20.23 GB, 20.44 GB, 20.59 GB, 20.84 GB, 21.01 GB, 21.09 GB, 21.28 GB, 21.50 GB, 21.55 GB, 21.56 GB, 21.70 GB, 22.03 GB, 22.66 GB, 22.80 GB, 23.15 GB, 23.42 GB, 23.91 GB, 25.11 GB, 26.05 GB, 27.90 GB, 30.34 GB, 33.12 GB, 34.16 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB, 34.28 GB - FooterProcessingTime: (Avg: 2s842ms ; Min: 43.111ms ; Max: 11s172ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.28 GB (36807534263) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m58s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.92 GB (14943702110) - PerReadThreadRawHdfsThroughput: 59.06 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 592.31M (592306926) - RowsReturned: 592.09M (592093934) - RowsReturnedRate: 63.22 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m52s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 18s484ms - TotalRawHdfsReadTime(*): 9m54s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 24.867ms, non-child: 24.867ms, % non-child: 100.00%) - CodegenTime: 369.132us - CompileTime: 4.250ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.493ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 13.666ms Instance 844a0200d7876345:20bb38b90000004c (host=vc1525.foo.com:22000):(Total: 246.825ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 6:22/5.40 GB 0:29/7.01 GB 5:21/5.16 GB 2:31/7.64 GB 7:28/6.45 GB 1:23/5.30 GB 3:36/8.82 GB 4:35/8.59 GB Fragment Instance Lifecycle Event Timeline: 1s757ms - Prepare Finished: 221.747ms (221.747ms) - Open Finished: 247.280ms (25.532ms) - First Batch Produced: 1s755ms (1s507ms) - First Batch Sent: 1s757ms (1.895ms) MemoryUsage(8m32s): 13.18 GB, 13.98 GB, 13.97 GB, 13.89 GB, 13.70 GB, 13.42 GB, 13.30 GB, 13.14 GB, 12.77 GB, 12.10 GB, 11.50 GB, 11.04 GB, 10.64 GB, 10.45 GB, 10.24 GB, 9.95 GB, 9.62 GB, 8.96 GB, 8.64 GB, 8.49 GB, 8.34 GB, 8.15 GB, 8.12 GB, 8.46 GB, 9.33 GB, 10.14 GB, 11.93 GB, 13.16 GB, 13.80 GB, 13.88 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB, 13.87 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.98 GB (15012545535) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.62 GB (172469602738) - RowsProduced: 605.04M (605041882) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 28m53s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 25.538ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 23.997ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s591ms - OpenTime: 25.540ms - ExecTreeOpenTime: 120.687us - PrepareTime: 221.209ms - ExecTreePrepareTime: 1.718ms KrpcDataStreamSender (dst_id=1):(Total: 11m9s, non-child: 11m9s, % non-child: 100.00%) - BytesSent: 79.36 GB (85213151344) - EosSent: 0 (0) - OverallThroughput: 121.36 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 605.04M (605040858) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m8s - UncompressedRowBatchSize: 155.96 GB (167465452498) HDFS_SCAN_NODE (id=0):(Total: 8s104ms, non-child: 8s104ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:22/5.40 GB 0:29/7.01 GB 5:21/5.16 GB 2:31/7.64 GB 7:28/6.45 GB 1:23/5.30 GB 3:36/8.82 GB 4:35/8.59 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.38 GB, 16.80 GB, 17.40 GB, 18.16 GB, 19.30 GB, 19.93 GB, 20.23 GB, 20.52 GB, 20.70 GB, 20.93 GB, 21.14 GB, 21.25 GB, 21.34 GB, 21.49 GB, 21.57 GB, 21.69 GB, 21.89 GB, 21.92 GB, 22.17 GB, 22.42 GB, 22.77 GB, 22.97 GB, 23.59 GB, 24.60 GB, 26.33 GB, 27.80 GB, 30.95 GB, 33.04 GB, 34.17 GB, 34.42 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB - FooterProcessingTime: (Avg: 2s229ms ; Min: 30.256ms ; Max: 8s076ms ; Number of samples: 177) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.51 GB (37054700787) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m55s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 177 (177) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.98 GB (15009454381) - PerReadThreadRawHdfsThroughput: 49.50 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 605.25M (605254874) - RowsReturned: 605.04M (605041882) - RowsReturnedRate: 74.66 M/sec - ScanRangesComplete: 89 (89) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m32s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 33s927ms - TotalRawHdfsReadTime(*): 11m53s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 25.129ms, non-child: 25.129ms, % non-child: 100.00%) - CodegenTime: 376.240us - CompileTime: 3.938ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.721ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 13.935ms Instance 844a0200d7876345:20bb38b900000077 (host=vd1114.foo.com:22000):(Total: 246.512ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:20/4.95 GB 1:41/10.01 GB 5:29/7.18 GB 3:27/6.58 GB 4:19/4.63 GB 7:30/7.24 GB 2:34/8.26 GB 6:24/5.86 GB Fragment Instance Lifecycle Event Timeline: 1s528ms - Prepare Finished: 213.287ms (213.287ms) - Open Finished: 246.962ms (33.675ms) - First Batch Produced: 1s527ms (1s280ms) - First Batch Sent: 1s528ms (1.275ms) MemoryUsage(8m32s): 13.31 GB, 13.96 GB, 13.96 GB, 13.91 GB, 13.65 GB, 13.33 GB, 13.25 GB, 13.12 GB, 12.74 GB, 11.96 GB, 11.39 GB, 10.88 GB, 10.50 GB, 10.35 GB, 10.17 GB, 9.82 GB, 9.27 GB, 8.93 GB, 8.76 GB, 8.63 GB, 8.42 GB, 8.21 GB, 8.12 GB, 8.51 GB, 9.09 GB, 10.31 GB, 11.88 GB, 13.38 GB, 13.95 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (14995638852) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.72 GB (172570493482) - RowsProduced: 602.34M (602338508) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h13m - TotalStorageWaitTime: 24m17s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 33.668ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 31.996ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 13s778ms - OpenTime: 33.669ms - ExecTreeOpenTime: 162.485us - PrepareTime: 212.758ms - ExecTreePrepareTime: 1.870ms KrpcDataStreamSender (dst_id=1):(Total: 11m9s, non-child: 11m9s, % non-child: 100.00%) - BytesSent: 79.05 GB (84877019937) - EosSent: 0 (0) - OverallThroughput: 120.88 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 602.34M (602337484) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m8s - UncompressedRowBatchSize: 155.27 GB (166717869358) HDFS_SCAN_NODE (id=0):(Total: 13s307ms, non-child: 13s307ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:20/4.95 GB 1:41/10.01 GB 5:29/7.18 GB 3:27/6.58 GB 4:19/4.63 GB 7:30/7.24 GB 2:34/8.26 GB 6:24/5.86 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.47 GB, 16.78 GB, 17.38 GB, 18.05 GB, 19.30 GB, 19.86 GB, 20.21 GB, 20.48 GB, 20.61 GB, 20.88 GB, 21.09 GB, 21.16 GB, 21.27 GB, 21.43 GB, 21.49 GB, 21.49 GB, 21.59 GB, 22.11 GB, 22.40 GB, 22.62 GB, 22.84 GB, 23.17 GB, 23.61 GB, 24.71 GB, 25.80 GB, 27.95 GB, 30.66 GB, 33.28 GB, 34.32 GB, 34.47 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB - FooterProcessingTime: (Avg: 1s880ms ; Min: 51.726ms ; Max: 10s778ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.54 GB (37087864210) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 11m7s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.96 GB (14992604870) - PerReadThreadRawHdfsThroughput: 61.21 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 602.55M (602551500) - RowsReturned: 602.34M (602338508) - RowsReturnedRate: 45.26 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 10m10s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 21s477ms - TotalRawHdfsReadTime(*): 9m37s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 33.156ms, non-child: 33.156ms, % non-child: 100.00%) - CodegenTime: 447.368us - CompileTime: 3.971ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.896ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 21.788ms Instance 844a0200d7876345:20bb38b90000000e (host=vc1531.foo.com:22000):(Total: 246.156ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:20/4.84 GB 7:25/6.19 GB 1:24/5.86 GB 2:27/6.43 GB 3:27/6.61 GB 6:29/7.06 GB 0:40/9.79 GB 5:32/7.81 GB Fragment Instance Lifecycle Event Timeline: 1s922ms - Prepare Finished: 216.757ms (216.757ms) - Open Finished: 246.628ms (29.870ms) - First Batch Produced: 1s920ms (1s674ms) - First Batch Sent: 1s922ms (1.728ms) MemoryUsage(8m32s): 13.06 GB, 13.97 GB, 13.97 GB, 13.95 GB, 13.82 GB, 13.58 GB, 13.32 GB, 13.21 GB, 13.03 GB, 12.66 GB, 11.91 GB, 11.44 GB, 10.97 GB, 10.54 GB, 10.30 GB, 10.12 GB, 9.89 GB, 9.51 GB, 8.87 GB, 8.40 GB, 8.63 GB, 8.54 GB, 8.42 GB, 8.58 GB, 8.61 GB, 9.12 GB, 9.73 GB, 11.01 GB, 13.00 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.98 GB (15006486029) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.28 GB (188201747221) - RowsProduced: 592.52M (592517678) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 34m4s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 29.866ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 28.996ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7s991ms - OpenTime: 29.868ms - ExecTreeOpenTime: 192.883us - PrepareTime: 216.218ms - ExecTreePrepareTime: 2.056ms KrpcDataStreamSender (dst_id=1):(Total: 11m20s, non-child: 11m20s, % non-child: 100.00%) - BytesSent: 77.79 GB (83527038389) - EosSent: 0 (0) - OverallThroughput: 116.98 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 592.52M (592516654) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m19s - UncompressedRowBatchSize: 152.74 GB (163998341004) HDFS_SCAN_NODE (id=0):(Total: 7s429ms, non-child: 7s429ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:20/4.84 GB 7:25/6.19 GB 1:24/5.86 GB 2:27/6.43 GB 3:27/6.61 GB 6:29/7.06 GB 0:40/9.79 GB 5:32/7.81 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.78 GB, 16.31 GB, 17.13 GB, 17.80 GB, 18.62 GB, 19.67 GB, 20.00 GB, 20.33 GB, 20.58 GB, 20.70 GB, 20.93 GB, 21.17 GB, 21.23 GB, 21.31 GB, 21.46 GB, 21.54 GB, 21.54 GB, 21.57 GB, 21.71 GB, 21.97 GB, 22.60 GB, 22.84 GB, 23.22 GB, 24.11 GB, 24.78 GB, 25.90 GB, 27.01 GB, 29.26 GB, 32.79 GB, 34.26 GB, 34.31 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB, 34.32 GB - FooterProcessingTime: (Avg: 2s926ms ; Min: 26.240ms ; Max: 17s005ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.32 GB (36845985075) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 14m10s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (15003382428) - PerReadThreadRawHdfsThroughput: 42.49 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 592.73M (592730670) - RowsReturned: 592.52M (592517678) - RowsReturnedRate: 79.76 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m7s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 21s790ms - TotalRawHdfsReadTime(*): 13m46s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 29.379ms, non-child: 29.379ms, % non-child: 100.00%) - CodegenTime: 463.506us - CompileTime: 5.046ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.172ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 16.610ms Instance 844a0200d7876345:20bb38b90000007b (host=vd1126.foo.com:22000):(Total: 246.075ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:23/5.63 GB 6:22/5.32 GB 0:31/7.59 GB 5:29/7.10 GB 7:32/7.89 GB 1:27/6.55 GB 3:31/7.52 GB 4:29/7.11 GB Fragment Instance Lifecycle Event Timeline: 1s961ms - Prepare Finished: 210.517ms (210.517ms) - Open Finished: 246.788ms (36.271ms) - First Batch Produced: 1s959ms (1s712ms) - First Batch Sent: 1s961ms (1.976ms) MemoryUsage(8m32s): 13.12 GB, 13.91 GB, 13.90 GB, 13.86 GB, 13.70 GB, 13.36 GB, 13.22 GB, 13.18 GB, 12.97 GB, 12.42 GB, 11.68 GB, 11.27 GB, 10.71 GB, 10.48 GB, 10.29 GB, 10.14 GB, 9.80 GB, 9.18 GB, 8.71 GB, 8.74 GB, 8.73 GB, 8.44 GB, 8.37 GB, 8.53 GB, 8.75 GB, 9.63 GB, 11.05 GB, 13.01 GB, 13.93 GB, 13.96 GB, 13.96 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.96 GB (14985227881) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.70 GB (172551802940) - RowsProduced: 600.73M (600732664) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 28m27s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 36.267ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 33.995ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 11s155ms - OpenTime: 36.268ms - ExecTreeOpenTime: 138.313us - PrepareTime: 209.684ms - ExecTreePrepareTime: 1.757ms KrpcDataStreamSender (dst_id=1):(Total: 11m23s, non-child: 11m23s, % non-child: 100.00%) - BytesSent: 78.89 GB (84704638797) - EosSent: 0 (0) - OverallThroughput: 118.21 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 600.73M (600731640) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m22s - UncompressedRowBatchSize: 154.85 GB (166272666283) HDFS_SCAN_NODE (id=0):(Total: 10s685ms, non-child: 10s685ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:23/5.63 GB 6:22/5.32 GB 0:31/7.59 GB 5:29/7.10 GB 7:32/7.89 GB 1:27/6.55 GB 3:31/7.52 GB 4:29/7.11 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.97 GB, 16.56 GB, 17.20 GB, 17.79 GB, 18.89 GB, 19.70 GB, 20.04 GB, 20.50 GB, 20.63 GB, 20.81 GB, 21.12 GB, 21.22 GB, 21.28 GB, 21.46 GB, 21.60 GB, 21.68 GB, 21.76 GB, 21.86 GB, 22.12 GB, 22.58 GB, 22.91 GB, 23.07 GB, 23.64 GB, 24.55 GB, 25.17 GB, 26.74 GB, 29.26 GB, 32.67 GB, 34.27 GB, 34.42 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB, 34.49 GB - FooterProcessingTime: (Avg: 2s260ms ; Min: 21.427ms ; Max: 17s106ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.49 GB (37029439267) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m58s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.95 GB (14982091212) - PerReadThreadRawHdfsThroughput: 46.32 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 600.95M (600945656) - RowsReturned: 600.73M (600732664) - RowsReturnedRate: 56.22 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m6s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 18s070ms - TotalRawHdfsReadTime(*): 12m42s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 35.830ms, non-child: 35.830ms, % non-child: 100.00%) - CodegenTime: 407.698us - CompileTime: 4.301ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.466ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 23.495ms Instance 844a0200d7876345:20bb38b90000002b (host=va1029.foo.com:22000):(Total: 245.877ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:32/7.69 GB 7:27/6.25 GB 4:28/6.81 GB 2:25/6.07 GB 1:29/6.90 GB 3:29/7.09 GB 0:28/6.90 GB 6:29/7.04 GB Fragment Instance Lifecycle Event Timeline: 1s784ms - Prepare Finished: 208.688ms (208.688ms) - Open Finished: 246.545ms (37.856ms) - First Batch Produced: 1s783ms (1s536ms) - First Batch Sent: 1s784ms (1.948ms) MemoryUsage(8m32s): 13.08 GB, 13.84 GB, 13.89 GB, 13.83 GB, 13.70 GB, 13.44 GB, 13.34 GB, 13.27 GB, 12.99 GB, 12.34 GB, 11.67 GB, 11.23 GB, 10.73 GB, 10.55 GB, 10.50 GB, 10.35 GB, 10.01 GB, 9.28 GB, 8.74 GB, 8.77 GB, 8.55 GB, 8.22 GB, 8.05 GB, 7.97 GB, 8.74 GB, 9.43 GB, 11.07 GB, 13.20 GB, 13.70 GB, 13.69 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB, 13.68 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.90 GB (14924557447) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.11 GB (188019547778) - RowsProduced: 599.83M (599834878) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 25m32s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 37.852ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 35.994ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s460ms - OpenTime: 37.854ms - ExecTreeOpenTime: 264.989us - PrepareTime: 207.950ms - ExecTreePrepareTime: 2.287ms KrpcDataStreamSender (dst_id=1):(Total: 11m12s, non-child: 11m12s, % non-child: 100.00%) - BytesSent: 78.74 GB (84551758459) - EosSent: 0 (0) - OverallThroughput: 119.90 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 599.83M (599833854) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m11s - UncompressedRowBatchSize: 154.62 GB (166024016095) HDFS_SCAN_NODE (id=0):(Total: 8s934ms, non-child: 8s934ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:32/7.69 GB 7:27/6.25 GB 4:28/6.81 GB 2:25/6.07 GB 1:29/6.90 GB 3:29/7.09 GB 0:28/6.90 GB 6:29/7.04 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.95 GB, 16.53 GB, 17.17 GB, 17.74 GB, 19.05 GB, 19.86 GB, 20.17 GB, 20.52 GB, 20.66 GB, 20.88 GB, 21.21 GB, 21.29 GB, 21.37 GB, 21.54 GB, 21.72 GB, 21.91 GB, 21.97 GB, 21.98 GB, 22.12 GB, 22.45 GB, 22.66 GB, 22.78 GB, 23.14 GB, 23.69 GB, 25.18 GB, 26.47 GB, 29.40 GB, 33.13 GB, 33.95 GB, 34.04 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB - FooterProcessingTime: (Avg: 1s926ms ; Min: 52.602ms ; Max: 12s208ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.13 GB (36645484646) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m38s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.90 GB (14921457917) - PerReadThreadRawHdfsThroughput: 51.14 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 600.05M (600047870) - RowsReturned: 599.83M (599834878) - RowsReturnedRate: 67.14 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m40s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 24s052ms - TotalRawHdfsReadTime(*): 11m23s - TotalReadThroughput: 1.21 MB/sec CodeGen:(Total: 37.225ms, non-child: 37.225ms, % non-child: 100.00%) - CodegenTime: 575.934us - CompileTime: 4.479ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 9.116ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 23.079ms Instance 844a0200d7876345:20bb38b90000005b (host=vc1522.foo.com:22000):(Total: 245.530ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:18/4.39 GB 1:33/7.67 GB 2:24/5.94 GB 6:26/6.12 GB 3:37/9.09 GB 7:30/7.31 GB 5:37/8.95 GB 4:22/5.39 GB Fragment Instance Lifecycle Event Timeline: 2s163ms - Prepare Finished: 218.758ms (218.758ms) - Open Finished: 245.970ms (27.212ms) - First Batch Produced: 2s161ms (1s915ms) - First Batch Sent: 2s163ms (2.172ms) MemoryUsage(8m32s): 13.16 GB, 13.94 GB, 13.94 GB, 13.92 GB, 13.75 GB, 13.40 GB, 13.19 GB, 13.16 GB, 12.99 GB, 12.57 GB, 11.76 GB, 11.30 GB, 10.73 GB, 10.37 GB, 10.28 GB, 10.12 GB, 9.83 GB, 9.46 GB, 9.09 GB, 8.83 GB, 8.97 GB, 8.73 GB, 8.40 GB, 8.07 GB, 8.24 GB, 8.90 GB, 9.81 GB, 12.06 GB, 13.41 GB, 13.81 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB, 13.80 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.94 GB (14970386452) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.54 GB (172381036288) - RowsProduced: 592.02M (592022167) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 26m36s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 27.208ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 26.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s874ms - OpenTime: 27.209ms - ExecTreeOpenTime: 252.080us - PrepareTime: 218.250ms - ExecTreePrepareTime: 1.884ms KrpcDataStreamSender (dst_id=1):(Total: 11m14s, non-child: 11m14s, % non-child: 100.00%) - BytesSent: 77.71 GB (83440898281) - EosSent: 0 (0) - OverallThroughput: 118.06 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 592.02M (592021143) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m12s - UncompressedRowBatchSize: 152.61 GB (163861635245) HDFS_SCAN_NODE (id=0):(Total: 9s337ms, non-child: 9s337ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:18/4.39 GB 1:33/7.67 GB 2:24/5.94 GB 6:26/6.12 GB 3:37/9.09 GB 7:30/7.31 GB 5:37/8.95 GB 4:22/5.39 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.78 GB, 16.40 GB, 17.11 GB, 17.62 GB, 18.65 GB, 19.57 GB, 19.89 GB, 20.27 GB, 20.42 GB, 20.57 GB, 20.88 GB, 21.02 GB, 21.08 GB, 21.19 GB, 21.37 GB, 21.41 GB, 21.42 GB, 21.74 GB, 22.31 GB, 22.46 GB, 22.97 GB, 23.20 GB, 23.25 GB, 23.48 GB, 24.12 GB, 25.38 GB, 27.03 GB, 30.98 GB, 33.30 GB, 34.11 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB, 34.17 GB - FooterProcessingTime: (Avg: 2s103ms ; Min: 10.498ms ; Max: 8s647ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.17 GB (36689851997) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m31s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.94 GB (14967245962) - PerReadThreadRawHdfsThroughput: 53.96 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 592.24M (592235159) - RowsReturned: 592.02M (592022167) - RowsReturnedRate: 63.41 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m52s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 25s051ms - TotalRawHdfsReadTime(*): 10m48s - TotalReadThroughput: 1.21 MB/sec CodeGen:(Total: 26.731ms, non-child: 26.731ms, % non-child: 100.00%) - CodegenTime: 310.150us - CompileTime: 3.978ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.566ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 15.630ms Instance 844a0200d7876345:20bb38b90000007a (host=vd1131.foo.com:22000):(Total: 245.468ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:23/5.46 GB 3:34/8.33 GB 6:30/7.28 GB 4:32/7.81 GB 7:33/8.06 GB 2:21/5.16 GB 5:23/5.63 GB 0:28/6.89 GB Fragment Instance Lifecycle Event Timeline: 1s821ms - Prepare Finished: 214.787ms (214.787ms) - Open Finished: 246.056ms (31.269ms) - First Batch Produced: 1s819ms (1s573ms) - First Batch Sent: 1s821ms (2.128ms) MemoryUsage(8m32s): 12.10 GB, 12.72 GB, 12.71 GB, 12.65 GB, 12.38 GB, 12.14 GB, 12.09 GB, 11.85 GB, 11.17 GB, 10.50 GB, 9.92 GB, 9.58 GB, 9.50 GB, 9.22 GB, 8.76 GB, 8.03 GB, 7.91 GB, 7.65 GB, 7.39 GB, 6.99 GB, 7.48 GB, 8.35 GB, 9.98 GB, 12.27 GB, 12.63 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.72 GB (13654700423) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.40 GB (171150354649) - RowsProduced: 587.92M (587919723) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 22m15s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 31.266ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 31.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s040ms - OpenTime: 31.268ms - ExecTreeOpenTime: 299.119us - PrepareTime: 214.124ms - ExecTreePrepareTime: 1.831ms KrpcDataStreamSender (dst_id=1):(Total: 11m23s, non-child: 11m23s, % non-child: 100.00%) - BytesSent: 77.18 GB (82866488353) - EosSent: 0 (0) - OverallThroughput: 115.61 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 587.92M (587918699) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m22s - UncompressedRowBatchSize: 151.55 GB (162725809499) HDFS_SCAN_NODE (id=0):(Total: 7s523ms, non-child: 7s523ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:23/5.46 GB 3:34/8.33 GB 6:30/7.28 GB 4:32/7.81 GB 7:33/8.06 GB 2:21/5.16 GB 5:23/5.63 GB 0:28/6.89 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 11.95 GB, 15.21 GB, 15.82 GB, 16.54 GB, 17.80 GB, 18.22 GB, 18.60 GB, 18.72 GB, 18.96 GB, 19.23 GB, 19.31 GB, 19.45 GB, 19.61 GB, 19.62 GB, 19.76 GB, 20.00 GB, 20.24 GB, 20.42 GB, 20.74 GB, 20.92 GB, 22.14 GB, 23.68 GB, 26.61 GB, 30.53 GB, 31.18 GB, 31.29 GB, 31.40 GB, 31.58 GB, 31.87 GB, 31.98 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB, 32.07 GB - FooterProcessingTime: (Avg: 2s050ms ; Min: 35.905ms ; Max: 13s715ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.07 GB (34431415956) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m54s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.71 GB (13651568110) - PerReadThreadRawHdfsThroughput: 51.94 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 588.12M (588124523) - RowsReturned: 587.92M (587919723) - RowsReturnedRate: 78.14 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m5s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 27s013ms - TotalRawHdfsReadTime(*): 10m32s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 30.633ms, non-child: 30.633ms, % non-child: 100.00%) - CodegenTime: 442.872us - CompileTime: 4.153ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.789ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 19.164ms Instance 844a0200d7876345:20bb38b900000006 (host=va1044.foo.com:22000):(Total: 244.866ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:25/6.12 GB 6:24/5.87 GB 5:24/5.91 GB 2:23/5.57 GB 3:38/9.29 GB 0:33/8.12 GB 1:33/7.91 GB 4:23/5.66 GB Fragment Instance Lifecycle Event Timeline: 1s898ms - Prepare Finished: 214.048ms (214.048ms) - Open Finished: 245.352ms (31.304ms) - First Batch Produced: 1s896ms (1s650ms) - First Batch Sent: 1s898ms (1.968ms) MemoryUsage(8m32s): 13.07 GB, 13.95 GB, 13.94 GB, 13.88 GB, 13.75 GB, 13.48 GB, 13.24 GB, 13.09 GB, 12.86 GB, 12.49 GB, 11.83 GB, 11.29 GB, 10.85 GB, 10.42 GB, 10.21 GB, 10.05 GB, 9.86 GB, 9.44 GB, 8.91 GB, 8.75 GB, 8.78 GB, 8.79 GB, 8.80 GB, 8.66 GB, 8.92 GB, 9.48 GB, 10.09 GB, 11.72 GB, 12.86 GB, 13.75 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.96 GB (14985536199) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.37 GB (188304308648) - RowsProduced: 593.31M (593308865) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 33m33s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 31.293ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 30.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 15s319ms - OpenTime: 31.294ms - ExecTreeOpenTime: 151.210us - PrepareTime: 213.493ms - ExecTreePrepareTime: 2.350ms KrpcDataStreamSender (dst_id=1):(Total: 11m5s, non-child: 11m5s, % non-child: 100.00%) - BytesSent: 77.83 GB (83568598797) - EosSent: 0 (0) - OverallThroughput: 119.75 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 593.31M (593307841) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m4s - UncompressedRowBatchSize: 152.94 GB (164216791028) HDFS_SCAN_NODE (id=0):(Total: 14s827ms, non-child: 14s827ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:25/6.12 GB 6:24/5.87 GB 5:24/5.91 GB 2:23/5.57 GB 3:38/9.29 GB 0:33/8.12 GB 1:33/7.91 GB 4:23/5.66 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.91 GB, 16.37 GB, 17.19 GB, 17.83 GB, 18.69 GB, 19.60 GB, 19.94 GB, 20.26 GB, 20.50 GB, 20.63 GB, 20.87 GB, 21.04 GB, 21.13 GB, 21.21 GB, 21.39 GB, 21.56 GB, 21.70 GB, 21.71 GB, 21.86 GB, 22.34 GB, 22.83 GB, 23.33 GB, 23.78 GB, 24.26 GB, 25.12 GB, 26.36 GB, 27.50 GB, 30.41 GB, 32.38 GB, 33.99 GB, 34.39 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB, 34.40 GB - FooterProcessingTime: (Avg: 2s869ms ; Min: 23.636ms ; Max: 12s576ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.40 GB (36935914776) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 12m9s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.95 GB (14982395755) - PerReadThreadRawHdfsThroughput: 44.62 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 593.52M (593521857) - RowsReturned: 593.31M (593308865) - RowsReturnedRate: 40.01 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m33s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 20s171ms - TotalRawHdfsReadTime(*): 13m9s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 30.793ms, non-child: 30.793ms, % non-child: 100.00%) - CodegenTime: 478.359us - CompileTime: 4.334ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.780ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 18.047ms Instance 844a0200d7876345:20bb38b90000007e (host=vd1130.foo.com:22000):(Total: 244.232ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:25/6.19 GB 3:39/9.58 GB 1:32/7.76 GB 6:33/8.13 GB 4:16/3.93 GB 5:24/5.82 GB 7:27/6.63 GB 2:29/6.95 GB Fragment Instance Lifecycle Event Timeline: 1s939ms - Prepare Finished: 207.911ms (207.911ms) - Open Finished: 244.727ms (36.816ms) - First Batch Produced: 1s937ms (1s692ms) - First Batch Sent: 1s939ms (2.300ms) MemoryUsage(8m32s): 12.10 GB, 12.74 GB, 12.70 GB, 12.51 GB, 12.25 GB, 12.14 GB, 11.82 GB, 11.21 GB, 10.52 GB, 9.97 GB, 9.66 GB, 9.51 GB, 9.23 GB, 8.66 GB, 7.93 GB, 7.55 GB, 7.26 GB, 7.01 GB, 7.24 GB, 8.05 GB, 9.33 GB, 11.18 GB, 12.28 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.75 GB (13685338184) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.44 GB (171200212402) - RowsProduced: 606.38M (606384861) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 24m1s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 36.813ms - TotalThreadsSysTime: 999.000us - TotalThreadsUserTime: 34.995ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s518ms - OpenTime: 36.815ms - ExecTreeOpenTime: 153.290us - PrepareTime: 207.344ms - ExecTreePrepareTime: 1.780ms KrpcDataStreamSender (dst_id=1):(Total: 11m28s, non-child: 11m28s, % non-child: 100.00%) - BytesSent: 79.62 GB (85494333235) - EosSent: 0 (0) - OverallThroughput: 118.42 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 606.38M (606383837) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m27s - UncompressedRowBatchSize: 156.31 GB (167836183696) HDFS_SCAN_NODE (id=0):(Total: 8s002ms, non-child: 8s002ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:25/6.19 GB 3:39/9.58 GB 1:32/7.76 GB 6:33/8.13 GB 4:16/3.93 GB 5:24/5.82 GB 7:27/6.63 GB 2:29/6.95 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.62 GB, 15.57 GB, 16.28 GB, 17.46 GB, 18.21 GB, 18.58 GB, 18.81 GB, 19.03 GB, 19.29 GB, 19.41 GB, 19.55 GB, 19.71 GB, 19.90 GB, 19.90 GB, 19.90 GB, 20.03 GB, 20.24 GB, 20.57 GB, 21.62 GB, 23.31 GB, 25.62 GB, 28.75 GB, 30.63 GB, 31.37 GB, 31.47 GB, 31.62 GB, 31.84 GB, 32.03 GB, 32.29 GB, 32.65 GB, 32.93 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB, 32.94 GB - FooterProcessingTime: (Avg: 1s903ms ; Min: 31.021ms ; Max: 11s109ms ; Number of samples: 161) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.94 GB (35368988008) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m19s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 161 (161) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.74 GB (13682230126) - PerReadThreadRawHdfsThroughput: 49.42 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 606.59M (606589661) - RowsReturned: 606.38M (606384861) - RowsReturnedRate: 75.77 M/sec - ScanRangesComplete: 81 (81) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m35s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 20s553ms - TotalRawHdfsReadTime(*): 11m22s - TotalReadThroughput: 1.17 MB/sec CodeGen:(Total: 36.161ms, non-child: 36.161ms, % non-child: 100.00%) - CodegenTime: 741.147us - CompileTime: 4.318ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.712ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.790ms Instance 844a0200d7876345:20bb38b90000003e (host=vd1337.foo.com:22000):(Total: 244.191ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:34/8.38 GB 7:28/6.77 GB 4:28/6.74 GB 3:27/6.61 GB 2:30/7.35 GB 6:21/5.16 GB 1:31/7.54 GB 5:25/6.11 GB Fragment Instance Lifecycle Event Timeline: 1s534ms - Prepare Finished: 210.092ms (210.092ms) - Open Finished: 244.684ms (34.591ms) - First Batch Produced: 1s533ms (1s288ms) - First Batch Sent: 1s534ms (1.704ms) MemoryUsage(8m32s): 11.96 GB, 12.68 GB, 12.65 GB, 12.56 GB, 12.30 GB, 12.03 GB, 11.89 GB, 11.51 GB, 10.91 GB, 10.31 GB, 9.87 GB, 9.53 GB, 9.36 GB, 8.96 GB, 8.50 GB, 8.12 GB, 7.95 GB, 7.91 GB, 7.76 GB, 7.72 GB, 8.47 GB, 9.34 GB, 10.71 GB, 11.82 GB, 12.60 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.70 GB (13639258808) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.44 GB (171201769455) - RowsProduced: 593.45M (593448367) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 28m46s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 34.589ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 33.995ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 16s925ms - OpenTime: 34.590ms - ExecTreeOpenTime: 157.254us - PrepareTime: 209.522ms - ExecTreePrepareTime: 1.846ms KrpcDataStreamSender (dst_id=1):(Total: 11m31s, non-child: 11m31s, % non-child: 100.00%) - BytesSent: 77.88 GB (83627542732) - EosSent: 0 (0) - OverallThroughput: 115.33 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 593.45M (593447343) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m30s - UncompressedRowBatchSize: 152.98 GB (164256439799) HDFS_SCAN_NODE (id=0):(Total: 16s392ms, non-child: 16s392ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:34/8.38 GB 7:28/6.77 GB 4:28/6.74 GB 3:27/6.61 GB 2:30/7.35 GB 6:21/5.16 GB 1:31/7.54 GB 5:25/6.11 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.20 GB, 15.28 GB, 15.99 GB, 16.84 GB, 17.82 GB, 18.29 GB, 18.57 GB, 18.77 GB, 19.00 GB, 19.26 GB, 19.45 GB, 19.57 GB, 19.67 GB, 19.71 GB, 19.87 GB, 20.14 GB, 20.61 GB, 21.17 GB, 21.60 GB, 22.09 GB, 23.74 GB, 25.41 GB, 27.78 GB, 29.75 GB, 31.15 GB, 31.44 GB, 31.64 GB, 31.89 GB, 32.09 GB, 32.28 GB, 32.42 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB, 32.43 GB - FooterProcessingTime: (Avg: 3s093ms ; Min: 31.130ms ; Max: 19s663ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.43 GB (34818929652) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m14s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.70 GB (13636044619) - PerReadThreadRawHdfsThroughput: 48.34 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 593.65M (593653167) - RowsReturned: 593.45M (593448367) - RowsReturnedRate: 36.20 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m41s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 24s590ms - TotalRawHdfsReadTime(*): 11m26s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 34.114ms, non-child: 34.114ms, % non-child: 100.00%) - CodegenTime: 421.164us - CompileTime: 4.529ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.368ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 21.648ms Instance 844a0200d7876345:20bb38b900000036 (host=va1042.foo.com:22000):(Total: 243.889ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:30/7.35 GB 4:19/4.62 GB 3:25/6.11 GB 5:29/7.07 GB 0:34/8.30 GB 1:29/7.11 GB 7:28/6.77 GB 6:30/7.26 GB Fragment Instance Lifecycle Event Timeline: 1s521ms - Prepare Finished: 205.621ms (205.621ms) - Open Finished: 244.785ms (39.164ms) - First Batch Produced: 1s520ms (1s275ms) - First Batch Sent: 1s521ms (1.503ms) MemoryUsage(8m32s): 13.27 GB, 13.96 GB, 13.96 GB, 13.92 GB, 13.74 GB, 13.40 GB, 13.24 GB, 13.21 GB, 13.01 GB, 12.50 GB, 11.67 GB, 11.24 GB, 10.65 GB, 10.39 GB, 10.31 GB, 10.14 GB, 9.85 GB, 9.31 GB, 9.08 GB, 9.02 GB, 8.84 GB, 8.46 GB, 8.12 GB, 8.02 GB, 8.28 GB, 9.11 GB, 10.58 GB, 12.54 GB, 13.96 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (14995824945) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.69 GB (172544268435) - RowsProduced: 595.33M (595326692) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 24m43s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 39.161ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 36.994ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 6s472ms - OpenTime: 39.162ms - ExecTreeOpenTime: 241.517us - PrepareTime: 204.651ms - ExecTreePrepareTime: 1.835ms KrpcDataStreamSender (dst_id=1):(Total: 11m9s, non-child: 11m9s, % non-child: 100.00%) - BytesSent: 78.14 GB (83903708514) - EosSent: 0 (0) - OverallThroughput: 119.51 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 595.33M (595325668) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m8s - UncompressedRowBatchSize: 153.46 GB (164775299314) HDFS_SCAN_NODE (id=0):(Total: 5s941ms, non-child: 5s941ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:30/7.35 GB 4:19/4.62 GB 3:25/6.11 GB 5:29/7.07 GB 0:34/8.30 GB 1:29/7.11 GB 7:28/6.77 GB 6:30/7.26 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.94 GB, 16.52 GB, 17.16 GB, 17.68 GB, 18.82 GB, 19.65 GB, 19.97 GB, 20.33 GB, 20.46 GB, 20.63 GB, 20.94 GB, 21.05 GB, 21.12 GB, 21.26 GB, 21.43 GB, 21.50 GB, 21.64 GB, 21.85 GB, 22.52 GB, 22.73 GB, 22.87 GB, 22.89 GB, 22.99 GB, 23.54 GB, 24.33 GB, 25.79 GB, 28.39 GB, 31.84 GB, 34.24 GB, 34.40 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB - FooterProcessingTime: (Avg: 1s933ms ; Min: 46.474ms ; Max: 9s661ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.48 GB (37025015180) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 11m24s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.96 GB (14992747255) - PerReadThreadRawHdfsThroughput: 53.75 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 595.54M (595539684) - RowsReturned: 595.33M (595326692) - RowsReturnedRate: 100.20 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m17s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 31s393ms - TotalRawHdfsReadTime(*): 10m56s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 38.642ms, non-child: 38.642ms, % non-child: 100.00%) - CodegenTime: 485.778us - CompileTime: 5.214ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.358ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 24.536ms Instance 844a0200d7876345:20bb38b900000035 (host=vc1515.foo.com:22000):(Total: 243.347ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:21/5.11 GB 1:30/7.20 GB 7:25/6.15 GB 2:26/6.36 GB 3:27/6.34 GB 6:35/8.47 GB 5:35/8.59 GB 0:28/6.79 GB Fragment Instance Lifecycle Event Timeline: 1s521ms - Prepare Finished: 217.724ms (217.724ms) - Open Finished: 243.906ms (26.181ms) - First Batch Produced: 1s518ms (1s275ms) - First Batch Sent: 1s521ms (2.079ms) MemoryUsage(8m32s): 13.22 GB, 13.97 GB, 13.95 GB, 13.85 GB, 13.60 GB, 13.29 GB, 13.21 GB, 13.03 GB, 12.61 GB, 11.83 GB, 11.29 GB, 10.80 GB, 10.44 GB, 10.27 GB, 10.11 GB, 9.94 GB, 9.48 GB, 8.87 GB, 8.77 GB, 8.58 GB, 8.36 GB, 8.25 GB, 8.27 GB, 8.83 GB, 9.64 GB, 10.87 GB, 12.65 GB, 13.75 GB, 13.86 GB, 13.85 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (14998270773) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.60 GB (172438098193) - RowsProduced: 608.93M (608933795) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h17m - TotalStorageWaitTime: 26m11s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 26.178ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 24.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s783ms - OpenTime: 26.180ms - ExecTreeOpenTime: 145.672us - PrepareTime: 217.078ms - ExecTreePrepareTime: 1.836ms KrpcDataStreamSender (dst_id=1):(Total: 11m13s, non-child: 11m13s, % non-child: 100.00%) - BytesSent: 79.95 GB (85848168261) - EosSent: 0 (0) - OverallThroughput: 121.61 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 608.93M (608932771) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m11s - UncompressedRowBatchSize: 156.97 GB (168542326832) HDFS_SCAN_NODE (id=0):(Total: 9s252ms, non-child: 9s252ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:21/5.11 GB 1:30/7.20 GB 7:25/6.15 GB 2:26/6.36 GB 3:27/6.34 GB 6:35/8.47 GB 5:35/8.59 GB 0:28/6.79 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.39 GB, 16.82 GB, 17.41 GB, 18.14 GB, 19.34 GB, 19.88 GB, 20.21 GB, 20.47 GB, 20.61 GB, 20.87 GB, 21.07 GB, 21.14 GB, 21.24 GB, 21.40 GB, 21.53 GB, 21.80 GB, 21.95 GB, 22.16 GB, 22.54 GB, 22.69 GB, 22.96 GB, 23.40 GB, 24.03 GB, 25.30 GB, 26.78 GB, 29.00 GB, 32.11 GB, 34.03 GB, 34.32 GB, 34.45 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB, 34.54 GB - FooterProcessingTime: (Avg: 1s989ms ; Min: 30.291ms ; Max: 8s537ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.54 GB (37087025053) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 6m32s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (14995121396) - PerReadThreadRawHdfsThroughput: 47.99 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 609.15M (609146787) - RowsReturned: 608.93M (608933795) - RowsReturnedRate: 65.81 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m43s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 27s716ms - TotalRawHdfsReadTime(*): 12m16s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 25.707ms, non-child: 25.707ms, % non-child: 100.00%) - CodegenTime: 478.264us - CompileTime: 3.971ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.779ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.505ms Instance 844a0200d7876345:20bb38b90000004d (host=ve1140.foo.com:22000):(Total: 243.221ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 3:36/8.74 GB 6:28/6.90 GB 0:26/6.28 GB 4:29/7.14 GB 7:23/5.61 GB 5:21/5.09 GB 1:24/5.90 GB 2:38/9.18 GB Fragment Instance Lifecycle Event Timeline: 1s674ms - Prepare Finished: 212.874ms (212.874ms) - Open Finished: 243.896ms (31.022ms) - First Batch Produced: 1s672ms (1s428ms) - First Batch Sent: 1s674ms (2.111ms) MemoryUsage(8m32s): 12.07 GB, 12.70 GB, 12.70 GB, 12.66 GB, 12.39 GB, 12.12 GB, 12.06 GB, 11.86 GB, 11.27 GB, 10.56 GB, 10.00 GB, 9.60 GB, 9.46 GB, 9.26 GB, 8.80 GB, 8.17 GB, 7.92 GB, 7.77 GB, 7.48 GB, 7.25 GB, 7.46 GB, 8.32 GB, 9.75 GB, 11.64 GB, 12.53 GB, 12.69 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.70 GB (13640354262) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.44 GB (171202174685) - RowsProduced: 585.58M (585583420) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h17m - TotalStorageWaitTime: 23m37s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 31.030ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 28.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7s400ms - OpenTime: 31.032ms - ExecTreeOpenTime: 315.675us - PrepareTime: 212.116ms - ExecTreePrepareTime: 2.168ms KrpcDataStreamSender (dst_id=1):(Total: 11m38s, non-child: 11m38s, % non-child: 100.00%) - BytesSent: 76.88 GB (82552921489) - EosSent: 0 (0) - OverallThroughput: 112.70 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 585.58M (585582396) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m37s - UncompressedRowBatchSize: 150.95 GB (162079455163) HDFS_SCAN_NODE (id=0):(Total: 6s852ms, non-child: 6s852ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:36/8.74 GB 6:28/6.90 GB 0:26/6.28 GB 4:29/7.14 GB 7:23/5.61 GB 5:21/5.09 GB 1:24/5.90 GB 2:38/9.18 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 11.92 GB, 15.13 GB, 15.74 GB, 16.41 GB, 17.63 GB, 18.13 GB, 18.49 GB, 18.66 GB, 18.87 GB, 19.14 GB, 19.23 GB, 19.35 GB, 19.51 GB, 19.56 GB, 19.56 GB, 19.81 GB, 20.18 GB, 20.44 GB, 20.65 GB, 21.06 GB, 21.90 GB, 23.54 GB, 26.06 GB, 29.32 GB, 30.89 GB, 31.27 GB, 31.41 GB, 31.60 GB, 31.82 GB, 32.01 GB, 32.11 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB - FooterProcessingTime: (Avg: 1s963ms ; Min: 21.299ms ; Max: 8s749ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.12 GB (34484753756) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 5m15s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.70 GB (13637262945) - PerReadThreadRawHdfsThroughput: 48.86 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 585.79M (585788220) - RowsReturned: 585.58M (585583420) - RowsReturnedRate: 85.45 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m13s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 21s366ms - TotalRawHdfsReadTime(*): 11m13s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 30.425ms, non-child: 30.425ms, % non-child: 100.00%) - CodegenTime: 418.216us - CompileTime: 4.108ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.254ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 19.471ms Instance 844a0200d7876345:20bb38b900000039 (host=ve1137.foo.com:22000):(Total: 242.249ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:26/6.40 GB 4:34/8.24 GB 6:27/6.56 GB 3:27/6.61 GB 0:33/8.06 GB 1:28/6.89 GB 5:25/6.19 GB 2:24/5.82 GB Fragment Instance Lifecycle Event Timeline: 2s171ms - Prepare Finished: 216.951ms (216.951ms) - Open Finished: 242.721ms (25.769ms) - First Batch Produced: 2s169ms (1s926ms) - First Batch Sent: 2s171ms (1.800ms) MemoryUsage(8m32s): 12.07 GB, 12.72 GB, 12.72 GB, 12.65 GB, 12.37 GB, 12.16 GB, 12.09 GB, 11.78 GB, 11.03 GB, 10.42 GB, 9.84 GB, 9.60 GB, 9.48 GB, 9.21 GB, 8.66 GB, 8.03 GB, 7.84 GB, 7.54 GB, 7.20 GB, 7.05 GB, 7.63 GB, 8.58 GB, 10.35 GB, 12.33 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.72 GB (13661512901) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.44 GB (171194539085) - RowsProduced: 589.90M (589901122) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 23m38s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 25.754ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 24.996ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s344ms - OpenTime: 25.769ms - ExecTreeOpenTime: 272.718us - PrepareTime: 216.406ms - ExecTreePrepareTime: 2.068ms KrpcDataStreamSender (dst_id=1):(Total: 11m34s, non-child: 11m34s, % non-child: 100.00%) - BytesSent: 77.40 GB (83108614798) - EosSent: 0 (0) - OverallThroughput: 114.15 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 589.90M (589900098) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m32s - UncompressedRowBatchSize: 152.06 GB (163273547815) HDFS_SCAN_NODE (id=0):(Total: 8s795ms, non-child: 8s795ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:26/6.40 GB 4:34/8.24 GB 6:27/6.56 GB 3:27/6.61 GB 0:33/8.06 GB 1:28/6.89 GB 5:25/6.19 GB 2:24/5.82 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.03 GB, 15.28 GB, 15.86 GB, 16.66 GB, 17.88 GB, 18.30 GB, 18.65 GB, 18.79 GB, 19.05 GB, 19.27 GB, 19.36 GB, 19.52 GB, 19.64 GB, 19.74 GB, 19.81 GB, 20.05 GB, 20.23 GB, 20.38 GB, 20.58 GB, 21.10 GB, 22.41 GB, 24.10 GB, 27.25 GB, 30.61 GB, 31.25 GB, 31.35 GB, 31.48 GB, 31.68 GB, 31.90 GB, 32.05 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB, 32.16 GB - FooterProcessingTime: (Avg: 2s259ms ; Min: 19.254ms ; Max: 12s539ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.16 GB (34530457600) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m9s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.72 GB (13658298708) - PerReadThreadRawHdfsThroughput: 49.42 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 590.11M (590105922) - RowsReturned: 589.90M (589901122) - RowsReturnedRate: 67.07 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m26s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 18s714ms - TotalRawHdfsReadTime(*): 11m6s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 25.187ms, non-child: 25.187ms, % non-child: 100.00%) - CodegenTime: 377.067us - CompileTime: 3.887ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.800ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 13.927ms Instance 844a0200d7876345:20bb38b900000040 (host=ve1142.foo.com:22000):(Total: 242.134ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:26/6.39 GB 3:26/6.40 GB 6:27/6.53 GB 2:36/8.64 GB 4:34/8.31 GB 7:20/4.84 GB 1:31/7.48 GB 5:24/5.91 GB Fragment Instance Lifecycle Event Timeline: 1s746ms - Prepare Finished: 212.761ms (212.761ms) - Open Finished: 242.529ms (29.768ms) - First Batch Produced: 1s744ms (1s501ms) - First Batch Sent: 1s746ms (1.661ms) MemoryUsage(8m32s): 12.06 GB, 12.69 GB, 12.64 GB, 12.46 GB, 12.14 GB, 11.99 GB, 11.78 GB, 11.28 GB, 10.53 GB, 9.99 GB, 9.52 GB, 9.43 GB, 9.21 GB, 8.87 GB, 8.40 GB, 8.07 GB, 7.80 GB, 7.66 GB, 7.50 GB, 7.96 GB, 8.76 GB, 10.19 GB, 11.69 GB, 12.63 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.70 GB (13638935426) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.43 GB (171185773237) - RowsProduced: 600.79M (600786603) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 24m4s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 29.765ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 26.995ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s365ms - OpenTime: 29.766ms - ExecTreeOpenTime: 203.739us - PrepareTime: 212.307ms - ExecTreePrepareTime: 2.397ms KrpcDataStreamSender (dst_id=1):(Total: 11m21s, non-child: 11m21s, % non-child: 100.00%) - BytesSent: 78.90 GB (84718840998) - EosSent: 0 (0) - OverallThroughput: 118.50 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 600.79M (600785579) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m20s - UncompressedRowBatchSize: 154.87 GB (166287597486) HDFS_SCAN_NODE (id=0):(Total: 9s837ms, non-child: 9s837ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:26/6.39 GB 3:26/6.40 GB 6:27/6.53 GB 2:36/8.64 GB 4:34/8.31 GB 7:20/4.84 GB 1:31/7.48 GB 5:24/5.91 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.56 GB, 15.47 GB, 16.09 GB, 17.14 GB, 18.00 GB, 18.38 GB, 18.64 GB, 18.80 GB, 19.07 GB, 19.21 GB, 19.31 GB, 19.58 GB, 19.69 GB, 19.83 GB, 20.20 GB, 20.53 GB, 20.65 GB, 21.05 GB, 21.55 GB, 22.84 GB, 24.35 GB, 26.92 GB, 29.56 GB, 31.22 GB, 31.40 GB, 31.51 GB, 31.70 GB, 31.92 GB, 32.16 GB, 32.43 GB, 32.65 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB, 32.66 GB - FooterProcessingTime: (Avg: 2s067ms ; Min: 17.670ms ; Max: 9s131ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.66 GB (35068451973) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m7s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.70 GB (13635783241) - PerReadThreadRawHdfsThroughput: 48.38 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 600.99M (600991403) - RowsReturned: 600.79M (600786603) - RowsReturnedRate: 61.07 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m23s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 22s996ms - TotalRawHdfsReadTime(*): 11m31s - TotalReadThroughput: 1.16 MB/sec CodeGen:(Total: 29.309ms, non-child: 29.309ms, % non-child: 100.00%) - CodegenTime: 373.574us - CompileTime: 4.006ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.408ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 18.461ms Instance 844a0200d7876345:20bb38b900000002 (host=vd1118.foo.com:22000):(Total: 241.749ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:30/7.32 GB 2:30/7.34 GB 5:30/7.38 GB 0:31/7.55 GB 6:29/7.03 GB 3:24/5.86 GB 7:20/4.95 GB 4:30/7.32 GB Fragment Instance Lifecycle Event Timeline: 1s597ms - Prepare Finished: 207.944ms (207.944ms) - Open Finished: 242.435ms (34.491ms) - First Batch Produced: 1s595ms (1s352ms) - First Batch Sent: 1s597ms (2.219ms) MemoryUsage(8m32s): 13.33 GB, 13.98 GB, 13.98 GB, 13.94 GB, 13.76 GB, 13.46 GB, 13.33 GB, 13.27 GB, 12.98 GB, 12.41 GB, 11.66 GB, 11.23 GB, 10.72 GB, 10.49 GB, 10.49 GB, 10.25 GB, 9.92 GB, 9.22 GB, 8.72 GB, 8.70 GB, 8.50 GB, 8.20 GB, 7.88 GB, 7.89 GB, 8.71 GB, 9.59 GB, 11.14 GB, 13.13 GB, 13.93 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15018395381) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.36 GB (188294284108) - RowsProduced: 599.49M (599494893) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 23m46s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 34.486ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 31.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s826ms - OpenTime: 34.487ms - ExecTreeOpenTime: 169.454us - PrepareTime: 207.183ms - ExecTreePrepareTime: 1.773ms KrpcDataStreamSender (dst_id=1):(Total: 11m30s, non-child: 11m30s, % non-child: 100.00%) - BytesSent: 78.70 GB (84499417363) - EosSent: 0 (0) - OverallThroughput: 116.76 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 599.49M (599493869) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m28s - UncompressedRowBatchSize: 154.53 GB (165930567742) HDFS_SCAN_NODE (id=0):(Total: 8s323ms, non-child: 8s323ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:30/7.32 GB 2:30/7.34 GB 5:30/7.38 GB 0:31/7.55 GB 6:29/7.03 GB 3:24/5.86 GB 7:20/4.95 GB 4:30/7.32 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.16 GB, 16.62 GB, 17.24 GB, 17.81 GB, 19.01 GB, 19.79 GB, 20.11 GB, 20.46 GB, 20.59 GB, 20.79 GB, 21.10 GB, 21.18 GB, 21.26 GB, 21.41 GB, 21.66 GB, 21.74 GB, 21.85 GB, 21.85 GB, 22.06 GB, 22.37 GB, 22.58 GB, 22.80 GB, 22.87 GB, 23.57 GB, 25.19 GB, 26.75 GB, 29.52 GB, 32.99 GB, 34.30 GB, 34.41 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB, 34.51 GB - FooterProcessingTime: (Avg: 1s876ms ; Min: 23.105ms ; Max: 7s502ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.51 GB (37054687130) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m49s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.98 GB (15015320427) - PerReadThreadRawHdfsThroughput: 55.55 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 599.71M (599707885) - RowsReturned: 599.49M (599494893) - RowsReturnedRate: 72.02 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 30s914ms - TotalRawHdfsReadTime(*): 10m36s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 33.991ms, non-child: 33.991ms, % non-child: 100.00%) - CodegenTime: 419.082us - CompileTime: 4.203ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.640ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 21.668ms Instance 844a0200d7876345:20bb38b90000006a (host=vc1535.foo.com:22000):(Total: 241.741ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:24/5.86 GB 6:28/6.82 GB 4:25/6.08 GB 3:31/7.27 GB 0:20/4.83 GB 7:32/7.81 GB 1:41/10.00 GB 5:26/6.24 GB Fragment Instance Lifecycle Event Timeline: 1s584ms - Prepare Finished: 213.999ms (213.999ms) - Open Finished: 242.459ms (28.460ms) - First Batch Produced: 1s582ms (1s340ms) - First Batch Sent: 1s584ms (2.005ms) MemoryUsage(8m32s): 13.18 GB, 13.94 GB, 13.94 GB, 13.92 GB, 13.75 GB, 13.40 GB, 13.20 GB, 13.16 GB, 12.93 GB, 12.40 GB, 11.65 GB, 11.16 GB, 10.65 GB, 10.37 GB, 10.25 GB, 10.03 GB, 9.72 GB, 9.33 GB, 8.88 GB, 8.88 GB, 8.87 GB, 8.52 GB, 8.18 GB, 8.05 GB, 8.54 GB, 9.32 GB, 10.73 GB, 12.66 GB, 13.68 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.96 GB (14991483427) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.71 GB (172564369436) - RowsProduced: 594.29M (594292435) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 26m1s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 28.432ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 27.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s285ms - OpenTime: 28.434ms - ExecTreeOpenTime: 280.962us - PrepareTime: 213.233ms - ExecTreePrepareTime: 1.878ms KrpcDataStreamSender (dst_id=1):(Total: 11m18s, non-child: 11m18s, % non-child: 100.00%) - BytesSent: 78.02 GB (83770591110) - EosSent: 0 (0) - OverallThroughput: 117.68 MB/sec - PeakMemoryUsage: 3.58 MB (3752608) - RowsReturned: 594.29M (594291411) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m17s - UncompressedRowBatchSize: 153.19 GB (164489418800) HDFS_SCAN_NODE (id=0):(Total: 7s745ms, non-child: 7s745ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:24/5.86 GB 6:28/6.82 GB 4:25/6.08 GB 3:31/7.27 GB 0:20/4.83 GB 7:32/7.81 GB 1:41/10.00 GB 5:26/6.24 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.89 GB, 16.45 GB, 17.13 GB, 17.68 GB, 18.75 GB, 19.62 GB, 19.93 GB, 20.29 GB, 20.44 GB, 20.62 GB, 20.90 GB, 21.02 GB, 21.09 GB, 21.23 GB, 21.36 GB, 21.40 GB, 21.45 GB, 21.81 GB, 22.08 GB, 22.58 GB, 22.97 GB, 22.98 GB, 23.07 GB, 23.52 GB, 24.72 GB, 26.13 GB, 28.60 GB, 31.96 GB, 33.77 GB, 34.30 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB - FooterProcessingTime: (Avg: 1s855ms ; Min: 27.768ms ; Max: 8s949ms ; Number of samples: 177) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.41 GB (36951891699) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m56s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 177 (177) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.96 GB (14988345941) - PerReadThreadRawHdfsThroughput: 52.49 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 594.51M (594505427) - RowsReturned: 594.29M (594292435) - RowsReturnedRate: 76.73 M/sec - ScanRangesComplete: 89 (89) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m11s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 24s876ms - TotalRawHdfsReadTime(*): 11m11s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 27.898ms, non-child: 27.898ms, % non-child: 100.00%) - CodegenTime: 347.850us - CompileTime: 4.433ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.063ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 15.869ms Instance 844a0200d7876345:20bb38b900000022 (host=va1030.foo.com:22000):(Total: 241.266ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:22/5.41 GB 6:33/8.13 GB 2:33/8.01 GB 5:31/7.38 GB 0:21/5.16 GB 3:31/7.59 GB 1:24/5.87 GB 4:27/6.57 GB Fragment Instance Lifecycle Event Timeline: 1s767ms - Prepare Finished: 215.342ms (215.342ms) - Open Finished: 242.049ms (26.706ms) - First Batch Produced: 1s765ms (1s523ms) - First Batch Sent: 1s767ms (2.061ms) MemoryUsage(8m32s): 13.11 GB, 13.88 GB, 13.95 GB, 13.89 GB, 13.69 GB, 13.39 GB, 13.27 GB, 13.09 GB, 12.72 GB, 12.02 GB, 11.41 GB, 10.95 GB, 10.64 GB, 10.52 GB, 10.28 GB, 9.98 GB, 9.46 GB, 8.98 GB, 8.75 GB, 8.64 GB, 8.34 GB, 8.03 GB, 8.28 GB, 8.67 GB, 9.62 GB, 10.94 GB, 12.33 GB, 13.60 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB, 13.91 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (15003118278) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.33 GB (188264018705) - RowsProduced: 608.22M (608220887) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 28m49s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 26.719ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 24.996ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 12s633ms - OpenTime: 26.719ms - ExecTreeOpenTime: 144.950us - PrepareTime: 214.467ms - ExecTreePrepareTime: 1.827ms KrpcDataStreamSender (dst_id=1):(Total: 11m10s, non-child: 11m10s, % non-child: 100.00%) - BytesSent: 79.86 GB (85746113276) - EosSent: 0 (0) - OverallThroughput: 121.98 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 608.22M (608219863) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m9s - UncompressedRowBatchSize: 156.78 GB (168345154270) HDFS_SCAN_NODE (id=0):(Total: 12s127ms, non-child: 12s127ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:22/5.41 GB 6:33/8.13 GB 2:33/8.01 GB 5:31/7.38 GB 0:21/5.16 GB 3:31/7.59 GB 1:24/5.87 GB 4:27/6.57 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.30 GB, 16.74 GB, 17.44 GB, 18.14 GB, 19.32 GB, 19.93 GB, 20.28 GB, 20.56 GB, 20.71 GB, 20.97 GB, 21.18 GB, 21.26 GB, 21.49 GB, 21.68 GB, 21.76 GB, 21.85 GB, 21.92 GB, 22.13 GB, 22.47 GB, 22.73 GB, 22.90 GB, 23.07 GB, 24.03 GB, 25.09 GB, 26.81 GB, 29.15 GB, 31.60 GB, 33.78 GB, 34.37 GB, 34.51 GB, 34.60 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB, 34.61 GB - FooterProcessingTime: (Avg: 2s250ms ; Min: 21.274ms ; Max: 10s620ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.61 GB (37159022465) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m47s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (15000018777) - PerReadThreadRawHdfsThroughput: 48.61 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 608.43M (608433879) - RowsReturned: 608.22M (608220887) - RowsReturnedRate: 50.15 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m20s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 33s673ms - TotalRawHdfsReadTime(*): 12m9s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 26.297ms, non-child: 26.297ms, % non-child: 100.00%) - CodegenTime: 381.450us - CompileTime: 4.030ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.383ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 15.278ms Instance 844a0200d7876345:20bb38b900000050 (host=va1032.foo.com:22000):(Total: 241.213ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:29/7.06 GB 7:25/5.99 GB 3:30/7.39 GB 5:28/6.89 GB 4:31/7.40 GB 2:28/6.82 GB 1:23/5.62 GB 6:31/7.55 GB Fragment Instance Lifecycle Event Timeline: 1s898ms - Prepare Finished: 206.257ms (206.257ms) - Open Finished: 241.693ms (35.435ms) - First Batch Produced: 1s896ms (1s654ms) - First Batch Sent: 1s898ms (1.750ms) MemoryUsage(8m32s): 13.25 GB, 13.98 GB, 13.99 GB, 13.95 GB, 13.80 GB, 13.50 GB, 13.34 GB, 13.29 GB, 13.10 GB, 12.54 GB, 11.79 GB, 11.35 GB, 10.82 GB, 10.52 GB, 10.43 GB, 10.28 GB, 9.95 GB, 9.51 GB, 8.94 GB, 8.76 GB, 8.60 GB, 8.29 GB, 8.03 GB, 7.83 GB, 7.98 GB, 8.81 GB, 9.93 GB, 12.06 GB, 13.60 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB, 13.89 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15018731358) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.65 GB (172496300525) - RowsProduced: 593.51M (593511899) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 25m52s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 35.426ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 5 (5) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s452ms - OpenTime: 35.427ms - ExecTreeOpenTime: 208.153us - PrepareTime: 205.720ms - ExecTreePrepareTime: 1.902ms KrpcDataStreamSender (dst_id=1):(Total: 11m4s, non-child: 11m4s, % non-child: 100.00%) - BytesSent: 77.92 GB (83669045998) - EosSent: 0 (0) - OverallThroughput: 120.10 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 593.51M (593510875) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m3s - UncompressedRowBatchSize: 152.99 GB (164274014044) HDFS_SCAN_NODE (id=0):(Total: 9s954ms, non-child: 9s954ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:29/7.06 GB 7:25/5.99 GB 3:30/7.39 GB 5:28/6.89 GB 4:31/7.40 GB 2:28/6.82 GB 1:23/5.62 GB 6:31/7.55 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.91 GB, 16.50 GB, 17.20 GB, 17.73 GB, 18.87 GB, 19.73 GB, 20.06 GB, 20.43 GB, 20.56 GB, 20.74 GB, 21.06 GB, 21.17 GB, 21.24 GB, 21.37 GB, 21.54 GB, 21.57 GB, 21.64 GB, 21.91 GB, 22.16 GB, 22.33 GB, 22.42 GB, 22.58 GB, 22.81 GB, 23.21 GB, 23.93 GB, 25.43 GB, 27.43 GB, 31.18 GB, 33.83 GB, 34.37 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB - FooterProcessingTime: (Avg: 2s000ms ; Min: 16.502ms ; Max: 7s983ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.41 GB (36951734516) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m54s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.98 GB (15015640149) - PerReadThreadRawHdfsThroughput: 47.47 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 593.72M (593724891) - RowsReturned: 593.51M (593511899) - RowsReturnedRate: 59.62 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m48s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 29s131ms - TotalRawHdfsReadTime(*): 12m22s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 34.915ms, non-child: 34.915ms, % non-child: 100.00%) - CodegenTime: 460.659us - CompileTime: 4.066ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.703ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 21.651ms Instance 844a0200d7876345:20bb38b90000001c (host=va1037.foo.com:22000):(Total: 241.190ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 6:25/6.07 GB 2:27/6.54 GB 5:30/7.39 GB 4:27/6.30 GB 0:30/7.30 GB 1:31/7.60 GB 3:29/7.14 GB 7:27/6.50 GB Fragment Instance Lifecycle Event Timeline: 1s647ms - Prepare Finished: 209.331ms (209.331ms) - Open Finished: 241.757ms (32.426ms) - First Batch Produced: 1s645ms (1s403ms) - First Batch Sent: 1s647ms (2.136ms) MemoryUsage(8m32s): 13.34 GB, 13.94 GB, 13.93 GB, 13.91 GB, 13.78 GB, 13.42 GB, 13.22 GB, 13.19 GB, 13.09 GB, 12.75 GB, 11.92 GB, 11.52 GB, 10.93 GB, 10.56 GB, 10.47 GB, 10.41 GB, 10.16 GB, 9.78 GB, 9.03 GB, 8.90 GB, 8.89 GB, 8.61 GB, 8.25 GB, 7.92 GB, 7.72 GB, 8.18 GB, 9.12 GB, 10.75 GB, 13.20 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.95 GB (14981929758) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.24 GB (188165586922) - RowsProduced: 585.46M (585456136) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h13m - TotalStorageWaitTime: 21m54s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 32.419ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 29.995ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 17s425ms - OpenTime: 32.420ms - ExecTreeOpenTime: 315.708us - PrepareTime: 208.694ms - ExecTreePrepareTime: 1.863ms KrpcDataStreamSender (dst_id=1):(Total: 11m13s, non-child: 11m13s, % non-child: 100.00%) - BytesSent: 76.92 GB (82595568674) - EosSent: 0 (0) - OverallThroughput: 117.02 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 585.46M (585455112) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m11s - UncompressedRowBatchSize: 150.92 GB (162044302759) HDFS_SCAN_NODE (id=0):(Total: 16s941ms, non-child: 16s941ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:25/6.07 GB 2:27/6.54 GB 5:30/7.39 GB 4:27/6.30 GB 0:30/7.30 GB 1:31/7.60 GB 3:29/7.14 GB 7:27/6.50 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.85 GB, 16.35 GB, 17.02 GB, 17.50 GB, 18.62 GB, 19.56 GB, 19.86 GB, 20.27 GB, 20.39 GB, 20.64 GB, 21.00 GB, 21.15 GB, 21.19 GB, 21.31 GB, 21.52 GB, 21.56 GB, 21.57 GB, 21.73 GB, 21.90 GB, 22.34 GB, 22.54 GB, 22.61 GB, 22.74 GB, 22.96 GB, 23.25 GB, 24.17 GB, 25.82 GB, 28.85 GB, 33.04 GB, 34.11 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB, 34.18 GB - FooterProcessingTime: (Avg: 1s816ms ; Min: 34.541ms ; Max: 7s608ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.18 GB (36697709161) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m51s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.95 GB (14978811864) - PerReadThreadRawHdfsThroughput: 59.13 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 585.67M (585669128) - RowsReturned: 585.46M (585456136) - RowsReturnedRate: 34.56 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 10m20s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 24s281ms - TotalRawHdfsReadTime(*): 9m51s - TotalReadThroughput: 1.21 MB/sec CodeGen:(Total: 31.749ms, non-child: 31.749ms, % non-child: 100.00%) - CodegenTime: 433.716us - CompileTime: 4.356ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.030ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 19.748ms Instance 844a0200d7876345:20bb38b900000078 (host=va1041.foo.com:22000):(Total: 240.704ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:21/5.13 GB 0:23/5.62 GB 1:29/6.97 GB 2:34/8.23 GB 7:35/8.58 GB 3:27/6.53 GB 5:21/5.13 GB 6:35/8.58 GB Fragment Instance Lifecycle Event Timeline: 1s798ms - Prepare Finished: 211.389ms (211.389ms) - Open Finished: 241.192ms (29.803ms) - First Batch Produced: 1s796ms (1s555ms) - First Batch Sent: 1s798ms (2.096ms) MemoryUsage(8m32s): 13.21 GB, 13.99 GB, 13.98 GB, 13.89 GB, 13.67 GB, 13.39 GB, 13.30 GB, 13.09 GB, 12.71 GB, 12.07 GB, 11.38 GB, 10.96 GB, 10.60 GB, 10.42 GB, 10.12 GB, 9.83 GB, 9.55 GB, 9.04 GB, 8.74 GB, 8.49 GB, 8.21 GB, 8.02 GB, 8.27 GB, 8.81 GB, 9.76 GB, 10.88 GB, 12.10 GB, 13.59 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 14.00 GB (15028755530) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.70 GB (172551290778) - RowsProduced: 609.07M (609066204) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 29m51s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 29.800ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 29.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s967ms - OpenTime: 29.801ms - ExecTreeOpenTime: 166.405us - PrepareTime: 210.794ms - ExecTreePrepareTime: 2.052ms KrpcDataStreamSender (dst_id=1):(Total: 11m21s, non-child: 11m21s, % non-child: 100.00%) - BytesSent: 79.98 GB (85872643421) - EosSent: 0 (0) - OverallThroughput: 120.19 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 609.07M (609065180) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m20s - UncompressedRowBatchSize: 157.00 GB (168579178587) HDFS_SCAN_NODE (id=0):(Total: 10s420ms, non-child: 10s420ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:21/5.13 GB 0:23/5.62 GB 1:29/6.97 GB 2:34/8.23 GB 7:35/8.58 GB 3:27/6.53 GB 5:21/5.13 GB 6:35/8.58 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.42 GB, 16.83 GB, 17.41 GB, 18.22 GB, 19.33 GB, 19.97 GB, 20.24 GB, 20.54 GB, 20.71 GB, 20.91 GB, 21.15 GB, 21.25 GB, 21.35 GB, 21.48 GB, 21.56 GB, 21.71 GB, 21.97 GB, 22.13 GB, 22.50 GB, 22.57 GB, 22.65 GB, 22.98 GB, 23.92 GB, 25.33 GB, 27.05 GB, 29.01 GB, 31.17 GB, 33.79 GB, 34.48 GB, 34.62 GB, 34.70 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB, 34.71 GB - FooterProcessingTime: (Avg: 1s976ms ; Min: 18.724ms ; Max: 7s187ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.71 GB (37264777841) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m17s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.99 GB (15025516923) - PerReadThreadRawHdfsThroughput: 51.83 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 609.28M (609279196) - RowsReturned: 609.07M (609066204) - RowsReturnedRate: 58.45 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m4s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 24s589ms - TotalRawHdfsReadTime(*): 11m25s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 29.356ms, non-child: 29.356ms, % non-child: 100.00%) - CodegenTime: 397.697us - CompileTime: 4.188ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.363ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 17.310ms Instance 844a0200d7876345:20bb38b900000058 (host=ve1134.foo.com:22000):(Total: 240.606ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 3:27/6.65 GB 2:30/7.39 GB 7:31/7.51 GB 1:19/4.70 GB 0:27/6.60 GB 5:31/7.56 GB 4:28/6.86 GB 6:31/7.55 GB Fragment Instance Lifecycle Event Timeline: 1s624ms - Prepare Finished: 201.504ms (201.504ms) - Open Finished: 241.202ms (39.698ms) - First Batch Produced: 1s622ms (1s381ms) - First Batch Sent: 1s624ms (2.062ms) MemoryUsage(8m32s): 11.95 GB, 12.66 GB, 12.60 GB, 12.42 GB, 12.15 GB, 11.91 GB, 11.69 GB, 11.29 GB, 10.58 GB, 10.10 GB, 9.66 GB, 9.37 GB, 9.19 GB, 9.04 GB, 8.67 GB, 8.07 GB, 7.92 GB, 7.81 GB, 7.88 GB, 8.12 GB, 8.77 GB, 9.42 GB, 10.87 GB, 12.15 GB, 12.70 GB, 12.71 GB, 12.70 GB, 12.70 GB, 12.69 GB, 12.69 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.72 GB (13655234881) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.46 GB (171213656099) - RowsProduced: 595.11M (595106080) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h13m - TotalStorageWaitTime: 27m40s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 39.695ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 37.994ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 13s075ms - OpenTime: 39.697ms - ExecTreeOpenTime: 155.138us - PrepareTime: 200.780ms - ExecTreePrepareTime: 1.869ms KrpcDataStreamSender (dst_id=1):(Total: 11m25s, non-child: 11m25s, % non-child: 100.00%) - BytesSent: 78.11 GB (83870452642) - EosSent: 0 (0) - OverallThroughput: 116.73 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 595.11M (595105056) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m23s - UncompressedRowBatchSize: 153.40 GB (164714544596) HDFS_SCAN_NODE (id=0):(Total: 12s550ms, non-child: 12s550ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:27/6.65 GB 2:30/7.39 GB 7:31/7.51 GB 1:19/4.70 GB 0:27/6.60 GB 5:31/7.56 GB 4:28/6.86 GB 6:31/7.55 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.36 GB, 15.37 GB, 16.06 GB, 16.91 GB, 17.86 GB, 18.24 GB, 18.54 GB, 18.70 GB, 18.92 GB, 19.23 GB, 19.35 GB, 19.48 GB, 19.72 GB, 20.03 GB, 20.28 GB, 20.38 GB, 20.76 GB, 21.17 GB, 21.84 GB, 22.85 GB, 24.25 GB, 25.47 GB, 28.10 GB, 30.33 GB, 31.31 GB, 31.46 GB, 31.57 GB, 31.79 GB, 32.07 GB, 32.34 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB, 32.52 GB - FooterProcessingTime: (Avg: 2s320ms ; Min: 40.061ms ; Max: 11s164ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.52 GB (34920893912) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m41s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.71 GB (13652131068) - PerReadThreadRawHdfsThroughput: 45.96 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 595.31M (595310880) - RowsReturned: 595.11M (595106080) - RowsReturnedRate: 47.42 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m10s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 22s033ms - TotalRawHdfsReadTime(*): 12m4s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 39.266ms, non-child: 39.266ms, % non-child: 100.00%) - CodegenTime: 377.439us - CompileTime: 5.015ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.141ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 25.624ms Instance 844a0200d7876345:20bb38b900000073 (host=vd1140.foo.com:22000):(Total: 240.286ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:25/6.15 GB 2:32/7.80 GB 5:23/5.65 GB 4:31/7.60 GB 7:20/4.83 GB 3:33/7.99 GB 1:30/7.29 GB 6:30/7.36 GB Fragment Instance Lifecycle Event Timeline: 1s665ms - Prepare Finished: 205.748ms (205.748ms) - Open Finished: 241.099ms (35.351ms) - First Batch Produced: 1s663ms (1s422ms) - First Batch Sent: 1s665ms (1.988ms) MemoryUsage(8m32s): 11.98 GB, 12.69 GB, 12.67 GB, 12.58 GB, 12.36 GB, 12.09 GB, 11.97 GB, 11.75 GB, 11.24 GB, 10.51 GB, 10.04 GB, 9.57 GB, 9.35 GB, 9.26 GB, 9.00 GB, 8.48 GB, 7.94 GB, 7.81 GB, 7.86 GB, 7.68 GB, 7.64 GB, 8.13 GB, 9.01 GB, 10.69 GB, 12.26 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.71 GB (13644019758) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.48 GB (171236344158) - RowsProduced: 581.84M (581841643) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 28m6s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 35.348ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 33.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s391ms - OpenTime: 35.349ms - ExecTreeOpenTime: 262.644us - PrepareTime: 204.845ms - ExecTreePrepareTime: 2.065ms KrpcDataStreamSender (dst_id=1):(Total: 11m30s, non-child: 11m30s, % non-child: 100.00%) - BytesSent: 76.36 GB (81991016001) - EosSent: 0 (0) - OverallThroughput: 113.26 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 581.84M (581840619) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m29s - UncompressedRowBatchSize: 149.98 GB (161043700102) HDFS_SCAN_NODE (id=0):(Total: 9s846ms, non-child: 9s846ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:25/6.15 GB 2:32/7.80 GB 5:23/5.65 GB 4:31/7.60 GB 7:20/4.83 GB 3:33/7.99 GB 1:30/7.29 GB 6:30/7.36 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 11.91 GB, 15.17 GB, 15.84 GB, 16.49 GB, 17.66 GB, 18.13 GB, 18.47 GB, 18.66 GB, 18.82 GB, 19.11 GB, 19.21 GB, 19.29 GB, 19.45 GB, 19.67 GB, 19.81 GB, 20.04 GB, 20.16 GB, 20.41 GB, 21.08 GB, 21.42 GB, 22.01 GB, 23.16 GB, 24.75 GB, 27.75 GB, 30.47 GB, 31.26 GB, 31.36 GB, 31.50 GB, 31.77 GB, 32.05 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB, 32.19 GB - FooterProcessingTime: (Avg: 2s849ms ; Min: 33.310ms ; Max: 13s345ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.19 GB (34568230983) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m25s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.70 GB (13641002134) - PerReadThreadRawHdfsThroughput: 51.58 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 582.05M (582046443) - RowsReturned: 581.84M (581841643) - RowsReturnedRate: 59.09 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m3s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 21s566ms - TotalRawHdfsReadTime(*): 10m39s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 34.832ms, non-child: 34.832ms, % non-child: 100.00%) - CodegenTime: 381.279us - CompileTime: 4.773ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.462ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 21.972ms Instance 844a0200d7876345:20bb38b90000003b (host=ve1129.foo.com:22000):(Total: 240.181ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 3:26/6.37 GB 2:30/7.22 GB 6:29/7.10 GB 4:24/5.86 GB 1:24/5.94 GB 0:35/8.41 GB 5:25/6.11 GB 7:32/7.82 GB Fragment Instance Lifecycle Event Timeline: 1s841ms - Prepare Finished: 206.681ms (206.681ms) - Open Finished: 240.642ms (33.961ms) - First Batch Produced: 1s839ms (1s599ms) - First Batch Sent: 1s841ms (2.105ms) MemoryUsage(8m32s): 12.14 GB, 12.73 GB, 12.72 GB, 12.64 GB, 12.34 GB, 12.12 GB, 12.00 GB, 11.69 GB, 10.97 GB, 10.34 GB, 9.82 GB, 9.55 GB, 9.35 GB, 9.04 GB, 8.45 GB, 7.87 GB, 7.65 GB, 7.47 GB, 7.31 GB, 7.42 GB, 8.01 GB, 9.11 GB, 10.68 GB, 12.32 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.73 GB (13665427323) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.43 GB (171181419357) - RowsProduced: 592.27M (592273469) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 21m53s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 33.959ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s394ms - OpenTime: 33.960ms - ExecTreeOpenTime: 166.140us - PrepareTime: 206.149ms - ExecTreePrepareTime: 2.091ms KrpcDataStreamSender (dst_id=1):(Total: 11m29s, non-child: 11m29s, % non-child: 100.00%) - BytesSent: 77.79 GB (83522585309) - EosSent: 0 (0) - OverallThroughput: 115.45 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 592.27M (592272445) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m28s - UncompressedRowBatchSize: 152.67 GB (163930114763) HDFS_SCAN_NODE (id=0):(Total: 8s871ms, non-child: 8s871ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:26/6.37 GB 2:30/7.22 GB 6:29/7.10 GB 4:24/5.86 GB 1:24/5.94 GB 0:35/8.41 GB 5:25/6.11 GB 7:32/7.82 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.20 GB, 15.28 GB, 15.91 GB, 16.73 GB, 17.88 GB, 18.29 GB, 18.63 GB, 18.79 GB, 19.03 GB, 19.26 GB, 19.35 GB, 19.49 GB, 19.61 GB, 19.62 GB, 19.65 GB, 19.94 GB, 20.16 GB, 20.47 GB, 20.96 GB, 21.77 GB, 23.10 GB, 25.01 GB, 27.77 GB, 30.58 GB, 31.24 GB, 31.36 GB, 31.52 GB, 31.72 GB, 31.97 GB, 32.19 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.32 GB, 32.33 GB, 32.33 GB, 32.33 GB, 32.33 GB, 32.33 GB, 32.33 GB, 32.33 GB, 32.33 GB, 32.33 GB - FooterProcessingTime: (Avg: 2s138ms ; Min: 28.338ms ; Max: 10s306ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.33 GB (34711803831) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m57s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.72 GB (13662360588) - PerReadThreadRawHdfsThroughput: 48.69 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 592.48M (592478269) - RowsReturned: 592.27M (592273469) - RowsReturnedRate: 66.76 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m10s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 27s769ms - TotalRawHdfsReadTime(*): 11m19s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 33.503ms, non-child: 33.503ms, % non-child: 100.00%) - CodegenTime: 363.612us - CompileTime: 4.458ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.474ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 20.953ms Instance 844a0200d7876345:20bb38b900000067 (host=vc1527.foo.com:22000):(Total: 240.154ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:25/6.03 GB 1:30/7.28 GB 6:29/7.08 GB 3:29/6.99 GB 4:27/6.61 GB 7:28/6.78 GB 2:26/6.39 GB 5:32/7.73 GB Fragment Instance Lifecycle Event Timeline: 1s442ms - Prepare Finished: 215.220ms (215.220ms) - Open Finished: 240.609ms (25.388ms) - First Batch Produced: 1s440ms (1s199ms) - First Batch Sent: 1s442ms (2.154ms) MemoryUsage(8m32s): 13.34 GB, 13.91 GB, 13.90 GB, 13.81 GB, 13.49 GB, 13.14 GB, 13.09 GB, 12.96 GB, 12.61 GB, 11.78 GB, 11.23 GB, 10.66 GB, 10.39 GB, 10.29 GB, 10.16 GB, 9.87 GB, 9.28 GB, 8.98 GB, 8.83 GB, 8.70 GB, 8.47 GB, 8.13 GB, 8.24 GB, 8.91 GB, 9.94 GB, 11.78 GB, 13.41 GB, 13.90 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.92 GB (14951156034) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.68 GB (172525369267) - RowsProduced: 609.27M (609272995) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h17m - TotalStorageWaitTime: 21m39s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 25.397ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 24.997ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s665ms - OpenTime: 25.414ms - ExecTreeOpenTime: 138.094us - PrepareTime: 214.664ms - ExecTreePrepareTime: 2.236ms KrpcDataStreamSender (dst_id=1):(Total: 11m5s, non-child: 11m5s, % non-child: 100.00%) - BytesSent: 80.00 GB (85896902007) - EosSent: 0 (0) - OverallThroughput: 123.16 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 609.27M (609271971) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m3s - UncompressedRowBatchSize: 157.05 GB (168636295649) HDFS_SCAN_NODE (id=0):(Total: 8s215ms, non-child: 8s215ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:25/6.03 GB 1:30/7.28 GB 6:29/7.08 GB 3:29/6.99 GB 4:27/6.61 GB 7:28/6.78 GB 2:26/6.39 GB 5:32/7.73 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.38 GB, 16.67 GB, 17.25 GB, 17.94 GB, 19.23 GB, 19.72 GB, 20.09 GB, 20.33 GB, 20.59 GB, 20.90 GB, 21.04 GB, 21.12 GB, 21.26 GB, 21.41 GB, 21.58 GB, 21.81 GB, 22.00 GB, 22.40 GB, 22.58 GB, 22.90 GB, 23.19 GB, 23.33 GB, 24.03 GB, 25.33 GB, 27.19 GB, 30.47 GB, 33.30 GB, 34.20 GB, 34.34 GB, 34.48 GB, 34.57 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB - FooterProcessingTime: (Avg: 1s651ms ; Min: 36.065ms ; Max: 6s365ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.59 GB (37137590293) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m29s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.92 GB (14948101535) - PerReadThreadRawHdfsThroughput: 56.83 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 609.49M (609485987) - RowsReturned: 609.27M (609272995) - RowsReturnedRate: 74.16 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m6s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 22s654ms - TotalRawHdfsReadTime(*): 10m23s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 24.987ms, non-child: 24.987ms, % non-child: 100.00%) - CodegenTime: 378.936us - CompileTime: 3.891ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.735ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 13.907ms Instance 844a0200d7876345:20bb38b900000066 (host=ve1127.foo.com:22000):(Total: 239.114ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 3:22/5.45 GB 4:25/6.11 GB 7:31/7.56 GB 0:25/6.03 GB 2:35/8.45 GB 5:25/6.08 GB 6:23/5.58 GB 1:38/9.33 GB Fragment Instance Lifecycle Event Timeline: 1s646ms - Prepare Finished: 199.974ms (199.974ms) - Open Finished: 239.635ms (39.660ms) - First Batch Produced: 1s644ms (1s404ms) - First Batch Sent: 1s646ms (2.133ms) MemoryUsage(8m32s): 12.18 GB, 12.74 GB, 12.74 GB, 12.67 GB, 12.38 GB, 12.20 GB, 12.16 GB, 11.88 GB, 11.12 GB, 10.49 GB, 9.89 GB, 9.66 GB, 9.60 GB, 9.32 GB, 8.74 GB, 8.08 GB, 7.85 GB, 7.45 GB, 7.00 GB, 6.70 GB, 7.21 GB, 8.39 GB, 10.42 GB, 12.55 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.74 GB (13678508667) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.42 GB (171172551918) - RowsProduced: 589.91M (589911871) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h18m - TotalStorageWaitTime: 20m3s - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 39.651ms - TotalThreadsSysTime: 3.000ms - TotalThreadsUserTime: 35.994ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7s295ms - OpenTime: 39.653ms - ExecTreeOpenTime: 216.370us - PrepareTime: 199.387ms - ExecTreePrepareTime: 1.950ms KrpcDataStreamSender (dst_id=1):(Total: 11m28s, non-child: 11m28s, % non-child: 100.00%) - BytesSent: 77.42 GB (83131209319) - EosSent: 0 (0) - OverallThroughput: 115.14 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 589.91M (589910847) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m27s - UncompressedRowBatchSize: 152.06 GB (163277153017) HDFS_SCAN_NODE (id=0):(Total: 6s699ms, non-child: 6s699ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:22/5.45 GB 4:25/6.11 GB 7:31/7.56 GB 0:25/6.03 GB 2:35/8.45 GB 5:25/6.08 GB 6:23/5.58 GB 1:38/9.33 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.16 GB, 15.28 GB, 15.83 GB, 16.64 GB, 17.90 GB, 18.31 GB, 18.66 GB, 18.78 GB, 19.07 GB, 19.31 GB, 19.38 GB, 19.56 GB, 19.70 GB, 19.73 GB, 19.86 GB, 20.07 GB, 20.13 GB, 20.13 GB, 20.26 GB, 20.57 GB, 21.75 GB, 23.84 GB, 27.41 GB, 31.05 GB, 31.27 GB, 31.36 GB, 31.47 GB, 31.66 GB, 31.89 GB, 31.99 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.10 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB - FooterProcessingTime: (Avg: 1s806ms ; Min: 18.881ms ; Max: 8s341ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.11 GB (34477299065) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 5m22s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.74 GB (13675278072) - PerReadThreadRawHdfsThroughput: 52.91 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 590.12M (590116671) - RowsReturned: 589.91M (589911871) - RowsReturnedRate: 88.05 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m34s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 23s355ms - TotalRawHdfsReadTime(*): 10m21s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 39.171ms, non-child: 39.171ms, % non-child: 100.00%) - CodegenTime: 350.156us - CompileTime: 4.502ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.776ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 27.346ms Instance 844a0200d7876345:20bb38b900000003 (host=vd1117.foo.com:22000):(Total: 238.770ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 6:22/5.45 GB 1:26/6.40 GB 7:26/6.37 GB 4:21/5.16 GB 5:15/3.71 GB 2:36/8.45 GB 3:41/10.00 GB 0:37/9.04 GB Fragment Instance Lifecycle Event Timeline: 1s757ms - Prepare Finished: 206.417ms (206.417ms) - Open Finished: 239.289ms (32.872ms) - First Batch Produced: 1s755ms (1s516ms) - First Batch Sent: 1s757ms (1.868ms) MemoryUsage(8m32s): 13.26 GB, 13.98 GB, 13.98 GB, 13.90 GB, 13.65 GB, 13.37 GB, 13.29 GB, 13.13 GB, 12.72 GB, 11.92 GB, 11.41 GB, 10.91 GB, 10.55 GB, 10.41 GB, 10.24 GB, 9.99 GB, 9.51 GB, 8.86 GB, 8.76 GB, 8.55 GB, 8.26 GB, 8.07 GB, 7.92 GB, 8.45 GB, 9.22 GB, 10.40 GB, 12.56 GB, 13.57 GB, 13.93 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB, 13.94 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15017721437) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.36 GB (188290961521) - RowsProduced: 605.30M (605304955) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 25m26s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 32.869ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 30.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s875ms - OpenTime: 32.871ms - ExecTreeOpenTime: 158.900us - PrepareTime: 205.822ms - ExecTreePrepareTime: 1.797ms KrpcDataStreamSender (dst_id=1):(Total: 11m5s, non-child: 11m5s, % non-child: 100.00%) - BytesSent: 79.50 GB (85363704725) - EosSent: 0 (0) - OverallThroughput: 122.26 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 605.30M (605303931) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m4s - UncompressedRowBatchSize: 156.03 GB (167537575630) HDFS_SCAN_NODE (id=0):(Total: 10s369ms, non-child: 10s369ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:22/5.45 GB 1:26/6.40 GB 7:26/6.37 GB 4:21/5.16 GB 5:15/3.71 GB 2:36/8.45 GB 3:41/10.00 GB 0:37/9.04 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.42 GB, 16.81 GB, 17.43 GB, 18.15 GB, 19.44 GB, 19.95 GB, 20.31 GB, 20.54 GB, 20.68 GB, 20.97 GB, 21.15 GB, 21.23 GB, 21.34 GB, 21.51 GB, 21.59 GB, 21.71 GB, 21.83 GB, 21.99 GB, 22.36 GB, 22.43 GB, 22.62 GB, 22.96 GB, 23.41 GB, 24.65 GB, 26.08 GB, 28.17 GB, 31.90 GB, 33.66 GB, 34.35 GB, 34.49 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.57 GB, 34.57 GB, 34.57 GB, 34.57 GB, 34.57 GB, 34.57 GB, 34.57 GB, 34.57 GB, 34.57 GB - FooterProcessingTime: (Avg: 2s011ms ; Min: 21.008ms ; Max: 8s964ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.57 GB (37117726091) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 11m22s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.98 GB (15014613711) - PerReadThreadRawHdfsThroughput: 48.62 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 605.52M (605517947) - RowsReturned: 605.30M (605304955) - RowsReturnedRate: 58.37 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 10m10s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 21s512ms - TotalRawHdfsReadTime(*): 12m8s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 32.430ms, non-child: 32.430ms, % non-child: 100.00%) - CodegenTime: 428.631us - CompileTime: 4.231ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.819ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 20.829ms Instance 844a0200d7876345:20bb38b900000043 (host=ve1138.foo.com:22000):(Total: 237.577ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:21/5.08 GB 5:34/8.30 GB 2:28/6.90 GB 4:31/7.29 GB 3:20/4.86 GB 7:23/5.65 GB 0:34/8.18 GB 6:34/8.26 GB Fragment Instance Lifecycle Event Timeline: 1s575ms - Prepare Finished: 202.934ms (202.934ms) - Open Finished: 238.025ms (35.091ms) - First Batch Produced: 1s573ms (1s335ms) - First Batch Sent: 1s575ms (2.071ms) MemoryUsage(8m32s): 12.17 GB, 12.65 GB, 12.63 GB, 12.47 GB, 12.06 GB, 11.86 GB, 11.79 GB, 11.40 GB, 10.55 GB, 9.93 GB, 9.35 GB, 9.18 GB, 9.11 GB, 8.93 GB, 8.68 GB, 8.34 GB, 8.35 GB, 8.00 GB, 7.60 GB, 7.67 GB, 8.47 GB, 9.79 GB, 11.77 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.70 GB (13639660718) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.45 GB (171208570279) - RowsProduced: 593.63M (593631423) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h17m - TotalStorageWaitTime: 17m46s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 35.088ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 31.995ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s119ms - OpenTime: 35.089ms - ExecTreeOpenTime: 168.795us - PrepareTime: 202.403ms - ExecTreePrepareTime: 2.132ms KrpcDataStreamSender (dst_id=1):(Total: 11m21s, non-child: 11m21s, % non-child: 100.00%) - BytesSent: 77.92 GB (83671185940) - EosSent: 0 (0) - OverallThroughput: 117.13 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 593.63M (593630399) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m19s - UncompressedRowBatchSize: 153.02 GB (164306781712) HDFS_SCAN_NODE (id=0):(Total: 8s623ms, non-child: 8s623ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:21/5.08 GB 5:34/8.30 GB 2:28/6.90 GB 4:31/7.29 GB 3:20/4.86 GB 7:23/5.65 GB 0:34/8.18 GB 6:34/8.26 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.14 GB, 15.19 GB, 15.75 GB, 16.56 GB, 17.66 GB, 18.06 GB, 18.39 GB, 18.51 GB, 18.79 GB, 18.94 GB, 19.02 GB, 19.20 GB, 19.37 GB, 19.68 GB, 20.37 GB, 20.79 GB, 21.18 GB, 21.30 GB, 21.38 GB, 22.09 GB, 23.60 GB, 25.94 GB, 29.49 GB, 31.13 GB, 31.26 GB, 31.41 GB, 31.58 GB, 31.81 GB, 31.96 GB, 32.08 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB, 32.22 GB - FooterProcessingTime: (Avg: 1s709ms ; Min: 18.734ms ; Max: 7s542ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.22 GB (34596520340) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 6m51s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.70 GB (13636356388) - PerReadThreadRawHdfsThroughput: 58.00 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 593.84M (593836223) - RowsReturned: 593.63M (593631423) - RowsReturnedRate: 68.84 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m5s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 22s062ms - TotalRawHdfsReadTime(*): 9m28s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 34.590ms, non-child: 34.590ms, % non-child: 100.00%) - CodegenTime: 467.727us - CompileTime: 4.363ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.193ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.390ms Instance 844a0200d7876345:20bb38b900000045 (host=vd1142.foo.com:22000):(Total: 237.339ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 3:30/7.32 GB 0:24/5.81 GB 4:25/6.02 GB 7:29/7.07 GB 2:30/7.34 GB 6:25/6.03 GB 5:25/6.11 GB 1:37/9.12 GB Fragment Instance Lifecycle Event Timeline: 1s500ms - Prepare Finished: 205.181ms (205.181ms) - Open Finished: 237.831ms (32.649ms) - First Batch Produced: 1s498ms (1s260ms) - First Batch Sent: 1s500ms (2.217ms) MemoryUsage(8m32s): 12.14 GB, 12.69 GB, 12.66 GB, 12.49 GB, 12.13 GB, 11.98 GB, 11.84 GB, 11.41 GB, 10.63 GB, 10.05 GB, 9.54 GB, 9.34 GB, 9.26 GB, 8.89 GB, 8.45 GB, 7.99 GB, 7.80 GB, 7.64 GB, 7.49 GB, 7.62 GB, 8.55 GB, 9.67 GB, 11.40 GB, 12.59 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.69 GB (13628045730) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.43 GB (171182603493) - RowsProduced: 595.13M (595125150) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 20m32s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 32.637ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 31.995ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 13s456ms - OpenTime: 32.638ms - ExecTreeOpenTime: 140.607us - PrepareTime: 204.621ms - ExecTreePrepareTime: 2.039ms KrpcDataStreamSender (dst_id=1):(Total: 11m39s, non-child: 11m39s, % non-child: 100.00%) - BytesSent: 78.16 GB (83926395369) - EosSent: 0 (0) - OverallThroughput: 114.44 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 595.12M (595124126) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m38s - UncompressedRowBatchSize: 153.41 GB (164721285494) HDFS_SCAN_NODE (id=0):(Total: 12s902ms, non-child: 12s902ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:30/7.32 GB 0:24/5.81 GB 4:25/6.02 GB 7:29/7.07 GB 2:30/7.34 GB 6:25/6.03 GB 5:25/6.11 GB 1:37/9.12 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.54 GB, 15.40 GB, 16.01 GB, 17.01 GB, 17.91 GB, 18.28 GB, 18.56 GB, 18.71 GB, 18.98 GB, 19.14 GB, 19.24 GB, 19.39 GB, 19.61 GB, 19.69 GB, 19.98 GB, 20.33 GB, 20.50 GB, 20.89 GB, 21.31 GB, 22.17 GB, 23.94 GB, 25.91 GB, 28.95 GB, 31.03 GB, 31.26 GB, 31.39 GB, 31.54 GB, 31.76 GB, 32.00 GB, 32.20 GB, 32.39 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB, 32.40 GB - FooterProcessingTime: (Avg: 1s715ms ; Min: 24.251ms ; Max: 8s861ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.40 GB (34786613497) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m13s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.69 GB (13624984877) - PerReadThreadRawHdfsThroughput: 55.04 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 595.33M (595329950) - RowsReturned: 595.13M (595125150) - RowsReturnedRate: 46.12 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m20s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 24s759ms - TotalRawHdfsReadTime(*): 10m2s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 32.161ms, non-child: 32.161ms, % non-child: 100.00%) - CodegenTime: 480.138us - CompileTime: 4.170ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.476ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 20.965ms Instance 844a0200d7876345:20bb38b900000069 (host=ve1125.foo.com:22000):(Total: 236.550ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:32/7.50 GB 7:21/5.16 GB 1:20/4.91 GB 2:32/7.59 GB 4:40/9.60 GB 6:35/8.51 GB 3:27/6.27 GB 5:22/5.40 GB Fragment Instance Lifecycle Event Timeline: 1s806ms - Prepare Finished: 202.563ms (202.563ms) - Open Finished: 237.021ms (34.457ms) - First Batch Produced: 1s805ms (1s568ms) - First Batch Sent: 1s806ms (1.310ms) MemoryUsage(8m32s): 11.96 GB, 12.62 GB, 12.66 GB, 12.57 GB, 12.20 GB, 11.91 GB, 11.88 GB, 11.65 GB, 11.01 GB, 10.28 GB, 9.74 GB, 9.35 GB, 9.26 GB, 9.10 GB, 8.76 GB, 8.35 GB, 8.25 GB, 8.11 GB, 7.84 GB, 7.46 GB, 7.60 GB, 8.58 GB, 10.15 GB, 11.73 GB, 12.45 GB, 12.43 GB, 12.38 GB, 12.37 GB, 12.37 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB, 12.49 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.67 GB (13599932128) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.24 GB (170986142182) - RowsProduced: 586.30M (586300309) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 22m - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 34.454ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 5 (5) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s801ms - OpenTime: 34.455ms - ExecTreeOpenTime: 289.479us - PrepareTime: 202.038ms - ExecTreePrepareTime: 2.281ms KrpcDataStreamSender (dst_id=1):(Total: 11m25s, non-child: 11m25s, % non-child: 100.00%) - BytesSent: 77.00 GB (82675566638) - EosSent: 0 (0) - OverallThroughput: 115.05 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 586.30M (586299285) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m24s - UncompressedRowBatchSize: 151.13 GB (162277994961) HDFS_SCAN_NODE (id=0):(Total: 8s309ms, non-child: 8s309ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:32/7.50 GB 7:21/5.16 GB 1:20/4.91 GB 2:32/7.59 GB 4:40/9.60 GB 6:35/8.51 GB 3:27/6.27 GB 5:22/5.40 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 11.85 GB, 15.09 GB, 15.72 GB, 16.47 GB, 17.60 GB, 18.06 GB, 18.40 GB, 18.56 GB, 18.79 GB, 19.02 GB, 19.11 GB, 19.24 GB, 19.38 GB, 19.51 GB, 19.81 GB, 20.29 GB, 20.77 GB, 21.08 GB, 21.38 GB, 21.41 GB, 22.11 GB, 23.91 GB, 26.72 GB, 29.50 GB, 30.78 GB, 30.92 GB, 31.08 GB, 31.30 GB, 31.51 GB, 31.76 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB, 31.78 GB - FooterProcessingTime: (Avg: 1s644ms ; Min: 19.595ms ; Max: 9s633ms ; Number of samples: 161) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 31.78 GB (34127103903) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m1s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 161 (161) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.66 GB (13596799822) - PerReadThreadRawHdfsThroughput: 52.63 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 586.51M (586505109) - RowsReturned: 586.30M (586300309) - RowsReturnedRate: 70.56 M/sec - ScanRangesComplete: 81 (81) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m41s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 25s124ms - TotalRawHdfsReadTime(*): 10m18s - TotalReadThroughput: 1.13 MB/sec CodeGen:(Total: 33.911ms, non-child: 33.911ms, % non-child: 100.00%) - CodegenTime: 365.945us - CompileTime: 4.970ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.520ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 19.775ms Instance 844a0200d7876345:20bb38b900000042 (host=ve1128.foo.com:22000):(Total: 236.189ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:23/5.55 GB 0:28/6.82 GB 4:40/9.73 GB 3:33/8.01 GB 5:30/7.36 GB 6:29/7.10 GB 2:20/4.87 GB 7:22/5.37 GB Fragment Instance Lifecycle Event Timeline: 1s635ms - Prepare Finished: 201.599ms (201.599ms) - Open Finished: 236.640ms (35.041ms) - First Batch Produced: 1s632ms (1s395ms) - First Batch Sent: 1s635ms (2.901ms) MemoryUsage(8m32s): 12.14 GB, 12.69 GB, 12.68 GB, 12.57 GB, 12.26 GB, 12.03 GB, 12.00 GB, 11.73 GB, 11.02 GB, 10.34 GB, 9.75 GB, 9.43 GB, 9.36 GB, 9.07 GB, 8.76 GB, 8.25 GB, 8.24 GB, 7.92 GB, 7.52 GB, 7.24 GB, 7.59 GB, 8.66 GB, 10.22 GB, 12.51 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB, 12.68 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.69 GB (13627074672) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.42 GB (171177553509) - RowsProduced: 589.35M (589350889) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 20m26s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 35.027ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s593ms - OpenTime: 35.029ms - ExecTreeOpenTime: 352.610us - PrepareTime: 201.095ms - ExecTreePrepareTime: 2.101ms KrpcDataStreamSender (dst_id=1):(Total: 11m27s, non-child: 11m27s, % non-child: 100.00%) - BytesSent: 77.38 GB (83090459822) - EosSent: 0 (0) - OverallThroughput: 115.20 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 589.35M (589349865) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m26s - UncompressedRowBatchSize: 151.92 GB (163122094492) HDFS_SCAN_NODE (id=0):(Total: 8s072ms, non-child: 8s072ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:23/5.55 GB 0:28/6.82 GB 4:40/9.73 GB 3:33/8.01 GB 5:30/7.36 GB 6:29/7.10 GB 2:20/4.87 GB 7:22/5.37 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.06 GB, 15.21 GB, 15.75 GB, 16.44 GB, 17.70 GB, 18.12 GB, 18.48 GB, 18.62 GB, 18.85 GB, 19.11 GB, 19.17 GB, 19.30 GB, 19.48 GB, 19.48 GB, 19.85 GB, 20.36 GB, 20.75 GB, 20.77 GB, 20.94 GB, 21.25 GB, 22.21 GB, 24.09 GB, 26.85 GB, 30.86 GB, 31.23 GB, 31.36 GB, 31.50 GB, 31.66 GB, 31.92 GB, 32.04 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB, 32.12 GB - FooterProcessingTime: (Avg: 1s926ms ; Min: 28.069ms ; Max: 13s222ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.12 GB (34489458250) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m32s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.69 GB (13623917801) - PerReadThreadRawHdfsThroughput: 54.32 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 589.56M (589555689) - RowsReturned: 589.35M (589350889) - RowsReturnedRate: 73.01 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m27s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 18s449ms - TotalRawHdfsReadTime(*): 10m5s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 34.298ms, non-child: 34.298ms, % non-child: 100.00%) - CodegenTime: 522.477us - CompileTime: 5.040ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.394ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 20.351ms Instance 844a0200d7876345:20bb38b90000000f (host=ve1136.foo.com:22000):(Total: 235.760ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:16/3.96 GB 2:27/6.60 GB 3:21/5.04 GB 0:30/7.38 GB 1:29/7.04 GB 6:31/7.59 GB 5:40/9.70 GB 7:31/7.55 GB Fragment Instance Lifecycle Event Timeline: 1s574ms - Prepare Finished: 199.953ms (199.953ms) - Open Finished: 236.480ms (36.527ms) - First Batch Produced: 1s573ms (1s337ms) - First Batch Sent: 1s574ms (1.437ms) MemoryUsage(8m32s): 12.02 GB, 12.65 GB, 12.60 GB, 12.43 GB, 12.28 GB, 12.16 GB, 11.96 GB, 11.54 GB, 10.79 GB, 10.23 GB, 9.78 GB, 9.60 GB, 9.34 GB, 9.31 GB, 8.72 GB, 8.14 GB, 7.82 GB, 7.49 GB, 7.25 GB, 7.76 GB, 8.23 GB, 9.55 GB, 11.36 GB, 12.28 GB, 12.68 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB, 12.66 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.69 GB (13623530819) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.06 GB (186900104333) - RowsProduced: 597.07M (597066881) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 24m23s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 36.529ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 35.994ms - TotalThreadsVoluntaryContextSwitches: 9 (9) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 11s216ms - OpenTime: 36.530ms - ExecTreeOpenTime: 160.877us - PrepareTime: 199.135ms - ExecTreePrepareTime: 1.858ms KrpcDataStreamSender (dst_id=1):(Total: 11m21s, non-child: 11m21s, % non-child: 100.00%) - BytesSent: 78.37 GB (84150965399) - EosSent: 0 (0) - OverallThroughput: 117.78 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 597.07M (597065857) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m19s - UncompressedRowBatchSize: 153.91 GB (165257667504) HDFS_SCAN_NODE (id=0):(Total: 10s705ms, non-child: 10s705ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:16/3.96 GB 2:27/6.60 GB 3:21/5.04 GB 0:30/7.38 GB 1:29/7.04 GB 6:31/7.59 GB 5:40/9.70 GB 7:31/7.55 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.39 GB, 15.36 GB, 15.97 GB, 16.96 GB, 18.06 GB, 18.45 GB, 18.73 GB, 18.89 GB, 19.16 GB, 19.37 GB, 19.46 GB, 19.62 GB, 19.74 GB, 20.20 GB, 20.29 GB, 20.34 GB, 20.46 GB, 20.57 GB, 20.95 GB, 22.37 GB, 23.41 GB, 25.81 GB, 29.00 GB, 30.62 GB, 31.37 GB, 31.45 GB, 31.59 GB, 31.80 GB, 31.98 GB, 32.29 GB, 32.47 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.48 GB, 32.49 GB, 32.49 GB, 32.49 GB, 32.49 GB, 32.49 GB, 32.49 GB, 32.49 GB, 32.49 GB - FooterProcessingTime: (Avg: 1s952ms ; Min: 27.580ms ; Max: 11s072ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.49 GB (34880661729) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m5s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.69 GB (13620528440) - PerReadThreadRawHdfsThroughput: 50.77 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 597.27M (597271681) - RowsReturned: 597.07M (597066881) - RowsReturnedRate: 55.77 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m20s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 21s271ms - TotalRawHdfsReadTime(*): 10m55s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 36.045ms, non-child: 36.045ms, % non-child: 100.00%) - CodegenTime: 435.486us - CompileTime: 4.122ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.220ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 23.159ms Instance 844a0200d7876345:20bb38b900000005 (host=vd1333.foo.com:22000):(Total: 235.511ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 6:24/5.76 GB 4:31/7.61 GB 5:26/6.24 GB 7:31/7.51 GB 3:25/6.15 GB 1:31/7.55 GB 0:34/8.29 GB 2:23/5.62 GB Fragment Instance Lifecycle Event Timeline: 1s722ms - Prepare Finished: 200.970ms (200.970ms) - Open Finished: 236.004ms (35.033ms) - First Batch Produced: 1s720ms (1s484ms) - First Batch Sent: 1s722ms (1.875ms) MemoryUsage(8m32s): 12.18 GB, 12.61 GB, 12.59 GB, 12.40 GB, 11.94 GB, 11.80 GB, 11.71 GB, 11.24 GB, 10.37 GB, 9.91 GB, 9.39 GB, 9.35 GB, 9.21 GB, 8.88 GB, 8.33 GB, 8.19 GB, 8.22 GB, 7.90 GB, 7.54 GB, 7.67 GB, 8.58 GB, 10.21 GB, 12.27 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.70 GB (13638905748) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.12 GB (186955879181) - RowsProduced: 596.44M (596438989) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 17m2s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 35.029ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 13s029ms - OpenTime: 35.030ms - ExecTreeOpenTime: 224.903us - PrepareTime: 200.403ms - ExecTreePrepareTime: 2.046ms KrpcDataStreamSender (dst_id=1):(Total: 11m21s, non-child: 11m21s, % non-child: 100.00%) - BytesSent: 78.32 GB (84100715416) - EosSent: 0 (0) - OverallThroughput: 117.64 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 596.44M (596437965) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m20s - UncompressedRowBatchSize: 153.75 GB (165083777225) HDFS_SCAN_NODE (id=0):(Total: 12s493ms, non-child: 12s493ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:24/5.76 GB 4:31/7.61 GB 5:26/6.24 GB 7:31/7.51 GB 3:25/6.15 GB 1:31/7.55 GB 0:34/8.29 GB 2:23/5.62 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.57 GB, 15.29 GB, 15.83 GB, 16.88 GB, 17.71 GB, 18.10 GB, 18.36 GB, 18.49 GB, 18.81 GB, 19.03 GB, 19.15 GB, 19.47 GB, 19.51 GB, 19.64 GB, 20.11 GB, 20.60 GB, 21.06 GB, 21.29 GB, 21.50 GB, 22.18 GB, 23.81 GB, 26.69 GB, 30.25 GB, 31.05 GB, 31.19 GB, 31.37 GB, 31.61 GB, 31.85 GB, 31.98 GB, 32.13 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB, 32.30 GB - FooterProcessingTime: (Avg: 1s576ms ; Min: 32.292ms ; Max: 6s936ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.30 GB (34684554801) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m27s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.70 GB (13635726371) - PerReadThreadRawHdfsThroughput: 58.52 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 596.64M (596643789) - RowsReturned: 596.44M (596438989) - RowsReturnedRate: 47.74 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m50s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 20s471ms - TotalRawHdfsReadTime(*): 9m25s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 34.539ms, non-child: 34.539ms, % non-child: 100.00%) - CodegenTime: 357.819us - CompileTime: 4.474ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.607ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.854ms Instance 844a0200d7876345:20bb38b900000025 (host=vd1141.foo.com:22000):(Total: 235.317ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:22/5.35 GB 1:30/7.35 GB 6:29/7.06 GB 5:32/7.84 GB 3:24/5.82 GB 0:35/8.50 GB 7:25/6.12 GB 2:27/6.57 GB Fragment Instance Lifecycle Event Timeline: 1s702ms - Prepare Finished: 207.781ms (207.781ms) - Open Finished: 235.733ms (27.951ms) - First Batch Produced: 1s700ms (1s464ms) - First Batch Sent: 1s702ms (2.176ms) MemoryUsage(8m32s): 12.18 GB, 12.67 GB, 12.66 GB, 12.47 GB, 12.05 GB, 11.96 GB, 11.77 GB, 11.18 GB, 10.36 GB, 9.76 GB, 9.38 GB, 9.27 GB, 9.04 GB, 8.52 GB, 7.90 GB, 8.15 GB, 7.87 GB, 7.49 GB, 7.46 GB, 8.39 GB, 9.54 GB, 11.79 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.70 GB (13641772745) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.13 GB (186968232352) - RowsProduced: 605.05M (605053867) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h17m - TotalStorageWaitTime: 18m15s - TotalThreadsInvoluntaryContextSwitches: 2 (2) - TotalThreadsTotalWallClockTime: 27.959ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 27.996ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s043ms - OpenTime: 27.960ms - ExecTreeOpenTime: 154.474us - PrepareTime: 207.281ms - ExecTreePrepareTime: 1.820ms KrpcDataStreamSender (dst_id=1):(Total: 11m, non-child: 11m, % non-child: 100.00%) - BytesSent: 79.39 GB (85245646966) - EosSent: 0 (0) - OverallThroughput: 123.14 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 605.05M (605052843) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 10m58s - UncompressedRowBatchSize: 155.97 GB (167468648927) HDFS_SCAN_NODE (id=0):(Total: 9s561ms, non-child: 9s561ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:22/5.35 GB 1:30/7.35 GB 6:29/7.06 GB 5:32/7.84 GB 3:24/5.82 GB 0:35/8.50 GB 7:25/6.12 GB 2:27/6.57 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.52 GB, 15.35 GB, 15.95 GB, 17.09 GB, 17.91 GB, 18.32 GB, 18.53 GB, 18.73 GB, 19.01 GB, 19.10 GB, 19.23 GB, 19.38 GB, 19.46 GB, 19.61 GB, 19.99 GB, 20.76 GB, 20.98 GB, 21.09 GB, 21.75 GB, 23.52 GB, 25.56 GB, 29.50 GB, 31.03 GB, 31.16 GB, 31.41 GB, 31.60 GB, 31.83 GB, 31.97 GB, 32.13 GB, 32.53 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB, 32.80 GB - FooterProcessingTime: (Avg: 1s615ms ; Min: 14.559ms ; Max: 11s723ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.80 GB (35223092857) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 6m49s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.70 GB (13638607681) - PerReadThreadRawHdfsThroughput: 55.89 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 605.26M (605258667) - RowsReturned: 605.05M (605053867) - RowsReturnedRate: 63.28 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m53s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 19s906ms - TotalRawHdfsReadTime(*): 10m1s - TotalReadThroughput: 1.16 MB/sec CodeGen:(Total: 27.561ms, non-child: 27.561ms, % non-child: 100.00%) - CodegenTime: 319.392us - CompileTime: 3.683ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 5.282ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 18.121ms Instance 844a0200d7876345:20bb38b900000071 (host=ve1132.foo.com:22000):(Total: 234.944ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:25/6.07 GB 7:37/9.05 GB 3:29/6.81 GB 1:30/7.34 GB 0:35/8.62 GB 2:28/6.89 GB 6:22/5.38 GB 4:21/5.00 GB Fragment Instance Lifecycle Event Timeline: 1s570ms - Prepare Finished: 209.064ms (209.064ms) - Open Finished: 235.566ms (26.501ms) - First Batch Produced: 1s568ms (1s332ms) - First Batch Sent: 1s570ms (2.228ms) MemoryUsage(8m32s): 12.04 GB, 12.70 GB, 12.68 GB, 12.57 GB, 12.27 GB, 12.07 GB, 11.93 GB, 11.55 GB, 10.88 GB, 10.24 GB, 9.74 GB, 9.48 GB, 9.33 GB, 8.98 GB, 8.49 GB, 8.14 GB, 7.93 GB, 7.77 GB, 7.43 GB, 7.61 GB, 8.36 GB, 9.43 GB, 11.00 GB, 11.85 GB, 12.58 GB, 12.70 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.71 GB (13646162254) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.44 GB (171200310156) - RowsProduced: 590.70M (590697507) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 23m55s - TotalThreadsInvoluntaryContextSwitches: 2 (2) - TotalThreadsTotalWallClockTime: 26.499ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 23.997ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 13s325ms - OpenTime: 26.500ms - ExecTreeOpenTime: 180.227us - PrepareTime: 208.348ms - ExecTreePrepareTime: 1.894ms KrpcDataStreamSender (dst_id=1):(Total: 11m33s, non-child: 11m33s, % non-child: 100.00%) - BytesSent: 77.50 GB (83219749988) - EosSent: 0 (0) - OverallThroughput: 114.40 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 590.70M (590696483) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m32s - UncompressedRowBatchSize: 152.27 GB (163494447245) HDFS_SCAN_NODE (id=0):(Total: 12s782ms, non-child: 12s782ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:25/6.07 GB 7:37/9.05 GB 3:29/6.81 GB 1:30/7.34 GB 0:35/8.62 GB 2:28/6.89 GB 6:22/5.38 GB 4:21/5.00 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.23 GB, 15.31 GB, 15.93 GB, 16.84 GB, 17.83 GB, 18.29 GB, 18.57 GB, 18.77 GB, 19.01 GB, 19.19 GB, 19.31 GB, 19.44 GB, 19.62 GB, 19.71 GB, 19.88 GB, 20.22 GB, 20.59 GB, 20.95 GB, 21.08 GB, 21.94 GB, 23.56 GB, 25.51 GB, 28.24 GB, 29.73 GB, 31.09 GB, 31.42 GB, 31.59 GB, 31.80 GB, 31.94 GB, 32.14 GB, 32.26 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB, 32.27 GB - FooterProcessingTime: (Avg: 1s956ms ; Min: 18.426ms ; Max: 7s803ms ; Number of samples: 161) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.27 GB (34646632752) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m23s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 161 (161) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.71 GB (13643038175) - PerReadThreadRawHdfsThroughput: 49.03 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 590.90M (590902307) - RowsReturned: 590.70M (590697507) - RowsReturnedRate: 46.21 M/sec - ScanRangesComplete: 81 (81) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m55s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 23s301ms - TotalRawHdfsReadTime(*): 11m13s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 26.024ms, non-child: 26.024ms, % non-child: 100.00%) - CodegenTime: 433.047us - CompileTime: 4.385ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.054ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.100ms Instance 844a0200d7876345:20bb38b90000003a (host=vd1111.foo.com:22000):(Total: 234.527ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:28/6.85 GB 1:31/7.39 GB 7:29/6.83 GB 3:36/8.76 GB 5:18/4.41 GB 6:33/8.10 GB 2:24/5.90 GB 4:27/6.37 GB Fragment Instance Lifecycle Event Timeline: 1s156ms - Prepare Finished: 203.102ms (203.102ms) - Open Finished: 234.916ms (31.814ms) - First Batch Produced: 1s154ms (919.991ms) - First Batch Sent: 1s156ms (1.899ms) MemoryUsage(8m32s): 13.21 GB, 13.99 GB, 13.97 GB, 13.95 GB, 13.82 GB, 13.50 GB, 13.32 GB, 13.27 GB, 13.05 GB, 12.49 GB, 11.78 GB, 11.28 GB, 10.85 GB, 10.63 GB, 10.53 GB, 10.31 GB, 9.89 GB, 9.22 GB, 8.85 GB, 8.66 GB, 8.38 GB, 8.10 GB, 7.76 GB, 7.77 GB, 8.30 GB, 9.38 GB, 11.09 GB, 12.95 GB, 13.79 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15023857611) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.68 GB (172525062263) - RowsProduced: 599.50M (599495765) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h18m - TotalStorageWaitTime: 28m13s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 31.810ms - TotalThreadsSysTime: 2.999ms - TotalThreadsUserTime: 28.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 6s701ms - OpenTime: 31.812ms - ExecTreeOpenTime: 314.536us - PrepareTime: 202.587ms - ExecTreePrepareTime: 1.898ms KrpcDataStreamSender (dst_id=1):(Total: 11m6s, non-child: 11m6s, % non-child: 100.00%) - BytesSent: 78.72 GB (84528604186) - EosSent: 0 (0) - OverallThroughput: 121.03 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 599.49M (599494741) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m4s - UncompressedRowBatchSize: 154.53 GB (165929991638) HDFS_SCAN_NODE (id=0):(Total: 6s246ms, non-child: 6s246ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:28/6.85 GB 1:31/7.39 GB 7:29/6.83 GB 3:36/8.76 GB 5:18/4.41 GB 6:33/8.10 GB 2:24/5.90 GB 4:27/6.37 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.92 GB, 16.51 GB, 17.22 GB, 17.78 GB, 18.94 GB, 19.80 GB, 20.13 GB, 20.46 GB, 20.61 GB, 20.81 GB, 21.10 GB, 21.22 GB, 21.43 GB, 21.60 GB, 21.73 GB, 21.76 GB, 21.77 GB, 21.77 GB, 22.09 GB, 22.32 GB, 22.39 GB, 22.62 GB, 22.76 GB, 23.38 GB, 24.57 GB, 26.51 GB, 29.47 GB, 32.66 GB, 34.13 GB, 34.43 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB, 34.52 GB - FooterProcessingTime: (Avg: 1s899ms ; Min: 27.016ms ; Max: 8s547ms ; Number of samples: 177) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.52 GB (37064102104) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 5m56s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 177 (177) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.99 GB (15020782526) - PerReadThreadRawHdfsThroughput: 53.72 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 599.71M (599708757) - RowsReturned: 599.50M (599495765) - RowsReturnedRate: 95.98 M/sec - ScanRangesComplete: 89 (89) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m33s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 30s316ms - TotalRawHdfsReadTime(*): 10m57s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 31.169ms, non-child: 31.169ms, % non-child: 100.00%) - CodegenTime: 422.763us - CompileTime: 4.645ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.183ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 17.677ms Instance 844a0200d7876345:20bb38b900000016 (host=vd1103.foo.com:22000):(Total: 233.844ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:20/4.85 GB 1:25/6.11 GB 6:25/6.07 GB 0:33/8.13 GB 7:35/8.44 GB 4:27/6.65 GB 5:24/5.87 GB 3:34/8.16 GB Fragment Instance Lifecycle Event Timeline: 1s912ms - Prepare Finished: 206.101ms (206.101ms) - Open Finished: 234.364ms (28.262ms) - First Batch Produced: 1s910ms (1s676ms) - First Batch Sent: 1s912ms (2.058ms) MemoryUsage(8m32s): 13.22 GB, 13.97 GB, 13.95 GB, 13.86 GB, 13.60 GB, 13.29 GB, 13.22 GB, 13.06 GB, 12.67 GB, 11.89 GB, 11.34 GB, 10.82 GB, 10.46 GB, 10.30 GB, 10.14 GB, 9.85 GB, 9.43 GB, 9.05 GB, 8.77 GB, 8.73 GB, 8.47 GB, 8.26 GB, 8.20 GB, 8.67 GB, 9.44 GB, 10.72 GB, 12.58 GB, 13.75 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (15001573938) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.37 GB (188302406418) - RowsProduced: 607.71M (607709266) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 27m2s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 28.231ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 25.996ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 11s574ms - OpenTime: 28.232ms - ExecTreeOpenTime: 197.446us - PrepareTime: 205.532ms - ExecTreePrepareTime: 2.074ms KrpcDataStreamSender (dst_id=1):(Total: 11m15s, non-child: 11m15s, % non-child: 100.00%) - BytesSent: 79.76 GB (85645973972) - EosSent: 0 (0) - OverallThroughput: 120.85 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 607.71M (607708242) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m14s - UncompressedRowBatchSize: 156.65 GB (168202794438) HDFS_SCAN_NODE (id=0):(Total: 11s065ms, non-child: 11s065ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:20/4.85 GB 1:25/6.11 GB 6:25/6.07 GB 0:33/8.13 GB 7:35/8.44 GB 4:27/6.65 GB 5:24/5.87 GB 3:34/8.16 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.44 GB, 16.84 GB, 17.45 GB, 18.18 GB, 19.40 GB, 19.90 GB, 20.23 GB, 20.48 GB, 20.63 GB, 20.89 GB, 21.09 GB, 21.15 GB, 21.26 GB, 21.43 GB, 21.51 GB, 21.62 GB, 21.89 GB, 22.32 GB, 22.47 GB, 22.83 GB, 22.97 GB, 23.32 GB, 23.82 GB, 24.98 GB, 26.35 GB, 28.65 GB, 31.92 GB, 34.00 GB, 34.43 GB, 34.56 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB, 34.64 GB - FooterProcessingTime: (Avg: 2s252ms ; Min: 27.266ms ; Max: 9s920ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.64 GB (37195919972) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m54s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (14998359720) - PerReadThreadRawHdfsThroughput: 53.67 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 607.92M (607922258) - RowsReturned: 607.71M (607709266) - RowsReturnedRate: 54.92 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m39s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 27s307ms - TotalRawHdfsReadTime(*): 11m - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 27.643ms, non-child: 27.643ms, % non-child: 100.00%) - CodegenTime: 475.935us - CompileTime: 4.514ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.504ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 16.133ms Instance 844a0200d7876345:20bb38b90000002e (host=vd1330.foo.com:22000):(Total: 233.591ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:27/6.32 GB 7:31/7.60 GB 1:33/7.82 GB 6:29/7.06 GB 3:28/6.90 GB 2:34/8.42 GB 0:23/5.56 GB 4:21/5.13 GB Fragment Instance Lifecycle Event Timeline: 1s689ms - Prepare Finished: 197.775ms (197.775ms) - Open Finished: 234.175ms (36.400ms) - First Batch Produced: 1s688ms (1s453ms) - First Batch Sent: 1s689ms (1.850ms) MemoryUsage(8m32s): 12.06 GB, 12.69 GB, 12.67 GB, 12.54 GB, 12.28 GB, 12.02 GB, 11.95 GB, 11.70 GB, 11.04 GB, 10.38 GB, 9.82 GB, 9.42 GB, 9.31 GB, 9.28 GB, 9.19 GB, 8.55 GB, 8.23 GB, 8.11 GB, 7.78 GB, 7.48 GB, 7.41 GB, 7.92 GB, 9.14 GB, 11.39 GB, 12.59 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.71 GB (13647943208) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.13 GB (186974007320) - RowsProduced: 584.38M (584384266) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 23m14s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 36.394ms - TotalThreadsSysTime: 2.999ms - TotalThreadsUserTime: 33.995ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7s200ms - OpenTime: 36.395ms - ExecTreeOpenTime: 301.143us - PrepareTime: 197.111ms - ExecTreePrepareTime: 2.095ms KrpcDataStreamSender (dst_id=1):(Total: 11m18s, non-child: 11m18s, % non-child: 100.00%) - BytesSent: 76.70 GB (82361329889) - EosSent: 0 (0) - OverallThroughput: 115.78 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 584.38M (584383242) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m17s - UncompressedRowBatchSize: 150.64 GB (161747866765) HDFS_SCAN_NODE (id=0):(Total: 6s718ms, non-child: 6s718ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:27/6.32 GB 7:31/7.60 GB 1:33/7.82 GB 6:29/7.06 GB 3:28/6.90 GB 2:34/8.42 GB 0:23/5.56 GB 4:21/5.13 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 11.98 GB, 15.18 GB, 15.78 GB, 16.40 GB, 17.64 GB, 18.11 GB, 18.49 GB, 18.61 GB, 18.83 GB, 19.10 GB, 19.16 GB, 19.27 GB, 19.45 GB, 19.70 GB, 20.21 GB, 20.44 GB, 20.63 GB, 20.86 GB, 21.04 GB, 21.32 GB, 21.79 GB, 22.81 GB, 25.05 GB, 29.02 GB, 31.09 GB, 31.35 GB, 31.43 GB, 31.60 GB, 31.86 GB, 32.05 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB, 32.11 GB - FooterProcessingTime: (Avg: 2s406ms ; Min: 24.362ms ; Max: 23s214ms ; Number of samples: 162) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.11 GB (34476333691) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 12m34s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 162 (162) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.71 GB (13644876431) - PerReadThreadRawHdfsThroughput: 53.93 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 584.59M (584589066) - RowsReturned: 584.38M (584384266) - RowsReturnedRate: 86.99 M/sec - ScanRangesComplete: 82 (82) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m36s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 25s262ms - TotalRawHdfsReadTime(*): 10m9s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 35.779ms, non-child: 35.779ms, % non-child: 100.00%) - CodegenTime: 469.321us - CompileTime: 4.712ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.364ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.076ms Instance 844a0200d7876345:20bb38b900000024 (host=vc1537.foo.com:22000):(Total: 233.388ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:19/4.70 GB 1:26/6.37 GB 0:27/6.46 GB 4:37/8.92 GB 3:20/4.95 GB 7:32/7.79 GB 6:29/7.14 GB 2:34/8.34 GB Fragment Instance Lifecycle Event Timeline: 1s988ms - Prepare Finished: 206.033ms (206.033ms) - Open Finished: 233.859ms (27.826ms) - First Batch Produced: 1s986ms (1s752ms) - First Batch Sent: 1s988ms (1.706ms) MemoryUsage(8m32s): 13.21 GB, 14.01 GB, 14.01 GB, 14.00 GB, 13.92 GB, 13.67 GB, 13.45 GB, 13.41 GB, 13.23 GB, 12.82 GB, 12.07 GB, 11.59 GB, 11.11 GB, 10.73 GB, 10.61 GB, 10.45 GB, 10.12 GB, 9.68 GB, 8.96 GB, 8.43 GB, 8.27 GB, 8.18 GB, 7.89 GB, 7.72 GB, 7.78 GB, 8.53 GB, 9.40 GB, 10.83 GB, 12.80 GB, 13.79 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB, 13.82 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 14.01 GB (15047797772) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.25 GB (188169539928) - RowsProduced: 588.61M (588610577) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h11m - TotalStorageWaitTime: 29m40s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 27.822ms - TotalThreadsSysTime: 3.999ms - TotalThreadsUserTime: 23.997ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 20s205ms - OpenTime: 27.823ms - ExecTreeOpenTime: 267.096us - PrepareTime: 205.510ms - ExecTreePrepareTime: 1.906ms KrpcDataStreamSender (dst_id=1):(Total: 11m4s, non-child: 11m4s, % non-child: 100.00%) - BytesSent: 77.25 GB (82944363686) - EosSent: 0 (0) - OverallThroughput: 119.07 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 588.61M (588609553) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m3s - UncompressedRowBatchSize: 151.73 GB (162917170924) HDFS_SCAN_NODE (id=0):(Total: 19s644ms, non-child: 19s644ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:19/4.70 GB 1:26/6.37 GB 0:27/6.46 GB 4:37/8.92 GB 3:20/4.95 GB 7:32/7.79 GB 6:29/7.14 GB 2:34/8.34 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.84 GB, 16.39 GB, 17.16 GB, 17.76 GB, 18.79 GB, 19.78 GB, 20.12 GB, 20.45 GB, 20.65 GB, 20.81 GB, 21.09 GB, 21.28 GB, 21.35 GB, 21.46 GB, 21.63 GB, 21.70 GB, 21.70 GB, 21.70 GB, 21.70 GB, 21.70 GB, 21.83 GB, 22.21 GB, 22.38 GB, 22.79 GB, 23.47 GB, 24.97 GB, 26.54 GB, 29.11 GB, 32.50 GB, 34.19 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB, 34.25 GB - FooterProcessingTime: (Avg: 2s202ms ; Min: 22.292ms ; Max: 8s689ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.25 GB (36772381881) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 14m16s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 14.01 GB (15044714602) - PerReadThreadRawHdfsThroughput: 47.39 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 588.82M (588823569) - RowsReturned: 588.61M (588610577) - RowsReturnedRate: 29.96 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 11m42s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 34s269ms - TotalRawHdfsReadTime(*): 12m19s - TotalReadThroughput: 1.21 MB/sec CodeGen:(Total: 27.291ms, non-child: 27.291ms, % non-child: 100.00%) - CodegenTime: 418.466us - CompileTime: 4.424ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.502ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.852ms Instance 844a0200d7876345:20bb38b90000001e (host=vd1331.foo.com:22000):(Total: 232.248ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:35/8.47 GB 4:15/3.68 GB 2:39/9.62 GB 3:26/6.32 GB 7:24/5.94 GB 6:28/6.81 GB 0:30/7.35 GB 5:27/6.65 GB Fragment Instance Lifecycle Event Timeline: 1s634ms - Prepare Finished: 199.044ms (199.044ms) - Open Finished: 232.871ms (33.827ms) - First Batch Produced: 1s632ms (1s399ms) - First Batch Sent: 1s634ms (1.905ms) MemoryUsage(8m32s): 12.21 GB, 12.73 GB, 12.73 GB, 12.67 GB, 12.36 GB, 12.16 GB, 12.11 GB, 11.84 GB, 11.00 GB, 10.44 GB, 9.81 GB, 9.62 GB, 9.51 GB, 9.20 GB, 8.47 GB, 7.89 GB, 7.84 GB, 7.39 GB, 6.95 GB, 6.91 GB, 7.70 GB, 8.87 GB, 11.43 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.69 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.73 GB (13669913597) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 174.11 GB (186944756357) - RowsProduced: 592.45M (592447213) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 19m12s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 33.825ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s065ms - OpenTime: 33.826ms - ExecTreeOpenTime: 225.404us - PrepareTime: 198.354ms - ExecTreePrepareTime: 2.208ms KrpcDataStreamSender (dst_id=1):(Total: 11m15s, non-child: 11m15s, % non-child: 100.00%) - BytesSent: 77.79 GB (83526915223) - EosSent: 0 (0) - OverallThroughput: 118.00 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 592.45M (592446189) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m13s - UncompressedRowBatchSize: 152.72 GB (163978632799) HDFS_SCAN_NODE (id=0):(Total: 8s600ms, non-child: 8s600ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:35/8.47 GB 4:15/3.68 GB 2:39/9.62 GB 3:26/6.32 GB 7:24/5.94 GB 6:28/6.81 GB 0:30/7.35 GB 5:27/6.65 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.11 GB, 15.26 GB, 15.84 GB, 16.67 GB, 17.90 GB, 18.28 GB, 18.66 GB, 18.77 GB, 19.07 GB, 19.27 GB, 19.35 GB, 19.54 GB, 19.65 GB, 19.65 GB, 19.66 GB, 19.97 GB, 20.23 GB, 20.23 GB, 20.34 GB, 21.08 GB, 22.55 GB, 24.64 GB, 29.07 GB, 31.16 GB, 31.26 GB, 31.40 GB, 31.54 GB, 31.77 GB, 31.92 GB, 32.07 GB, 32.27 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB - FooterProcessingTime: (Avg: 2s090ms ; Min: 16.893ms ; Max: 9s247ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.28 GB (34661887613) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m49s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.73 GB (13666855027) - PerReadThreadRawHdfsThroughput: 55.70 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 592.65M (592652013) - RowsReturned: 592.45M (592447213) - RowsReturnedRate: 68.88 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m14s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 17s568ms - TotalRawHdfsReadTime(*): 9m53s - TotalReadThroughput: 1.15 MB/sec CodeGen:(Total: 33.361ms, non-child: 33.361ms, % non-child: 100.00%) - CodegenTime: 319.269us - CompileTime: 4.254ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.120ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 21.427ms Instance 844a0200d7876345:20bb38b90000004a (host=vc1536.foo.com:22000):(Total: 231.595ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:18/4.46 GB 3:32/7.84 GB 1:26/6.07 GB 6:28/6.90 GB 7:30/7.34 GB 4:32/7.85 GB 2:31/7.53 GB 5:28/6.82 GB Fragment Instance Lifecycle Event Timeline: 1s432ms - Prepare Finished: 204.590ms (204.590ms) - Open Finished: 231.989ms (27.398ms) - First Batch Produced: 1s429ms (1s197ms) - First Batch Sent: 1s432ms (2.734ms) MemoryUsage(8m32s): 13.18 GB, 13.97 GB, 13.96 GB, 13.93 GB, 13.81 GB, 13.52 GB, 13.27 GB, 13.21 GB, 13.08 GB, 12.66 GB, 11.97 GB, 11.42 GB, 10.91 GB, 10.59 GB, 10.45 GB, 10.35 GB, 10.07 GB, 9.69 GB, 9.15 GB, 8.75 GB, 8.84 GB, 8.71 GB, 8.42 GB, 8.27 GB, 8.04 GB, 8.58 GB, 9.42 GB, 11.17 GB, 12.57 GB, 13.89 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.98 GB (15012456533) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.71 GB (172566043126) - RowsProduced: 588.64M (588639553) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 29m52s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 27.394ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 26.996ms - TotalThreadsVoluntaryContextSwitches: 16 (16) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s962ms - OpenTime: 27.396ms - ExecTreeOpenTime: 213.326us - PrepareTime: 204.134ms - ExecTreePrepareTime: 2.139ms KrpcDataStreamSender (dst_id=1):(Total: 11m28s, non-child: 11m28s, % non-child: 100.00%) - BytesSent: 77.30 GB (82999186489) - EosSent: 0 (0) - OverallThroughput: 114.92 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 588.64M (588638529) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m27s - UncompressedRowBatchSize: 151.74 GB (162924698588) HDFS_SCAN_NODE (id=0):(Total: 8s434ms, non-child: 8s434ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:18/4.46 GB 3:32/7.84 GB 1:26/6.07 GB 6:28/6.90 GB 7:30/7.34 GB 4:32/7.85 GB 2:31/7.53 GB 5:28/6.82 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.85 GB, 16.37 GB, 17.14 GB, 17.67 GB, 18.59 GB, 19.61 GB, 19.96 GB, 20.29 GB, 20.50 GB, 20.64 GB, 20.90 GB, 21.10 GB, 21.17 GB, 21.41 GB, 21.56 GB, 21.65 GB, 21.66 GB, 21.80 GB, 21.95 GB, 22.21 GB, 22.65 GB, 22.95 GB, 23.05 GB, 23.57 GB, 23.71 GB, 24.94 GB, 26.46 GB, 29.52 GB, 31.94 GB, 34.25 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB, 34.37 GB - FooterProcessingTime: (Avg: 2s159ms ; Min: 14.418ms ; Max: 10s666ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.37 GB (36902509581) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m36s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.98 GB (15009332427) - PerReadThreadRawHdfsThroughput: 52.59 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 588.85M (588852545) - RowsReturned: 588.64M (588639553) - RowsReturnedRate: 69.79 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m58s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 20s730ms - TotalRawHdfsReadTime(*): 11m9s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 26.910ms, non-child: 26.910ms, % non-child: 100.00%) - CodegenTime: 387.171us - CompileTime: 3.932ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.430ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 14.880ms Instance 844a0200d7876345:20bb38b900000019 (host=vc1507.foo.com:22000):(Total: 231.284ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 3:28/6.84 GB 5:30/7.01 GB 1:32/7.89 GB 4:20/4.71 GB 7:33/8.09 GB 0:25/6.01 GB 2:32/7.84 GB 6:25/6.11 GB Fragment Instance Lifecycle Event Timeline: 1s693ms - Prepare Finished: 202.830ms (202.830ms) - Open Finished: 231.707ms (28.876ms) - First Batch Produced: 1s691ms (1s460ms) - First Batch Sent: 1s693ms (1.341ms) MemoryUsage(8m32s): 13.15 GB, 13.93 GB, 13.93 GB, 13.87 GB, 13.67 GB, 13.34 GB, 13.16 GB, 13.09 GB, 12.86 GB, 12.35 GB, 11.59 GB, 11.14 GB, 10.59 GB, 10.31 GB, 10.19 GB, 10.07 GB, 9.85 GB, 9.43 GB, 9.08 GB, 8.87 GB, 9.09 GB, 8.85 GB, 8.56 GB, 8.55 GB, 8.75 GB, 9.36 GB, 10.51 GB, 12.16 GB, 13.58 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB, 13.84 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.93 GB (14960026988) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.26 GB (188182757334) - RowsProduced: 594.62M (594621261) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 26m15s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 28.872ms - TotalThreadsSysTime: 2.999ms - TotalThreadsUserTime: 24.996ms - TotalThreadsVoluntaryContextSwitches: 4 (4) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 9s123ms - OpenTime: 28.873ms - ExecTreeOpenTime: 247.111us - PrepareTime: 202.336ms - ExecTreePrepareTime: 1.722ms KrpcDataStreamSender (dst_id=1):(Total: 11m19s, non-child: 11m19s, % non-child: 100.00%) - BytesSent: 78.01 GB (83763063458) - EosSent: 0 (0) - OverallThroughput: 117.62 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 594.62M (594620237) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m17s - UncompressedRowBatchSize: 153.28 GB (164580882048) HDFS_SCAN_NODE (id=0):(Total: 8s548ms, non-child: 8s548ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:28/6.84 GB 5:30/7.01 GB 1:32/7.89 GB 4:20/4.71 GB 7:33/8.09 GB 0:25/6.01 GB 2:32/7.84 GB 6:25/6.11 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.99 GB, 16.53 GB, 17.17 GB, 17.73 GB, 18.82 GB, 19.62 GB, 19.93 GB, 20.29 GB, 20.42 GB, 20.59 GB, 20.89 GB, 21.00 GB, 21.06 GB, 21.19 GB, 21.34 GB, 21.51 GB, 21.65 GB, 21.93 GB, 22.37 GB, 22.60 GB, 23.26 GB, 23.40 GB, 23.57 GB, 24.24 GB, 24.94 GB, 26.08 GB, 28.14 GB, 31.12 GB, 33.64 GB, 34.17 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB, 34.22 GB - FooterProcessingTime: (Avg: 2s302ms ; Min: 23.384ms ; Max: 9s252ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.22 GB (36745519851) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m30s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.93 GB (14956927393) - PerReadThreadRawHdfsThroughput: 53.17 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 594.83M (594834253) - RowsReturned: 594.62M (594621261) - RowsReturnedRate: 69.55 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m15s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 28s575ms - TotalRawHdfsReadTime(*): 10m59s - TotalReadThroughput: 1.21 MB/sec CodeGen:(Total: 28.292ms, non-child: 28.292ms, % non-child: 100.00%) - CodegenTime: 476.394us - CompileTime: 4.606ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.666ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 15.463ms Instance 844a0200d7876345:20bb38b900000074 (host=vd1115.foo.com:22000):(Total: 230.177ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:28/6.65 GB 1:30/7.30 GB 3:21/5.09 GB 0:31/7.60 GB 5:31/7.63 GB 6:30/7.38 GB 4:32/7.88 GB 2:21/5.12 GB Fragment Instance Lifecycle Event Timeline: 1s602ms - Prepare Finished: 193.387ms (193.387ms) - Open Finished: 230.849ms (37.462ms) - First Batch Produced: 1s599ms (1s368ms) - First Batch Sent: 1s602ms (2.329ms) MemoryUsage(8m32s): 13.13 GB, 13.98 GB, 13.97 GB, 13.89 GB, 13.72 GB, 13.40 GB, 13.25 GB, 13.07 GB, 12.80 GB, 12.16 GB, 11.52 GB, 11.11 GB, 10.62 GB, 10.34 GB, 10.14 GB, 10.01 GB, 9.79 GB, 9.15 GB, 8.55 GB, 8.54 GB, 8.60 GB, 8.53 GB, 8.66 GB, 8.74 GB, 9.06 GB, 9.62 GB, 10.67 GB, 12.83 GB, 13.76 GB, 13.95 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.98 GB (15011809734) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.73 GB (172577584602) - RowsProduced: 602.40M (602397698) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h13m - TotalStorageWaitTime: 31m15s - TotalThreadsInvoluntaryContextSwitches: 5 (5) - TotalThreadsTotalWallClockTime: 37.459ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 34.995ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s902ms - OpenTime: 37.460ms - ExecTreeOpenTime: 189.024us - PrepareTime: 192.594ms - ExecTreePrepareTime: 1.608ms KrpcDataStreamSender (dst_id=1):(Total: 11m13s, non-child: 11m13s, % non-child: 100.00%) - BytesSent: 79.05 GB (84879892549) - EosSent: 0 (0) - OverallThroughput: 120.19 MB/sec - PeakMemoryUsage: 3.59 MB (3760800) - RowsReturned: 602.40M (602396674) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m12s - UncompressedRowBatchSize: 155.28 GB (166732749479) HDFS_SCAN_NODE (id=0):(Total: 8s407ms, non-child: 8s407ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:28/6.65 GB 1:30/7.30 GB 3:21/5.09 GB 0:31/7.60 GB 5:31/7.63 GB 6:30/7.38 GB 4:32/7.88 GB 2:21/5.12 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.23 GB, 16.78 GB, 17.44 GB, 18.07 GB, 19.22 GB, 19.91 GB, 20.18 GB, 20.55 GB, 20.66 GB, 20.83 GB, 21.15 GB, 21.23 GB, 21.28 GB, 21.43 GB, 21.56 GB, 21.71 GB, 21.84 GB, 21.93 GB, 22.10 GB, 22.44 GB, 22.87 GB, 23.38 GB, 24.18 GB, 24.99 GB, 25.74 GB, 26.77 GB, 28.76 GB, 32.56 GB, 34.19 GB, 34.49 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB, 34.56 GB - FooterProcessingTime: (Avg: 2s496ms ; Min: 15.987ms ; Max: 14s821ms ; Number of samples: 177) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.56 GB (37103636622) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 12m53s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 177 (177) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.98 GB (15008629989) - PerReadThreadRawHdfsThroughput: 47.15 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 602.61M (602610690) - RowsReturned: 602.40M (602397698) - RowsReturnedRate: 71.65 M/sec - ScanRangesComplete: 89 (89) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m32s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 26s186ms - TotalRawHdfsReadTime(*): 12m30s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 36.971ms, non-child: 36.971ms, % non-child: 100.00%) - CodegenTime: 427.712us - CompileTime: 4.203ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.338ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 24.833ms Instance 844a0200d7876345:20bb38b90000001f (host=va1046.foo.com:22000):(Total: 229.085ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:25/6.11 GB 0:36/8.68 GB 5:35/8.59 GB 7:27/6.45 GB 3:29/6.99 GB 2:20/4.84 GB 4:29/7.07 GB 6:24/5.88 GB Fragment Instance Lifecycle Event Timeline: 1s645ms - Prepare Finished: 193.329ms (193.329ms) - Open Finished: 229.796ms (36.466ms) - First Batch Produced: 1s644ms (1s414ms) - First Batch Sent: 1s645ms (1.347ms) MemoryUsage(8m32s): 13.27 GB, 13.97 GB, 13.97 GB, 13.95 GB, 13.82 GB, 13.53 GB, 13.32 GB, 13.25 GB, 13.08 GB, 12.60 GB, 11.89 GB, 11.37 GB, 10.91 GB, 10.55 GB, 10.40 GB, 10.22 GB, 9.92 GB, 9.44 GB, 8.78 GB, 8.46 GB, 8.51 GB, 8.43 GB, 8.26 GB, 8.02 GB, 8.36 GB, 8.87 GB, 10.01 GB, 11.60 GB, 12.85 GB, 13.74 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB, 13.76 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (15002554256) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.17 GB (188092279510) - RowsProduced: 590.31M (590314720) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 25m38s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 36.463ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 35.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 13s714ms - OpenTime: 36.464ms - ExecTreeOpenTime: 205.880us - PrepareTime: 192.539ms - ExecTreePrepareTime: 1.721ms KrpcDataStreamSender (dst_id=1):(Total: 11m8s, non-child: 11m8s, % non-child: 100.00%) - BytesSent: 77.46 GB (83171666255) - EosSent: 0 (0) - OverallThroughput: 118.58 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 590.31M (590313696) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m7s - UncompressedRowBatchSize: 152.17 GB (163389731828) HDFS_SCAN_NODE (id=0):(Total: 13s237ms, non-child: 13s237ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:25/6.11 GB 0:36/8.68 GB 5:35/8.59 GB 7:27/6.45 GB 3:29/6.99 GB 2:20/4.84 GB 4:29/7.07 GB 6:24/5.88 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.04 GB, 16.46 GB, 17.16 GB, 17.74 GB, 18.79 GB, 19.69 GB, 20.02 GB, 20.35 GB, 20.54 GB, 20.70 GB, 20.97 GB, 21.13 GB, 21.20 GB, 21.33 GB, 21.47 GB, 21.53 GB, 21.54 GB, 21.67 GB, 21.67 GB, 21.87 GB, 22.35 GB, 22.72 GB, 23.08 GB, 23.29 GB, 24.33 GB, 25.45 GB, 27.51 GB, 30.29 GB, 32.55 GB, 34.06 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB, 34.13 GB - FooterProcessingTime: (Avg: 2s071ms ; Min: 37.969ms ; Max: 8s947ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.13 GB (36649546494) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m53s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (14999438332) - PerReadThreadRawHdfsThroughput: 53.92 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 590.53M (590527712) - RowsReturned: 590.31M (590314720) - RowsReturnedRate: 44.59 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 10m5s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 31s295ms - TotalRawHdfsReadTime(*): 10m48s - TotalReadThroughput: 1.21 MB/sec CodeGen:(Total: 35.911ms, non-child: 35.911ms, % non-child: 100.00%) - CodegenTime: 487.212us - CompileTime: 4.554ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.230ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.617ms Instance 844a0200d7876345:20bb38b900000056 (host=vd1340.foo.com:22000):(Total: 227.318ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:18/4.46 GB 1:31/7.63 GB 3:30/7.31 GB 2:34/8.15 GB 6:21/5.13 GB 4:35/8.56 GB 5:40/9.49 GB 7:16/3.89 GB Fragment Instance Lifecycle Event Timeline: 1s774ms - Prepare Finished: 193.872ms (193.872ms) - Open Finished: 228.096ms (34.224ms) - First Batch Produced: 1s772ms (1s544ms) - First Batch Sent: 1s774ms (1.327ms) MemoryUsage(8m32s): 12.04 GB, 12.74 GB, 12.73 GB, 12.68 GB, 12.51 GB, 12.28 GB, 12.19 GB, 11.96 GB, 11.46 GB, 10.78 GB, 10.31 GB, 9.85 GB, 9.67 GB, 9.48 GB, 9.07 GB, 8.46 GB, 7.90 GB, 7.72 GB, 7.46 GB, 7.23 GB, 7.21 GB, 7.75 GB, 8.85 GB, 10.43 GB, 11.92 GB, 12.58 GB, 12.58 GB, 12.59 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB, 12.67 GB ThreadUsage(8m32s): 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.74 GB (13679455258) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.41 GB (171162066918) - RowsProduced: 582.65M (582653926) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h13m - TotalStorageWaitTime: 25m42s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 34.220ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 33.994ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 8s256ms - OpenTime: 34.222ms - ExecTreeOpenTime: 179.938us - PrepareTime: 193.001ms - ExecTreePrepareTime: 1.746ms KrpcDataStreamSender (dst_id=1):(Total: 11m26s, non-child: 11m26s, % non-child: 100.00%) - BytesSent: 76.42 GB (82052383375) - EosSent: 0 (0) - OverallThroughput: 114.02 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 582.65M (582652902) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m24s - UncompressedRowBatchSize: 150.19 GB (161268569487) HDFS_SCAN_NODE (id=0):(Total: 7s684ms, non-child: 7s684ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:18/4.46 GB 1:31/7.63 GB 3:30/7.31 GB 2:34/8.15 GB 6:21/5.13 GB 4:35/8.56 GB 5:40/9.49 GB 7:16/3.89 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.12 GB, 15.27 GB, 15.92 GB, 16.65 GB, 17.82 GB, 18.33 GB, 18.67 GB, 18.87 GB, 19.07 GB, 19.34 GB, 19.45 GB, 19.56 GB, 19.71 GB, 19.80 GB, 19.80 GB, 19.86 GB, 19.95 GB, 20.23 GB, 20.43 GB, 20.82 GB, 21.39 GB, 22.70 GB, 24.71 GB, 27.47 GB, 30.09 GB, 31.22 GB, 31.30 GB, 31.49 GB, 31.81 GB, 32.02 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB, 32.13 GB - FooterProcessingTime: (Avg: 2s619ms ; Min: 23.789ms ; Max: 12s919ms ; Number of samples: 161) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.13 GB (34501307184) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 11m45s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 161 (161) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.74 GB (13676355728) - PerReadThreadRawHdfsThroughput: 48.70 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 582.86M (582858726) - RowsReturned: 582.65M (582653926) - RowsReturnedRate: 75.82 M/sec - ScanRangesComplete: 81 (81) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m17s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 24s562ms - TotalRawHdfsReadTime(*): 11m15s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 33.770ms, non-child: 33.770ms, % non-child: 100.00%) - CodegenTime: 372.155us - CompileTime: 4.176ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.422ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 21.625ms Instance 844a0200d7876345:20bb38b900000055 (host=vd1341.foo.com:22000):(Total: 224.406ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:26/6.31 GB 1:27/6.65 GB 4:28/6.83 GB 0:33/8.06 GB 6:32/7.88 GB 3:34/8.12 GB 7:27/6.62 GB 2:18/4.42 GB Fragment Instance Lifecycle Event Timeline: 1s895ms - Prepare Finished: 191.933ms (191.933ms) - Open Finished: 224.879ms (32.945ms) - First Batch Produced: 1s893ms (1s668ms) - First Batch Sent: 1s895ms (1.752ms) MemoryUsage(8m32s): 12.11 GB, 12.70 GB, 12.69 GB, 12.60 GB, 12.33 GB, 12.09 GB, 11.99 GB, 11.79 GB, 11.10 GB, 10.45 GB, 9.86 GB, 9.49 GB, 9.34 GB, 9.17 GB, 8.78 GB, 8.08 GB, 7.99 GB, 7.91 GB, 7.64 GB, 7.56 GB, 7.75 GB, 8.21 GB, 9.53 GB, 12.06 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB, 12.72 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.72 GB (13654915563) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.47 GB (171226324656) - RowsProduced: 587.16M (587163254) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 22m46s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 32.942ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 31.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7s232ms - OpenTime: 32.944ms - ExecTreeOpenTime: 159.336us - PrepareTime: 191.378ms - ExecTreePrepareTime: 2.264ms KrpcDataStreamSender (dst_id=1):(Total: 11m30s, non-child: 11m30s, % non-child: 100.00%) - BytesSent: 77.06 GB (82743886781) - EosSent: 0 (0) - OverallThroughput: 114.24 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 587.16M (587162230) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m29s - UncompressedRowBatchSize: 151.36 GB (162516640155) HDFS_SCAN_NODE (id=0):(Total: 6s735ms, non-child: 6s735ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:26/6.31 GB 1:27/6.65 GB 4:28/6.83 GB 0:33/8.06 GB 6:32/7.88 GB 3:34/8.12 GB 7:27/6.62 GB 2:18/4.42 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 11.96 GB, 15.19 GB, 15.83 GB, 16.52 GB, 17.77 GB, 18.16 GB, 18.56 GB, 18.69 GB, 18.88 GB, 19.19 GB, 19.25 GB, 19.36 GB, 19.54 GB, 19.58 GB, 19.70 GB, 20.05 GB, 20.43 GB, 20.70 GB, 20.98 GB, 21.66 GB, 22.49 GB, 23.35 GB, 25.72 GB, 30.09 GB, 31.23 GB, 31.36 GB, 31.47 GB, 31.63 GB, 31.92 GB, 32.16 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB, 32.28 GB - FooterProcessingTime: (Avg: 2s191ms ; Min: 35.278ms ; Max: 13s827ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.28 GB (34657824807) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m40s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.71 GB (13651753759) - PerReadThreadRawHdfsThroughput: 56.36 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 587.37M (587368054) - RowsReturned: 587.16M (587163254) - RowsReturnedRate: 87.17 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 6m - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 14s925ms - TotalRawHdfsReadTime(*): 9m46s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 32.486ms, non-child: 32.486ms, % non-child: 100.00%) - CodegenTime: 381.308us - CompileTime: 4.324ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.451ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 20.202ms Instance 844a0200d7876345:20bb38b900000061 (host=va1045.foo.com:22000):(Total: 224.099ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:24/5.83 GB 1:38/9.21 GB 5:27/6.52 GB 4:31/7.49 GB 7:29/7.06 GB 2:32/7.67 GB 3:24/5.72 GB 6:21/5.12 GB Fragment Instance Lifecycle Event Timeline: 1s656ms - Prepare Finished: 189.608ms (189.608ms) - Open Finished: 224.537ms (34.929ms) - First Batch Produced: 1s654ms (1s430ms) - First Batch Sent: 1s656ms (1.928ms) MemoryUsage(8m32s): 13.01 GB, 13.84 GB, 13.78 GB, 13.65 GB, 13.51 GB, 13.21 GB, 13.05 GB, 12.83 GB, 12.47 GB, 11.80 GB, 11.18 GB, 10.74 GB, 10.34 GB, 10.20 GB, 10.03 GB, 9.88 GB, 9.51 GB, 9.26 GB, 8.97 GB, 9.00 GB, 8.97 GB, 8.83 GB, 9.02 GB, 9.43 GB, 10.02 GB, 10.99 GB, 12.30 GB, 13.40 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.92 GB (14944253967) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.66 GB (172508912446) - RowsProduced: 609.56M (609560620) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 33m20s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 34.925ms - TotalThreadsSysTime: 2.000ms - TotalThreadsUserTime: 31.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 12s839ms - OpenTime: 34.927ms - ExecTreeOpenTime: 198.579us - PrepareTime: 189.096ms - ExecTreePrepareTime: 2.103ms KrpcDataStreamSender (dst_id=1):(Total: 11m10s, non-child: 11m10s, % non-child: 100.00%) - BytesSent: 79.97 GB (85862713893) - EosSent: 0 (0) - OverallThroughput: 122.05 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 609.56M (609559596) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m9s - UncompressedRowBatchSize: 157.13 GB (168714948816) HDFS_SCAN_NODE (id=0):(Total: 12s350ms, non-child: 12s350ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:24/5.83 GB 1:38/9.21 GB 5:27/6.52 GB 4:31/7.49 GB 7:29/7.06 GB 2:32/7.67 GB 3:24/5.72 GB 6:21/5.12 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.29 GB, 16.70 GB, 17.35 GB, 18.04 GB, 19.25 GB, 19.82 GB, 20.13 GB, 20.40 GB, 20.54 GB, 20.77 GB, 21.00 GB, 21.09 GB, 21.20 GB, 21.45 GB, 21.66 GB, 21.92 GB, 22.15 GB, 22.60 GB, 22.92 GB, 23.38 GB, 23.82 GB, 24.18 GB, 25.03 GB, 26.11 GB, 27.27 GB, 29.08 GB, 31.40 GB, 33.41 GB, 34.38 GB, 34.53 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB, 34.63 GB - FooterProcessingTime: (Avg: 2s901ms ; Min: 47.283ms ; Max: 15s258ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.63 GB (37181157777) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m53s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.91 GB (14941115651) - PerReadThreadRawHdfsThroughput: 46.92 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 609.77M (609773612) - RowsReturned: 609.56M (609560620) - RowsReturnedRate: 49.35 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m10s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 32s300ms - TotalRawHdfsReadTime(*): 12m35s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 34.387ms, non-child: 34.387ms, % non-child: 100.00%) - CodegenTime: 459.440us - CompileTime: 4.308ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.552ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.044ms Instance 844a0200d7876345:20bb38b90000006d (host=vd1135.foo.com:22000):(Total: 218.341ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:29/7.18 GB 7:26/6.23 GB 3:31/7.63 GB 5:35/8.54 GB 4:33/8.14 GB 1:25/6.19 GB 6:19/4.66 GB 0:26/6.40 GB Fragment Instance Lifecycle Event Timeline: 1s488ms - Prepare Finished: 187.870ms (187.870ms) - Open Finished: 218.816ms (30.946ms) - First Batch Produced: 1s487ms (1s268ms) - First Batch Sent: 1s488ms (1.322ms) MemoryUsage(8m32s): 12.16 GB, 12.74 GB, 12.70 GB, 12.55 GB, 12.29 GB, 12.17 GB, 11.95 GB, 11.45 GB, 10.65 GB, 10.13 GB, 9.71 GB, 9.55 GB, 9.36 GB, 8.96 GB, 8.20 GB, 7.70 GB, 7.45 GB, 7.14 GB, 7.13 GB, 7.76 GB, 8.95 GB, 10.54 GB, 12.32 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.70 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB, 12.71 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.74 GB (13676157250) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.46 GB (171223949275) - RowsProduced: 603.03M (603028767) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 21m24s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 30.944ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 29.996ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7s398ms - OpenTime: 30.945ms - ExecTreeOpenTime: 139.174us - PrepareTime: 187.307ms - ExecTreePrepareTime: 2.203ms KrpcDataStreamSender (dst_id=1):(Total: 11m27s, non-child: 11m27s, % non-child: 100.00%) - BytesSent: 79.17 GB (85010111224) - EosSent: 0 (0) - OverallThroughput: 117.96 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 603.03M (603027743) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m25s - UncompressedRowBatchSize: 155.45 GB (166908720728) HDFS_SCAN_NODE (id=0):(Total: 6s870ms, non-child: 6s870ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:29/7.18 GB 7:26/6.23 GB 3:31/7.63 GB 5:35/8.54 GB 4:33/8.14 GB 1:25/6.19 GB 6:19/4.66 GB 0:26/6.40 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.71 GB, 15.51 GB, 16.14 GB, 17.25 GB, 18.12 GB, 18.49 GB, 18.76 GB, 18.94 GB, 19.24 GB, 19.37 GB, 19.49 GB, 19.66 GB, 19.83 GB, 19.99 GB, 19.99 GB, 20.01 GB, 20.20 GB, 20.51 GB, 21.20 GB, 22.68 GB, 24.81 GB, 27.61 GB, 30.67 GB, 31.33 GB, 31.41 GB, 31.54 GB, 31.74 GB, 31.99 GB, 32.17 GB, 32.46 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.72 GB, 32.73 GB, 32.73 GB, 32.73 GB, 32.73 GB, 32.73 GB, 32.73 GB, 32.73 GB, 32.73 GB, 32.73 GB - FooterProcessingTime: (Avg: 1s853ms ; Min: 26.966ms ; Max: 8s920ms ; Number of samples: 160) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.73 GB (35144435811) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m2s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 160 (160) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.73 GB (13673049779) - PerReadThreadRawHdfsThroughput: 51.77 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 603.23M (603233567) - RowsReturned: 603.03M (603028767) - RowsReturnedRate: 87.77 M/sec - ScanRangesComplete: 80 (80) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m19s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 18s616ms - TotalRawHdfsReadTime(*): 10m47s - TotalReadThroughput: 1.16 MB/sec CodeGen:(Total: 30.513ms, non-child: 30.513ms, % non-child: 100.00%) - CodegenTime: 405.615us - CompileTime: 4.444ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.091ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 17.490ms Instance 844a0200d7876345:20bb38b90000003f (host=vd1120.foo.com:22000):(Total: 217.199ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:27/6.65 GB 3:37/9.08 GB 4:34/8.38 GB 7:22/5.40 GB 5:34/8.26 GB 6:27/6.43 GB 2:22/5.36 GB 0:22/5.38 GB Fragment Instance Lifecycle Event Timeline: 1s976ms - Prepare Finished: 188.986ms (188.986ms) - Open Finished: 217.542ms (28.556ms) - First Batch Produced: 1s974ms (1s756ms) - First Batch Sent: 1s976ms (2.080ms) MemoryUsage(8m32s): 13.11 GB, 13.99 GB, 13.96 GB, 13.87 GB, 13.75 GB, 13.46 GB, 13.28 GB, 13.09 GB, 12.88 GB, 12.32 GB, 11.66 GB, 11.22 GB, 10.71 GB, 10.37 GB, 10.20 GB, 10.21 GB, 9.99 GB, 9.30 GB, 8.80 GB, 8.75 GB, 8.68 GB, 8.66 GB, 8.67 GB, 8.60 GB, 8.76 GB, 9.26 GB, 10.61 GB, 12.68 GB, 13.77 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB, 13.96 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.99 GB (15020274506) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.71 GB (172557075337) - RowsProduced: 601.75M (601745600) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 32m43s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 28.544ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 27.996ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 16s748ms - OpenTime: 28.555ms - ExecTreeOpenTime: 222.101us - PrepareTime: 188.571ms - ExecTreePrepareTime: 1.860ms KrpcDataStreamSender (dst_id=1):(Total: 10m56s, non-child: 10m56s, % non-child: 100.00%) - BytesSent: 78.99 GB (84813161010) - EosSent: 0 (0) - OverallThroughput: 123.13 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 601.74M (601744576) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 10m55s - UncompressedRowBatchSize: 155.12 GB (166553611401) HDFS_SCAN_NODE (id=0):(Total: 16s259ms, non-child: 16s259ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:27/6.65 GB 3:37/9.08 GB 4:34/8.38 GB 7:22/5.40 GB 5:34/8.26 GB 6:27/6.43 GB 2:22/5.36 GB 0:22/5.38 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.98 GB, 16.55 GB, 17.32 GB, 17.94 GB, 19.01 GB, 19.85 GB, 20.12 GB, 20.51 GB, 20.64 GB, 20.79 GB, 21.11 GB, 21.22 GB, 21.26 GB, 21.41 GB, 21.61 GB, 21.89 GB, 22.00 GB, 22.00 GB, 22.29 GB, 22.62 GB, 22.88 GB, 23.39 GB, 24.09 GB, 24.62 GB, 25.22 GB, 26.20 GB, 28.67 GB, 32.27 GB, 34.15 GB, 34.53 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB, 34.58 GB - FooterProcessingTime: (Avg: 2s895ms ; Min: 33.423ms ; Max: 15s515ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.58 GB (37132872398) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m56s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.99 GB (15017163960) - PerReadThreadRawHdfsThroughput: 48.23 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 601.96M (601958592) - RowsReturned: 601.75M (601745600) - RowsReturnedRate: 37.01 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m39s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 24s985ms - TotalRawHdfsReadTime(*): 12m14s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 28.030ms, non-child: 28.030ms, % non-child: 100.00%) - CodegenTime: 427.828us - CompileTime: 4.362ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.636ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 16.538ms Instance 844a0200d7876345:20bb38b900000062 (host=vd1121.foo.com:22000):(Total: 211.997ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:28/6.89 GB 3:31/7.67 GB 2:27/6.53 GB 7:25/6.11 GB 1:24/5.85 GB 0:32/7.92 GB 4:31/7.48 GB 6:24/5.90 GB Fragment Instance Lifecycle Event Timeline: 1s771ms - Prepare Finished: 177.869ms (177.869ms) - Open Finished: 212.490ms (34.620ms) - First Batch Produced: 1s769ms (1s556ms) - First Batch Sent: 1s771ms (2.066ms) MemoryUsage(8m32s): 13.28 GB, 14.01 GB, 14.01 GB, 14.00 GB, 13.86 GB, 13.60 GB, 13.43 GB, 13.39 GB, 13.18 GB, 12.71 GB, 11.89 GB, 11.50 GB, 10.98 GB, 10.65 GB, 10.57 GB, 10.38 GB, 10.08 GB, 9.62 GB, 8.91 GB, 8.66 GB, 8.44 GB, 8.12 GB, 7.79 GB, 7.67 GB, 7.96 GB, 8.70 GB, 9.50 GB, 11.56 GB, 13.56 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 14.01 GB (15041905612) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.69 GB (172535898899) - RowsProduced: 591.50M (591501926) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h13m - TotalStorageWaitTime: 27m12s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 34.612ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 11s462ms - OpenTime: 34.614ms - ExecTreeOpenTime: 155.980us - PrepareTime: 177.318ms - ExecTreePrepareTime: 1.865ms KrpcDataStreamSender (dst_id=1):(Total: 11m3s, non-child: 11m3s, % non-child: 100.00%) - BytesSent: 77.63 GB (83358237312) - EosSent: 0 (0) - OverallThroughput: 119.86 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 591.50M (591500902) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m1s - UncompressedRowBatchSize: 152.47 GB (163718088206) HDFS_SCAN_NODE (id=0):(Total: 10s949ms, non-child: 10s949ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:28/6.89 GB 3:31/7.67 GB 2:27/6.53 GB 7:25/6.11 GB 1:24/5.85 GB 0:32/7.92 GB 4:31/7.48 GB 6:24/5.90 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.93 GB, 16.51 GB, 17.16 GB, 17.76 GB, 18.85 GB, 19.78 GB, 20.10 GB, 20.48 GB, 20.64 GB, 20.80 GB, 21.13 GB, 21.26 GB, 21.34 GB, 21.45 GB, 21.64 GB, 21.68 GB, 21.68 GB, 21.77 GB, 21.92 GB, 22.11 GB, 22.11 GB, 22.21 GB, 22.29 GB, 22.89 GB, 23.93 GB, 25.27 GB, 26.68 GB, 30.32 GB, 33.72 GB, 34.41 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB - FooterProcessingTime: (Avg: 2s169ms ; Min: 15.416ms ; Max: 9s955ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.47 GB (37013838407) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 12m28s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 14.01 GB (15038806050) - PerReadThreadRawHdfsThroughput: 47.33 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 591.71M (591714918) - RowsReturned: 591.50M (591501926) - RowsReturnedRate: 54.02 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 10m31s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 30s390ms - TotalRawHdfsReadTime(*): 12m25s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 34.084ms, non-child: 34.084ms, % non-child: 100.00%) - CodegenTime: 485.213us - CompileTime: 5.030ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.700ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 19.804ms Instance 844a0200d7876345:20bb38b900000031 (host=va1039.foo.com:22000):(Total: 210.113ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 6:28/6.82 GB 0:31/7.67 GB 7:28/6.84 GB 3:30/7.36 GB 5:32/7.53 GB 2:21/5.13 GB 1:22/5.36 GB 4:31/7.64 GB Fragment Instance Lifecycle Event Timeline: 1s666ms - Prepare Finished: 176.909ms (176.909ms) - Open Finished: 210.604ms (33.695ms) - First Batch Produced: 1s663ms (1s453ms) - First Batch Sent: 1s666ms (2.123ms) MemoryUsage(8m32s): 13.14 GB, 13.92 GB, 13.91 GB, 13.84 GB, 13.67 GB, 13.46 GB, 13.30 GB, 13.20 GB, 12.93 GB, 12.39 GB, 11.69 GB, 11.27 GB, 10.75 GB, 10.50 GB, 10.33 GB, 10.04 GB, 9.72 GB, 9.26 GB, 8.72 GB, 8.60 GB, 8.57 GB, 8.45 GB, 8.28 GB, 8.44 GB, 8.99 GB, 9.51 GB, 10.95 GB, 12.68 GB, 13.65 GB, 13.93 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.94 GB (14965251962) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.34 GB (188271562026) - RowsProduced: 599.94M (599937693) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 29m44s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 33.692ms - TotalThreadsSysTime: 4.000ms - TotalThreadsUserTime: 28.996ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 7s514ms - OpenTime: 33.694ms - ExecTreeOpenTime: 160.407us - PrepareTime: 176.346ms - ExecTreePrepareTime: 2.049ms KrpcDataStreamSender (dst_id=1):(Total: 11m10s, non-child: 11m10s, % non-child: 100.00%) - BytesSent: 78.74 GB (84551262154) - EosSent: 0 (0) - OverallThroughput: 120.19 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 599.94M (599936669) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m9s - UncompressedRowBatchSize: 154.65 GB (166052081107) HDFS_SCAN_NODE (id=0):(Total: 7s022ms, non-child: 7s022ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:28/6.82 GB 0:31/7.67 GB 7:28/6.84 GB 3:30/7.36 GB 5:32/7.53 GB 2:21/5.13 GB 1:22/5.36 GB 4:31/7.64 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.05 GB, 16.57 GB, 17.23 GB, 17.88 GB, 18.95 GB, 19.85 GB, 20.17 GB, 20.50 GB, 20.68 GB, 20.85 GB, 21.13 GB, 21.28 GB, 21.35 GB, 21.50 GB, 21.62 GB, 21.65 GB, 21.67 GB, 21.97 GB, 22.07 GB, 22.40 GB, 22.80 GB, 23.14 GB, 23.52 GB, 24.40 GB, 25.61 GB, 26.58 GB, 29.12 GB, 32.13 GB, 33.85 GB, 34.40 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB - FooterProcessingTime: (Avg: 2s569ms ; Min: 47.128ms ; Max: 11s562ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.03 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.47 GB (37015046276) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m23s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.93 GB (14962087920) - PerReadThreadRawHdfsThroughput: 49.99 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 600.15M (600150685) - RowsReturned: 599.94M (599937693) - RowsReturnedRate: 85.43 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m56s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 26s523ms - TotalRawHdfsReadTime(*): 11m46s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 33.265ms, non-child: 33.265ms, % non-child: 100.00%) - CodegenTime: 470.435us - CompileTime: 4.286ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.776ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 20.671ms Instance 844a0200d7876345:20bb38b90000002d (host=vd1123.foo.com:22000):(Total: 208.976ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:29/6.83 GB 3:30/7.35 GB 5:26/6.39 GB 2:24/5.91 GB 6:23/5.62 GB 4:34/8.27 GB 0:26/6.29 GB 7:33/7.98 GB Fragment Instance Lifecycle Event Timeline: 1s972ms - Prepare Finished: 174.250ms (174.250ms) - Open Finished: 209.692ms (35.441ms) - First Batch Produced: 1s970ms (1s760ms) - First Batch Sent: 1s972ms (2.076ms) MemoryUsage(8m32s): 13.26 GB, 13.96 GB, 13.93 GB, 13.86 GB, 13.61 GB, 13.27 GB, 13.18 GB, 13.06 GB, 12.75 GB, 12.03 GB, 11.39 GB, 10.89 GB, 10.46 GB, 10.34 GB, 10.25 GB, 10.03 GB, 9.63 GB, 8.96 GB, 8.88 GB, 8.78 GB, 8.61 GB, 8.31 GB, 8.11 GB, 8.42 GB, 9.04 GB, 10.08 GB, 11.90 GB, 13.43 GB, 13.83 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB, 13.86 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.96 GB (14986673088) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.28 GB (188204366857) - RowsProduced: 602.68M (602679109) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h14m - TotalStorageWaitTime: 25m8s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 35.437ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 32.995ms - TotalThreadsVoluntaryContextSwitches: 2 (2) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 11s590ms - OpenTime: 35.439ms - ExecTreeOpenTime: 172.253us - PrepareTime: 173.427ms - ExecTreePrepareTime: 2.064ms KrpcDataStreamSender (dst_id=1):(Total: 11m8s, non-child: 11m8s, % non-child: 100.00%) - BytesSent: 79.12 GB (84951207424) - EosSent: 0 (0) - OverallThroughput: 121.14 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 602.68M (602678085) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m7s - UncompressedRowBatchSize: 155.36 GB (166811519339) HDFS_SCAN_NODE (id=0):(Total: 11s118ms, non-child: 11s118ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:29/6.83 GB 3:30/7.35 GB 5:26/6.39 GB 2:24/5.91 GB 6:23/5.62 GB 4:34/8.27 GB 0:26/6.29 GB 7:33/7.98 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.42 GB, 16.76 GB, 17.34 GB, 17.97 GB, 19.21 GB, 19.79 GB, 20.10 GB, 20.39 GB, 20.52 GB, 20.75 GB, 20.99 GB, 21.06 GB, 21.16 GB, 21.40 GB, 21.59 GB, 21.70 GB, 21.84 GB, 21.98 GB, 22.51 GB, 22.75 GB, 22.99 GB, 23.14 GB, 23.46 GB, 24.49 GB, 25.76 GB, 27.61 GB, 30.81 GB, 33.42 GB, 34.18 GB, 34.36 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.45 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB, 34.46 GB - FooterProcessingTime: (Avg: 2s331ms ; Min: 29.622ms ; Max: 8s680ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.46 GB (37001657745) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 10m19s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.95 GB (14983483446) - PerReadThreadRawHdfsThroughput: 52.31 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 602.89M (602892101) - RowsReturned: 602.68M (602679109) - RowsReturnedRate: 54.21 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m43s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 24s813ms - TotalRawHdfsReadTime(*): 11m14s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 34.942ms, non-child: 34.942ms, % non-child: 100.00%) - CodegenTime: 441.757us - CompileTime: 4.211ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 9.364ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 20.850ms Instance 844a0200d7876345:20bb38b900000030 (host=vd1124.foo.com:22000):(Total: 204.156ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:29/7.14 GB 2:37/9.12 GB 4:30/7.34 GB 6:25/6.15 GB 5:23/5.57 GB 0:33/8.09 GB 3:24/5.84 GB 7:23/5.52 GB Fragment Instance Lifecycle Event Timeline: 1s807ms - Prepare Finished: 167.801ms (167.801ms) - Open Finished: 204.644ms (36.842ms) - First Batch Produced: 1s805ms (1s600ms) - First Batch Sent: 1s807ms (2.106ms) MemoryUsage(8m32s): 13.26 GB, 13.98 GB, 13.97 GB, 13.94 GB, 13.76 GB, 13.44 GB, 13.31 GB, 13.22 GB, 12.95 GB, 12.35 GB, 11.62 GB, 11.17 GB, 10.65 GB, 10.46 GB, 10.32 GB, 10.04 GB, 9.67 GB, 9.11 GB, 8.89 GB, 8.80 GB, 8.52 GB, 8.22 GB, 8.02 GB, 8.23 GB, 8.90 GB, 9.62 GB, 11.12 GB, 13.06 GB, 13.92 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB, 13.95 GB ThreadUsage(8m32s): 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.98 GB (15008623305) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.37 GB (188307257381) - RowsProduced: 600.26M (600259626) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 25m55s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 36.840ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 33.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s629ms - OpenTime: 36.841ms - ExecTreeOpenTime: 249.724us - PrepareTime: 167.240ms - ExecTreePrepareTime: 2.122ms KrpcDataStreamSender (dst_id=1):(Total: 11m8s, non-child: 11m8s, % non-child: 100.00%) - BytesSent: 78.76 GB (84570516285) - EosSent: 0 (0) - OverallThroughput: 120.70 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 600.26M (600258602) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m6s - UncompressedRowBatchSize: 154.73 GB (166141901036) HDFS_SCAN_NODE (id=0):(Total: 10s129ms, non-child: 10s129ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:29/7.14 GB 2:37/9.12 GB 4:30/7.34 GB 6:25/6.15 GB 5:23/5.57 GB 0:33/8.09 GB 3:24/5.84 GB 7:23/5.52 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.12 GB, 16.63 GB, 17.27 GB, 17.89 GB, 19.06 GB, 19.81 GB, 20.12 GB, 20.46 GB, 20.60 GB, 20.79 GB, 21.08 GB, 21.18 GB, 21.26 GB, 21.41 GB, 21.53 GB, 21.55 GB, 21.60 GB, 21.85 GB, 22.37 GB, 22.64 GB, 22.68 GB, 22.82 GB, 23.16 GB, 24.13 GB, 25.48 GB, 26.76 GB, 29.42 GB, 32.74 GB, 34.27 GB, 34.48 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB, 34.59 GB - FooterProcessingTime: (Avg: 2s052ms ; Min: 44.113ms ; Max: 8s584ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.59 GB (37144807039) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 7m46s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (15005294427) - PerReadThreadRawHdfsThroughput: 55.72 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 600.47M (600472618) - RowsReturned: 600.26M (600259626) - RowsReturnedRate: 59.26 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m6s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 21s748ms - TotalRawHdfsReadTime(*): 10m35s - TotalReadThroughput: 1.23 MB/sec CodeGen:(Total: 36.208ms, non-child: 36.208ms, % non-child: 100.00%) - CodegenTime: 487.960us - CompileTime: 4.552ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 8.427ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 22.710ms Instance 844a0200d7876345:20bb38b90000002f (host=va1043.foo.com:22000):(Total: 202.396ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:17/4.17 GB 3:21/5.16 GB 5:31/7.57 GB 2:27/6.61 GB 0:23/5.54 GB 6:35/8.50 GB 1:31/7.44 GB 4:39/9.37 GB Fragment Instance Lifecycle Event Timeline: 1s369ms - Prepare Finished: 171.870ms (171.870ms) - Open Finished: 202.795ms (30.925ms) - First Batch Produced: 1s367ms (1s165ms) - First Batch Sent: 1s369ms (2.129ms) MemoryUsage(8m32s): 13.26 GB, 13.93 GB, 13.93 GB, 13.89 GB, 13.73 GB, 13.39 GB, 13.28 GB, 13.20 GB, 12.97 GB, 12.34 GB, 11.70 GB, 11.22 GB, 10.79 GB, 10.70 GB, 10.58 GB, 10.33 GB, 9.84 GB, 9.04 GB, 8.72 GB, 8.58 GB, 8.32 GB, 7.94 GB, 7.73 GB, 7.82 GB, 8.56 GB, 9.57 GB, 11.53 GB, 13.41 GB, 13.94 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.96 GB (14993490801) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.36 GB (188286936910) - RowsProduced: 600.66M (600656114) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 25m50s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 30.921ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 27.996ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 15s290ms - OpenTime: 30.922ms - ExecTreeOpenTime: 316.608us - PrepareTime: 171.413ms - ExecTreePrepareTime: 1.947ms KrpcDataStreamSender (dst_id=1):(Total: 10m48s, non-child: 10m48s, % non-child: 100.00%) - BytesSent: 78.86 GB (84672874455) - EosSent: 0 (0) - OverallThroughput: 124.50 MB/sec - PeakMemoryUsage: 3.58 MB (3752608) - RowsReturned: 600.66M (600655090) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 10m47s - UncompressedRowBatchSize: 154.83 GB (166251682980) HDFS_SCAN_NODE (id=0):(Total: 14s851ms, non-child: 14s851ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:17/4.17 GB 3:21/5.16 GB 5:31/7.57 GB 2:27/6.61 GB 0:23/5.54 GB 6:35/8.50 GB 1:31/7.44 GB 4:39/9.37 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 13.01 GB, 16.55 GB, 17.20 GB, 17.76 GB, 19.06 GB, 19.78 GB, 20.10 GB, 20.43 GB, 20.60 GB, 20.94 GB, 21.22 GB, 21.28 GB, 21.48 GB, 21.73 GB, 21.82 GB, 21.83 GB, 21.86 GB, 21.87 GB, 22.10 GB, 22.23 GB, 22.45 GB, 22.53 GB, 22.91 GB, 23.58 GB, 24.96 GB, 26.75 GB, 30.23 GB, 33.41 GB, 34.32 GB, 34.40 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB, 34.48 GB - FooterProcessingTime: (Avg: 2s101ms ; Min: 15.847ms ; Max: 9s214ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.48 GB (37025260261) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m46s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.96 GB (14990354340) - PerReadThreadRawHdfsThroughput: 50.61 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 600.87M (600869106) - RowsReturned: 600.66M (600656114) - RowsReturnedRate: 40.44 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 9m38s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 25s709ms - TotalRawHdfsReadTime(*): 11m37s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 30.301ms, non-child: 30.301ms, % non-child: 100.00%) - CodegenTime: 406.950us - CompileTime: 4.320ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.008ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 18.362ms Instance 844a0200d7876345:20bb38b900000059 (host=vc1542.foo.com:22000):(Total: 197.203ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:25/6.07 GB 7:29/6.92 GB 0:25/6.15 GB 3:21/5.20 GB 1:28/6.89 GB 6:36/8.88 GB 2:29/7.13 GB 4:32/7.78 GB Fragment Instance Lifecycle Event Timeline: 1s604ms - Prepare Finished: 167.655ms (167.655ms) - Open Finished: 197.617ms (29.961ms) - First Batch Produced: 1s602ms (1s405ms) - First Batch Sent: 1s604ms (2.024ms) MemoryUsage(8m32s): 13.24 GB, 13.97 GB, 13.97 GB, 13.94 GB, 13.77 GB, 13.50 GB, 13.32 GB, 13.28 GB, 13.14 GB, 12.75 GB, 11.99 GB, 11.47 GB, 10.96 GB, 10.54 GB, 10.42 GB, 10.30 GB, 10.07 GB, 9.90 GB, 9.36 GB, 8.99 GB, 8.80 GB, 8.55 GB, 8.22 GB, 7.97 GB, 8.01 GB, 8.51 GB, 9.27 GB, 10.85 GB, 12.89 GB, 13.79 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB, 13.93 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (15005432580) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 160.66 GB (172511153594) - RowsProduced: 588.12M (588119603) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h15m - TotalStorageWaitTime: 26m36s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 29.957ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 27.996ms - TotalThreadsVoluntaryContextSwitches: 1 (1) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s183ms - OpenTime: 29.958ms - ExecTreeOpenTime: 144.980us - PrepareTime: 167.173ms - ExecTreePrepareTime: 2.045ms KrpcDataStreamSender (dst_id=1):(Total: 11m13s, non-child: 11m13s, % non-child: 100.00%) - BytesSent: 77.17 GB (82864613454) - EosSent: 0 (0) - OverallThroughput: 117.28 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 588.12M (588118579) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m12s - UncompressedRowBatchSize: 151.60 GB (162781373662) HDFS_SCAN_NODE (id=0):(Total: 9s670ms, non-child: 9s670ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:25/6.07 GB 7:29/6.92 GB 0:25/6.15 GB 3:21/5.20 GB 1:28/6.89 GB 6:36/8.88 GB 2:29/7.13 GB 4:32/7.78 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.87 GB, 16.36 GB, 17.09 GB, 17.63 GB, 18.62 GB, 19.59 GB, 19.95 GB, 20.29 GB, 20.48 GB, 20.61 GB, 20.88 GB, 21.08 GB, 21.15 GB, 21.25 GB, 21.42 GB, 21.50 GB, 21.60 GB, 22.00 GB, 22.28 GB, 22.41 GB, 22.51 GB, 22.61 GB, 22.69 GB, 23.02 GB, 23.65 GB, 24.73 GB, 26.10 GB, 28.96 GB, 32.50 GB, 34.11 GB, 34.40 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB, 34.41 GB - FooterProcessingTime: (Avg: 2s283ms ; Min: 15.840ms ; Max: 11s698ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.41 GB (36944057165) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m18s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (15002259344) - PerReadThreadRawHdfsThroughput: 52.10 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 588.33M (588332595) - RowsReturned: 588.12M (588119603) - RowsReturnedRate: 60.82 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 7m12s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 19s402ms - TotalRawHdfsReadTime(*): 11m16s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 29.548ms, non-child: 29.548ms, % non-child: 100.00%) - CodegenTime: 413.620us - CompileTime: 4.178ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.777ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 16.892ms Instance 844a0200d7876345:20bb38b900000015 (host=va1027.foo.com:22000):(Total: 191.805ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 6:23/5.66 GB 1:23/5.58 GB 0:32/7.76 GB 3:30/7.27 GB 5:29/7.06 GB 2:27/6.68 GB 7:30/7.43 GB 4:29/7.03 GB Fragment Instance Lifecycle Event Timeline: 1s852ms - Prepare Finished: 162.377ms (162.377ms) - Open Finished: 192.392ms (30.014ms) - First Batch Produced: 1s850ms (1s657ms) - First Batch Sent: 1s852ms (1.901ms) MemoryUsage(8m32s): 13.15 GB, 13.92 GB, 13.92 GB, 13.85 GB, 13.69 GB, 13.37 GB, 13.18 GB, 13.10 GB, 12.87 GB, 12.42 GB, 11.71 GB, 11.29 GB, 10.74 GB, 10.44 GB, 10.39 GB, 10.28 GB, 10.11 GB, 9.52 GB, 9.01 GB, 8.89 GB, 8.81 GB, 8.61 GB, 8.26 GB, 8.10 GB, 8.43 GB, 9.24 GB, 10.63 GB, 12.51 GB, 13.85 GB, 13.93 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB, 13.92 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.94 GB (14967496499) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.34 GB (188273043643) - RowsProduced: 598.56M (598562930) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 31m23s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 29.999ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 28.995ms - TotalThreadsVoluntaryContextSwitches: 0 (0) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 6s023ms - OpenTime: 30.001ms - ExecTreeOpenTime: 282.367us - PrepareTime: 161.733ms - ExecTreePrepareTime: 1.614ms KrpcDataStreamSender (dst_id=1):(Total: 11m5s, non-child: 11m5s, % non-child: 100.00%) - BytesSent: 78.54 GB (84333733153) - EosSent: 0 (0) - OverallThroughput: 120.87 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 598.56M (598561906) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m4s - UncompressedRowBatchSize: 154.29 GB (165672030286) HDFS_SCAN_NODE (id=0):(Total: 5s573ms, non-child: 5s573ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:23/5.66 GB 1:23/5.58 GB 0:32/7.76 GB 3:30/7.27 GB 5:29/7.06 GB 2:27/6.68 GB 7:30/7.43 GB 4:29/7.03 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.89 GB, 16.43 GB, 17.18 GB, 17.72 GB, 18.82 GB, 19.67 GB, 19.98 GB, 20.33 GB, 20.46 GB, 20.74 GB, 21.07 GB, 21.18 GB, 21.24 GB, 21.39 GB, 21.63 GB, 21.79 GB, 22.05 GB, 22.16 GB, 22.40 GB, 22.62 GB, 22.92 GB, 23.20 GB, 23.29 GB, 23.69 GB, 24.64 GB, 26.10 GB, 28.58 GB, 31.91 GB, 34.19 GB, 34.41 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB - FooterProcessingTime: (Avg: 2s606ms ; Min: 30.436ms ; Max: 13s851ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.47 GB (37013159826) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 9m49s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.94 GB (14964405125) - PerReadThreadRawHdfsThroughput: 56.87 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 598.78M (598775922) - RowsReturned: 598.56M (598562930) - RowsReturnedRate: 107.39 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m6s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 22s994ms - TotalRawHdfsReadTime(*): 10m20s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 29.414ms, non-child: 29.414ms, % non-child: 100.00%) - CodegenTime: 424.112us - CompileTime: 3.918ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.757ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 18.115ms Instance 844a0200d7876345:20bb38b900000079 (host=vd1332.foo.com:22000):(Total: 131.851ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:28/6.77 GB 6:33/7.99 GB 1:27/6.29 GB 5:23/5.58 GB 2:25/6.03 GB 0:32/7.84 GB 3:23/5.62 GB 7:37/8.59 GB Fragment Instance Lifecycle Event Timeline: 1s617ms - Prepare Finished: 95.819ms (95.819ms) - Open Finished: 132.504ms (36.684ms) - First Batch Produced: 1s615ms (1s482ms) - First Batch Sent: 1s617ms (2.018ms) MemoryUsage(8m32s): 12.04 GB, 12.69 GB, 12.69 GB, 12.61 GB, 12.30 GB, 12.06 GB, 12.01 GB, 11.78 GB, 11.11 GB, 10.41 GB, 9.83 GB, 9.48 GB, 9.40 GB, 9.15 GB, 8.75 GB, 8.22 GB, 8.16 GB, 7.96 GB, 7.55 GB, 7.34 GB, 7.70 GB, 8.55 GB, 10.19 GB, 12.28 GB, 12.57 GB, 12.57 GB, 12.56 GB, 12.55 GB, 12.55 GB, 12.55 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB, 12.64 GB ThreadUsage(8m32s): 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 12.70 GB (13632925012) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 159.41 GB (171166687857) - RowsProduced: 589.38M (589384748) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h17m - TotalStorageWaitTime: 22m21s - TotalThreadsInvoluntaryContextSwitches: 4 (4) - TotalThreadsTotalWallClockTime: 36.660ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 35.994ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s096ms - OpenTime: 36.661ms - ExecTreeOpenTime: 258.505us - PrepareTime: 95.112ms - ExecTreePrepareTime: 2.329ms KrpcDataStreamSender (dst_id=1):(Total: 11m31s, non-child: 11m31s, % non-child: 100.00%) - BytesSent: 77.38 GB (83090341557) - EosSent: 0 (0) - OverallThroughput: 114.67 MB/sec - PeakMemoryUsage: 3.61 MB (3785376) - RowsReturned: 589.38M (589383724) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m29s - UncompressedRowBatchSize: 151.93 GB (163130828311) HDFS_SCAN_NODE (id=0):(Total: 9s589ms, non-child: 9s589ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:28/6.77 GB 6:33/7.99 GB 1:27/6.29 GB 5:23/5.58 GB 2:25/6.03 GB 0:32/7.84 GB 3:23/5.62 GB 7:37/8.59 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 11.96 GB, 15.20 GB, 15.77 GB, 16.51 GB, 17.73 GB, 18.16 GB, 18.52 GB, 18.66 GB, 18.89 GB, 19.16 GB, 19.23 GB, 19.36 GB, 19.52 GB, 19.55 GB, 19.77 GB, 20.22 GB, 20.64 GB, 20.84 GB, 20.94 GB, 21.37 GB, 22.38 GB, 23.91 GB, 26.81 GB, 30.50 GB, 31.07 GB, 31.21 GB, 31.36 GB, 31.54 GB, 31.78 GB, 31.92 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB, 32.14 GB - FooterProcessingTime: (Avg: 1s902ms ; Min: 20.630ms ; Max: 9s459ms ; Number of samples: 161) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 32.14 GB (34510518396) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 5m40s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 161 (161) - NumScannerThreadsStarted: 80 (80) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 12.69 GB (13629899210) - PerReadThreadRawHdfsThroughput: 51.28 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 589.59M (589589548) - RowsReturned: 589.38M (589384748) - RowsReturnedRate: 61.46 M/sec - ScanRangesComplete: 81 (81) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 5m3s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 24s747ms - TotalRawHdfsReadTime(*): 10m41s - TotalReadThroughput: 1.14 MB/sec CodeGen:(Total: 36.105ms, non-child: 36.105ms, % non-child: 100.00%) - CodegenTime: 427.367us - CompileTime: 4.415ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 7.667ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 23.510ms Instance 844a0200d7876345:20bb38b900000017 (host=va1031.foo.com:22000):(Total: 119.043ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:32/7.89 GB 0:31/7.45 GB 2:28/6.84 GB 4:29/7.09 GB 3:30/7.31 GB 7:27/6.59 GB 5:26/6.40 GB 6:21/5.12 GB Fragment Instance Lifecycle Event Timeline: 1s380ms - Prepare Finished: 88.142ms (88.142ms) - Open Finished: 119.587ms (31.444ms) - First Batch Produced: 1s378ms (1s258ms) - First Batch Sent: 1s380ms (2.277ms) MemoryUsage(8m32s): 13.14 GB, 13.96 GB, 13.96 GB, 13.92 GB, 13.76 GB, 13.47 GB, 13.24 GB, 13.17 GB, 13.02 GB, 12.60 GB, 11.86 GB, 11.36 GB, 10.84 GB, 10.41 GB, 10.25 GB, 10.15 GB, 10.02 GB, 9.66 GB, 9.23 GB, 8.95 GB, 8.87 GB, 8.62 GB, 8.58 GB, 8.43 GB, 8.33 GB, 8.94 GB, 9.81 GB, 11.53 GB, 13.26 GB, 13.95 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB, 13.97 GB ThreadUsage(8m32s): 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 13.97 GB (14998270930) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 175.39 GB (188323580396) - RowsProduced: 592.55M (592552849) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 6h16m - TotalStorageWaitTime: 31m3s - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 31.439ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 30.995ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 0.000ns - ExecTreeExecTime: 10s274ms - OpenTime: 31.441ms - ExecTreeOpenTime: 182.800us - PrepareTime: 87.537ms - ExecTreePrepareTime: 1.747ms KrpcDataStreamSender (dst_id=1):(Total: 11m13s, non-child: 11m13s, % non-child: 100.00%) - BytesSent: 77.76 GB (83491377612) - EosSent: 0 (0) - OverallThroughput: 118.15 MB/sec - PeakMemoryUsage: 3.59 MB (3768992) - RowsReturned: 592.55M (592551825) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 11m11s - UncompressedRowBatchSize: 152.75 GB (164008911198) HDFS_SCAN_NODE (id=0):(Total: 9s790ms, non-child: 9s790ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:32/7.89 GB 0:31/7.45 GB 2:28/6.84 GB 4:29/7.09 GB 3:30/7.31 GB 7:27/6.59 GB 5:26/6.40 GB 6:21/5.12 GB ExecOption: PARQUET Codegen Enabled BytesRead(8m32s): 12.77 GB, 16.25 GB, 17.08 GB, 17.64 GB, 18.54 GB, 19.56 GB, 19.91 GB, 20.26 GB, 20.46 GB, 20.58 GB, 20.85 GB, 21.04 GB, 21.10 GB, 21.19 GB, 21.36 GB, 21.47 GB, 21.70 GB, 21.92 GB, 22.34 GB, 22.57 GB, 22.78 GB, 22.93 GB, 23.45 GB, 23.93 GB, 24.25 GB, 25.48 GB, 27.04 GB, 30.10 GB, 33.12 GB, 34.38 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB, 34.47 GB - FooterProcessingTime: (Avg: 2s516ms ; Min: 31.029ms ; Max: 16s786ms ; Number of samples: 176) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 34.47 GB (37009722763) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 8m23s - MaxCompressedTextFileLength: 0 - NumColumns: 15 (15) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 0 (0) - NumRowGroups: 176 (176) - NumScannerThreadsStarted: 88 (88) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 13.97 GB (14995203987) - PerReadThreadRawHdfsThroughput: 51.72 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 592.77M (592765841) - RowsReturned: 592.55M (592552849) - RowsReturnedRate: 60.52 M/sec - ScanRangesComplete: 88 (88) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 8m3s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsOpenFileTime(*): 29s732ms - TotalRawHdfsReadTime(*): 11m22s - TotalReadThroughput: 1.22 MB/sec CodeGen:(Total: 30.964ms, non-child: 30.964ms, % non-child: 100.00%) - CodegenTime: 402.226us - CompileTime: 4.549ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.95 MB (2039504) - NumFunctions: 7 (7) - NumInstructions: 105 (105) - OptimizationTime: 6.955ms - PeakMemoryUsage: 52.50 KB (53760) - PrepareTime: 18.952ms