Query (id=8d4071b24aee8531:393af00400000000): Summary: Session ID: a64396565f6c320a:59a644927500cdbf Session Type: BEESWAX Start Time: 2017-04-14 16:26:00.649216000 End Time: 2017-04-14 16:26:47.943554000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.9.0-SNAPSHOT RELEASE (build f0c48ad47dbf981709c914c5506dc5358455a486) User: mmokhtar Connected User: mmokhtar Delegated User: Network Address: 10.20.122.101:58456 Default Db: scan_primitives_tpch_3tb Sql Statement: select count(*) from customer c, c.c_orders o, o.o_lineitems l where c_mktsegment = 'BUILDING' and o_orderdate < '1995-03-15' and l_shipdate > '1995-03-15' and o_orderkey = 10 Coordinator: foo01.boo:22000 Query Options (non default): Plan: ---------------- Estimated Per-Host Requirements: Memory=186.00MB VCores=1 WARNING: The following tables are missing relevant table and/or column statistics. scan_primitives_tpch_3tb.customer PLAN-ROOT SINK | 11:AGGREGATE [FINALIZE] | output: count:merge(*) | hosts=20 per-host-mem=unavailable | tuple-ids=3 row-size=8B cardinality=1 | 10:EXCHANGE [UNPARTITIONED] | hosts=20 per-host-mem=unavailable | tuple-ids=3 row-size=8B cardinality=1 | 09:AGGREGATE | output: count(*) | hosts=20 per-host-mem=10.00MB | tuple-ids=3 row-size=8B cardinality=1 | 01:SUBPLAN | hosts=20 per-host-mem=0B | tuple-ids=2,1,0 row-size=97B cardinality=9000000000 | |--08:NESTED LOOP JOIN [CROSS JOIN] | | hosts=20 per-host-mem=41B | | tuple-ids=2,1,0 row-size=97B cardinality=100 | | | |--02:SINGULAR ROW SRC | | parent-subplan=01 | | hosts=20 per-host-mem=0B | | tuple-ids=0 row-size=41B cardinality=1 | | | 04:SUBPLAN | | hosts=20 per-host-mem=0B | | tuple-ids=2,1 row-size=56B cardinality=100 | | | |--07:NESTED LOOP JOIN [CROSS JOIN] | | | hosts=20 per-host-mem=40B | | | tuple-ids=2,1 row-size=56B cardinality=10 | | | | | |--05:SINGULAR ROW SRC | | | parent-subplan=04 | | | hosts=20 per-host-mem=0B | | | tuple-ids=1 row-size=40B cardinality=1 | | | | | 06:UNNEST [o.o_lineitems l] | | parent-subplan=04 | | hosts=20 per-host-mem=0B | | tuple-ids=2 row-size=0B cardinality=10 | | | 03:UNNEST [c.c_orders o] | parent-subplan=01 | hosts=20 per-host-mem=0B | tuple-ids=1 row-size=0B cardinality=10 | 00:SCAN HDFS [scan_primitives_tpch_3tb.customer c, RANDOM] partitions=1/1 files=1099 size=1.72TB predicates: c_mktsegment = 'BUILDING', !empty(c.c_orders) predicates on o: !empty(o.o_lineitems), o_orderdate < '1995-03-15', o_orderkey = 10 predicates on l: l_shipdate > '1995-03-15' table stats: 450000000 rows total columns missing stats: c_orders parquet statistics predicates: c_mktsegment = 'BUILDING' parquet dictionary predicates: c_mktsegment = 'BUILDING' hosts=20 per-host-mem=176.00MB tuple-ids=0 row-size=41B cardinality=90000000 ---------------- Estimated Per-Host Mem: 195035217 Estimated Per-Host VCores: 1 Tables Missing Stats: scan_primitives_tpch_3tb.customer Request Pool: root.default ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ----------------------------------------------------------------------------------------------------------------------------------------- 11:AGGREGATE 1 234.529us 234.529us 1 1 96.00 KB -1.00 B FINALIZE 10:EXCHANGE 1 365.097us 365.097us 20 1 0 -1.00 B UNPARTITIONED 09:AGGREGATE 20 8s656ms 9s068ms 20 1 26.02 MB 10.00 MB 01:SUBPLAN 20 33.754ms 61.643ms 0 9.00B 14.00 MB 0 |--08:NESTED LOOP JOIN 20 102.625us 139.464us 0 100 24.00 KB 41.00 B CROSS JOIN | |--02:SINGULAR ROW SRC 20 0.000ns 0.000ns 0 1 0 0 | 04:SUBPLAN 20 0.000ns 0.000ns 0 100 8.00 KB 0 | |--07:NESTED LOOP JOIN 20 61.664us 83.166us 0 10 16.00 KB 40.00 B CROSS JOIN | | |--05:SINGULAR ROW SRC 20 0.000ns 0.000ns 0 1 0 0 | | 06:UNNEST 20 8.768us 16.312us 0 10 0 0 o.o_lineitems l | 03:UNNEST 20 11.023us 14.380us 0 10 0 0 c.c_orders o 00:SCAN HDFS 20 34s356ms 37s480ms 0 90.00M 459.33 MB 176.00 MB scan_primitives_tpch_3tb.cu... Errors: Planner Timeline: 19.402ms - Analysis finished: 3.354ms (3.354ms) - Equivalence classes computed: 3.630ms (275.900us) - Single node plan created: 15.780ms (12.150ms) - Runtime filters computed: 15.885ms (105.566us) - Distributed plan created: 16.118ms (232.238us) - Lineage info computed: 16.323ms (205.874us) - Planning finished: 19.402ms (3.078ms) Query Timeline: 47s329ms - Query submitted: 343.840us (343.840us) - Planning finished: 62.082ms (61.739ms) - Ready to start 21 fragment instances: 106.928ms (44.845ms) - All 21 fragment instances started: 117.619ms (10.691ms) - Rows available: 46s857ms (46s739ms) - First row fetched: 47s284ms (427.287ms) - Unregister query: 47s294ms (9.679ms) - ComputeScanRangeAssignmentTimer: 28.031ms ImpalaServer: - ClientFetchWaitTimer: 430.943ms - RowMaterializationTimer: 5.995ms Execution Profile 8d4071b24aee8531:393af00400000000:(Total: 46s754ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed Enabled ---------------------------------------------------------------------------------------------------------------------------- Fragment instance start latencies: Count: 21, min / max: 1ms / 5ms, 25th %-ile: 3ms, 50th %-ile: 3ms, 75th %-ile: 3ms, 90th %-ile: 3ms, 95th %-ile: 4ms, 99.9th %-ile: 5ms Per Node Peak Memory Usage: foo16.boo:22000(448.70 MB) foo07.boo:22000(460.93 MB) foo17.boo:22000(447.91 MB) foo02.boo:22000(452.13 MB) foo18.boo:22000(451.76 MB) foo20.boo:22000(447.83 MB) foo04.boo:22000(462.15 MB) foo10.boo:22000(459.37 MB) foo03.boo:22000(445.80 MB) foo21.boo:22000(435.80 MB) foo08.boo:22000(457.80 MB) foo15.boo:22000(455.57 MB) foo05.boo:22000(450.74 MB) foo14.boo:22000(457.92 MB) foo11.boo:22000(451.22 MB) foo13.boo:22000(452.09 MB) foo06.boo:22000(454.90 MB) foo09.boo:22000(450.62 MB) foo19.boo:22000(452.90 MB) foo01.boo:22000(449.94 MB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Coordinator Fragment F01: Instance 8d4071b24aee8531:393af00400000000 (host=foo01.boo:22000):(Total: 47s176ms, non-child: 427.997ms, % non-child: 0.91%) MemoryUsage(1s000ms): 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 36.01 KB, 82.01 KB, 98.01 KB, 100.01 KB, 100.01 KB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 104.02 KB (106512) - PerHostPeakMemUsage: 449.94 MB (471797199) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 46s725ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 47s153ms - TotalThreadsSysTime: 999.000us - TotalThreadsUserTime: 1.000ms - TotalThreadsVoluntaryContextSwitches: 24 (24) Fragment Instance Lifecycle Timings: - ExecTime: 427.382ms - ExecTreeExecTime: 6.374us - OpenTime: 46s725ms - ExecTreeOpenTime: 46s725ms - PrepareTime: 23.407ms - ExecTreePrepareTime: 96.872us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 22.985ms, non-child: 22.985ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 22.349ms AGGREGATION_NODE (id=11):(Total: 46s725ms, non-child: 234.529us, % non-child: 0.00%) - BuildTime: 53.149us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 96.00 KB (98304) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=10):(Total: 46s725ms, non-child: 365.097us, % non-child: 0.00%) BytesReceived(1s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 48.00 B, 232.00 B, 296.00 B, 304.00 B, 304.00 B - BytesReceived: 320.00 B (320) - ConvertRowBatchTime: 47.774us - DeserializeRowBatchTimer: 106.329us - FirstBatchArrivalWaitTime: 42s022ms - PeakMemoryUsage: 0 - RowsReturned: 20 (20) - RowsReturnedRate: 0 - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 43s184ms, non-child: 3.445ms, % non-child: 0.01%) split sizes: min: 88.07 GB, max: 88.26 GB, avg: 88.14 GB, stddev: 53.79 MB completion times: min:42s040ms max:46s741ms mean: 43s186ms stddev:1s017ms execution rates: min:1.89 GB/sec max:2.10 GB/sec mean:2.04 GB/sec stddev:47.07 MB/sec num instances: 20 - AverageThreadTokens: 24.74 - BloomFilterBytes: 0 - PeakMemoryUsage: 452.30 MB (474274344) - PerHostPeakMemUsage: 452.30 MB (474275163) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 255.651us - TotalStorageWaitTime: 33s254ms - TotalThreadsInvoluntaryContextSwitches: 340.91K (340908) - TotalThreadsTotalWallClockTime: 17m47s - TotalThreadsSysTime: 36s741ms - TotalThreadsUserTime: 14m24s - TotalThreadsVoluntaryContextSwitches: 70.18K (70181) Fragment Instance Lifecycle Timings: - ExecTime: 471.362us - ExecTreeExecTime: 4.118us - OpenTime: 43s157ms - ExecTreeOpenTime: 43s044ms - PrepareTime: 26.644ms - ExecTreePrepareTime: 2.151ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 52.733us, non-child: 52.733us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 663.41 KB/sec - OverallThroughput: 312.24 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.426us - TransmitDataRPCTime: 26.708us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 135.141ms, non-child: 135.141ms, % non-child: 100.00%) - CodegenTime: 1.259ms - CompileTime: 24.562ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 87.944ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 22.090ms AGGREGATION_NODE (id=9):(Total: 43s046ms, non-child: 8s656ms, % non-child: 20.11%) - BuildTime: 8s637ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.45 MB (26682543) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 34s390ms, non-child: 33.754ms, % non-child: 0.10%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 161.555us, non-child: 102.625us, % non-child: 63.52%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 58.930us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 70.433us, non-child: 61.664us, % non-child: 87.55%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 8.768us, non-child: 8.768us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 11.023us, non-child: 11.023us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 34s356ms, non-child: 34s356ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.22 - AverageScannerThreadConcurrency: 23.82 - BytesRead: 3.75 GB (4021993277) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.69 GB (3967470397) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.69 GB (3967470397) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 24 (24) - NumRowGroups: 714 (714) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 448.89 MB (470693401) - PerReadThreadRawHdfsThroughput: 427.36 MB/sec - RemoteScanRanges: 532 (532) - RowBatchQueueGetWaitTime: 34s088ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1155002415) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 714 (714) - ScannerThreadsInvoluntaryContextSwitches: 339.17K (339169) - ScannerThreadsTotalWallClockTime: 17m4s - MaterializeTupleTime(*): 16m27s - ScannerThreadsSysTime: 29s392ms - ScannerThreadsUserTime: 14m23s - ScannerThreadsVoluntaryContextSwitches: 65.10K (65097) - TotalRawHdfsReadTime(*): 8s987ms - TotalReadThroughput: 89.04 MB/sec Fragment F00: Instance 8d4071b24aee8531:393af00400000013 (host=foo20.boo:22000):(Total: 46s739ms, non-child: 4.527ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 2:25/3.12 GB 20:26/3.23 GB 18:31/3.78 GB 13:33/4.06 GB 7:35/4.36 GB 16:23/2.85 GB 11:27/3.33 GB 10:35/4.34 GB 6:28/3.45 GB 23:30/3.66 GB 12:33/4.09 GB 1:27/3.36 GB 22:30/3.69 GB 5:35/4.33 GB 17:28/3.50 GB 0:31/3.76 GB 19:25/3.08 GB 4:31/3.88 GB 21:27/3.36 GB 8:27/3.27 GB 3:27/3.34 GB 14:28/3.48 GB 9:37/4.54 GB 15:35/4.28 GB MemoryUsage(1s000ms): 265.13 MB, 335.80 MB, 323.88 MB, 341.92 MB, 333.22 MB, 356.62 MB, 376.32 MB, 351.83 MB, 366.43 MB, 381.54 MB, 368.38 MB, 343.13 MB, 379.08 MB, 354.53 MB, 406.93 MB, 380.74 MB, 380.26 MB, 368.36 MB, 386.05 MB, 374.34 MB, 392.50 MB, 367.76 MB, 361.43 MB, 399.56 MB, 399.18 MB, 375.83 MB, 404.11 MB, 364.01 MB, 382.60 MB, 378.84 MB, 378.35 MB, 352.16 MB, 364.95 MB, 364.01 MB, 355.28 MB, 402.18 MB, 366.25 MB, 370.44 MB, 393.84 MB, 353.84 MB, 365.42 MB, 370.17 MB, 370.67 MB, 365.69 MB, 399.05 MB, 335.63 MB ThreadUsage(1s000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 23 - AverageThreadTokens: 24.84 - BloomFilterBytes: 0 - PeakMemoryUsage: 447.83 MB (469584367) - PerHostPeakMemUsage: 447.83 MB (469584367) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 329.384us - TotalStorageWaitTime: 32s806ms - TotalThreadsInvoluntaryContextSwitches: 333.56K (333563) - TotalThreadsTotalWallClockTime: 19m15s - TotalThreadsSysTime: 37s006ms - TotalThreadsUserTime: 15m8s - TotalThreadsVoluntaryContextSwitches: 66.81K (66813) Fragment Instance Lifecycle Timings: - ExecTime: 559.611us - ExecTreeExecTime: 4.709us - OpenTime: 46s705ms - ExecTreeOpenTime: 46s565ms - PrepareTime: 33.658ms - ExecTreePrepareTime: 3.391ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 115.124us, non-child: 115.124us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 224.97 KB/sec - OverallThroughput: 135.72 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 24.773us - TransmitDataRPCTime: 69.454us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 166.529ms, non-child: 166.529ms, % non-child: 100.00%) - CodegenTime: 1.534ms - CompileTime: 30.327ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 108.496ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 26.997ms AGGREGATION_NODE (id=9):(Total: 46s568ms, non-child: 9s068ms, % non-child: 19.47%) ExecOption: Codegen Enabled - BuildTime: 9s047ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.64 MB (26885698) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 37s500ms, non-child: 19.322ms, % non-child: 0.05%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 215.682us, non-child: 139.464us, % non-child: 64.66%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 76.218us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 96.083us, non-child: 79.770us, % non-child: 83.02%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 16.312us, non-child: 16.312us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 12.807us, non-child: 12.807us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 37s480ms, non-child: 37s480ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:25/3.12 GB 20:26/3.23 GB 18:31/3.78 GB 13:33/4.06 GB 7:35/4.36 GB 16:23/2.85 GB 11:27/3.33 GB 10:35/4.34 GB 6:28/3.45 GB 23:30/3.66 GB 12:33/4.09 GB 1:27/3.36 GB 22:30/3.69 GB 5:35/4.33 GB 17:28/3.50 GB 0:31/3.76 GB 19:25/3.08 GB 4:31/3.88 GB 21:27/3.36 GB 8:27/3.27 GB 3:27/3.34 GB 14:28/3.48 GB 9:37/4.54 GB 15:35/4.28 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 714 out of 714 Hdfs Read Thread Concurrency Bucket: 0:84.95% 1:13.98% 2:1.075% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2856 BytesRead(1s000ms): 97.13 MB, 169.47 MB, 251.96 MB, 356.09 MB, 455.68 MB, 518.47 MB, 621.54 MB, 716.58 MB, 795.46 MB, 887.72 MB, 984.26 MB, 1.03 GB, 1.13 GB, 1.21 GB, 1.29 GB, 1.37 GB, 1.46 GB, 1.50 GB, 1.58 GB, 1.66 GB, 1.74 GB, 1.80 GB, 1.86 GB, 1.95 GB, 2.03 GB, 2.12 GB, 2.20 GB, 2.28 GB, 2.36 GB, 2.45 GB, 2.53 GB, 2.61 GB, 2.68 GB, 2.74 GB, 2.82 GB, 2.89 GB, 2.99 GB, 3.07 GB, 3.15 GB, 3.24 GB, 3.33 GB, 3.41 GB, 3.49 GB, 3.57 GB, 3.66 GB, 3.73 GB - FooterProcessingTime: (Avg: 14.069ms ; Min: 1.296ms ; Max: 148.213ms ; Number of samples: 714) - AverageHdfsReadThreadConcurrency: 0.16 - AverageScannerThreadConcurrency: 23.84 - BytesRead: 3.75 GB (4021597398) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.69 GB (3966403798) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.69 GB (3966403798) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 714 (714) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 445.53 MB (467176276) - PerReadThreadRawHdfsThroughput: 443.30 MB/sec - RemoteScanRanges: 539 (539) - RowBatchQueueGetWaitTime: 37s222ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.15B (1154891043) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 714 (714) - ScannerThreadsInvoluntaryContextSwitches: 331.90K (331898) - ScannerThreadsTotalWallClockTime: 18m28s - MaterializeTupleTime(*): 17m50s - ScannerThreadsSysTime: 29s768ms - ScannerThreadsUserTime: 15m8s - ScannerThreadsVoluntaryContextSwitches: 61.69K (61685) - TotalRawHdfsReadTime(*): 8s651ms - TotalReadThroughput: 82.48 MB/sec Instance 8d4071b24aee8531:393af00400000003 (host=foo01.boo:22000):(Total: 44s517ms, non-child: 3.288ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 4:36/4.46 GB 13:27/3.33 GB 11:28/3.48 GB 19:29/3.52 GB 2:25/3.10 GB 1:24/2.93 GB 18:28/3.48 GB 12:29/3.54 GB 9:28/3.46 GB 16:30/3.72 GB 15:31/3.81 GB 7:26/3.20 GB 21:30/3.64 GB 10:32/3.96 GB 3:32/3.92 GB 20:36/4.44 GB 14:30/3.71 GB 6:26/3.25 GB 23:31/3.82 GB 8:32/3.98 GB 22:31/3.83 GB 5:30/3.73 GB 0:29/3.58 GB 17:34/4.21 GB MemoryUsage(1s000ms): 144.71 MB, 324.09 MB, 344.62 MB, 339.94 MB, 330.23 MB, 344.56 MB, 370.30 MB, 346.15 MB, 364.89 MB, 349.11 MB, 361.83 MB, 356.63 MB, 371.37 MB, 375.82 MB, 368.24 MB, 358.28 MB, 370.89 MB, 350.63 MB, 373.26 MB, 351.18 MB, 336.57 MB, 356.82 MB, 352.18 MB, 349.22 MB, 368.50 MB, 335.68 MB, 342.61 MB, 378.39 MB, 356.73 MB, 385.60 MB, 371.45 MB, 373.37 MB, 372.63 MB, 353.62 MB, 347.76 MB, 374.64 MB, 352.87 MB, 358.63 MB, 364.70 MB, 356.51 MB, 369.47 MB, 358.20 MB, 370.41 MB, 336.53 MB ThreadUsage(1s000ms): 13, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 21 - AverageThreadTokens: 24.51 - BloomFilterBytes: 0 - PeakMemoryUsage: 449.93 MB (471780815) - PerHostPeakMemUsage: 449.94 MB (471797199) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 169.855us - TotalStorageWaitTime: 35s068ms - TotalThreadsInvoluntaryContextSwitches: 399.46K (399461) - TotalThreadsTotalWallClockTime: 18m18s - TotalThreadsSysTime: 38s796ms - TotalThreadsUserTime: 14m38s - TotalThreadsVoluntaryContextSwitches: 69.93K (69934) Fragment Instance Lifecycle Timings: - ExecTime: 389.621us - ExecTreeExecTime: 4.236us - OpenTime: 44s489ms - ExecTreeOpenTime: 44s368ms - PrepareTime: 27.381ms - ExecTreePrepareTime: 2.440ms DataStreamSender (dst_id=10):(Total: 57.856us, non-child: 57.856us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 486.55 KB/sec - OverallThroughput: 270.07 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 8.788us - TransmitDataRPCTime: 32.113us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 142.968ms, non-child: 142.968ms, % non-child: 100.00%) - CodegenTime: 1.164ms - CompileTime: 25.765ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 94.074ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 22.585ms AGGREGATION_NODE (id=9):(Total: 44s371ms, non-child: 8s648ms, % non-child: 19.49%) ExecOption: Codegen Enabled - BuildTime: 8s629ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.12 MB (26345068) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 35s723ms, non-child: 61.643ms, % non-child: 0.17%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 166.365us, non-child: 109.016us, % non-child: 65.53%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 57.349us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 68.805us, non-child: 60.527us, % non-child: 87.97%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 8.278us, non-child: 8.278us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 9.490us, non-child: 9.490us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 35s661ms, non-child: 35s661ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:36/4.46 GB 13:27/3.33 GB 11:28/3.48 GB 19:29/3.52 GB 2:25/3.10 GB 1:24/2.93 GB 18:28/3.48 GB 12:29/3.54 GB 9:28/3.46 GB 16:30/3.72 GB 15:31/3.81 GB 7:26/3.20 GB 21:30/3.64 GB 10:32/3.96 GB 3:32/3.92 GB 20:36/4.44 GB 14:30/3.71 GB 6:26/3.25 GB 23:31/3.82 GB 8:32/3.98 GB 22:31/3.83 GB 5:30/3.73 GB 0:29/3.58 GB 17:34/4.21 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 714 out of 714 Hdfs Read Thread Concurrency Bucket: 0:80.68% 1:15.91% 2:3.409% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2856 BytesRead(1s000ms): 58.34 MB, 153.52 MB, 242.04 MB, 345.45 MB, 430.16 MB, 506.91 MB, 608.48 MB, 702.10 MB, 783.60 MB, 877.58 MB, 969.95 MB, 1.02 GB, 1.11 GB, 1.20 GB, 1.28 GB, 1.36 GB, 1.46 GB, 1.53 GB, 1.62 GB, 1.70 GB, 1.78 GB, 1.87 GB, 1.95 GB, 2.03 GB, 2.12 GB, 2.20 GB, 2.29 GB, 2.38 GB, 2.46 GB, 2.54 GB, 2.63 GB, 2.71 GB, 2.79 GB, 2.89 GB, 2.97 GB, 3.05 GB, 3.15 GB, 3.22 GB, 3.30 GB, 3.41 GB, 3.47 GB, 3.56 GB, 3.66 GB, 3.71 GB - FooterProcessingTime: (Avg: 15.199ms ; Min: 909.381us ; Max: 123.304ms ; Number of samples: 714) - AverageHdfsReadThreadConcurrency: 0.23 - AverageScannerThreadConcurrency: 23.77 - BytesRead: 3.74 GB (4019366562) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.69 GB (3965196962) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.69 GB (3965196962) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 714 (714) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 447.98 MB (469745492) - PerReadThreadRawHdfsThroughput: 415.38 MB/sec - RemoteScanRanges: 529 (529) - RowBatchQueueGetWaitTime: 35s389ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.15B (1154257680) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 714 (714) - ScannerThreadsInvoluntaryContextSwitches: 397.99K (397988) - ScannerThreadsTotalWallClockTime: 17m33s - MaterializeTupleTime(*): 16m54s - ScannerThreadsSysTime: 31s333ms - ScannerThreadsUserTime: 14m37s - ScannerThreadsVoluntaryContextSwitches: 64.85K (64854) - TotalRawHdfsReadTime(*): 9s228ms - TotalReadThroughput: 86.14 MB/sec Instance 8d4071b24aee8531:393af00400000004 (host=foo13.boo:22000):(Total: 43s975ms, non-child: 3.199ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 9:30/3.71 GB 12:33/4.05 GB 18:33/4.05 GB 15:32/3.93 GB 10:32/3.94 GB 8:31/3.88 GB 4:32/3.92 GB 13:30/3.71 GB 21:32/4.00 GB 11:28/3.48 GB 5:36/4.43 GB 22:30/3.72 GB 14:30/3.71 GB 3:29/3.59 GB 20:22/2.75 GB 16:31/3.84 GB 0:28/3.46 GB 17:29/3.58 GB 19:33/4.11 GB 2:23/2.81 GB 7:31/3.84 GB 23:30/3.69 GB 6:21/2.61 GB 1:28/3.39 GB MemoryUsage(1s000ms): 375.81 MB, 332.27 MB, 322.68 MB, 355.74 MB, 375.55 MB, 363.74 MB, 374.81 MB, 371.66 MB, 375.21 MB, 399.85 MB, 371.77 MB, 379.80 MB, 355.58 MB, 383.05 MB, 369.40 MB, 370.21 MB, 375.16 MB, 370.75 MB, 335.73 MB, 374.69 MB, 375.11 MB, 367.01 MB, 369.59 MB, 356.22 MB, 362.23 MB, 349.21 MB, 361.47 MB, 366.02 MB, 369.61 MB, 377.92 MB, 396.98 MB, 365.04 MB, 376.67 MB, 341.91 MB, 349.53 MB, 372.12 MB, 358.68 MB, 349.19 MB, 367.07 MB, 368.59 MB, 367.14 MB, 387.25 MB, 352.08 MB ThreadUsage(1s000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 24 - AverageThreadTokens: 24.91 - BloomFilterBytes: 0 - PeakMemoryUsage: 452.09 MB (474045861) - PerHostPeakMemUsage: 452.09 MB (474045861) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 236.277us - TotalStorageWaitTime: 33s099ms - TotalThreadsInvoluntaryContextSwitches: 397.24K (397238) - TotalThreadsTotalWallClockTime: 18m8s - TotalThreadsSysTime: 38s289ms - TotalThreadsUserTime: 14m33s - TotalThreadsVoluntaryContextSwitches: 69.63K (69628) Fragment Instance Lifecycle Timings: - ExecTime: 376.143us - ExecTreeExecTime: 3.577us - OpenTime: 43s949ms - ExecTreeOpenTime: 43s839ms - PrepareTime: 25.797ms - ExecTreePrepareTime: 2.034ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 54.174us, non-child: 54.174us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 499.14 KB/sec - OverallThroughput: 288.42 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 7.273us - TransmitDataRPCTime: 31.304us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 130.914ms, non-child: 130.914ms, % non-child: 100.00%) - CodegenTime: 1.136ms - CompileTime: 23.372ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 85.518ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 21.503ms AGGREGATION_NODE (id=9):(Total: 43s841ms, non-child: 8s839ms, % non-child: 20.16%) ExecOption: Codegen Enabled - BuildTime: 8s822ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.12 MB (26345068) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 35s002ms, non-child: 31.944ms, % non-child: 0.09%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 170.825us, non-child: 106.852us, % non-child: 62.55%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 63.973us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 76.905us, non-child: 68.456us, % non-child: 89.01%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 8.449us, non-child: 8.449us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 14.380us, non-child: 14.380us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 34s970ms, non-child: 34s970ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 9:30/3.71 GB 12:33/4.05 GB 18:33/4.05 GB 15:32/3.93 GB 10:32/3.94 GB 8:31/3.88 GB 4:32/3.92 GB 13:30/3.71 GB 21:32/4.00 GB 11:28/3.48 GB 5:36/4.43 GB 22:30/3.72 GB 14:30/3.71 GB 3:29/3.59 GB 20:22/2.75 GB 16:31/3.84 GB 0:28/3.46 GB 17:29/3.58 GB 19:33/4.11 GB 2:23/2.81 GB 7:31/3.84 GB 23:30/3.69 GB 6:21/2.61 GB 1:28/3.39 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 714 out of 714 Hdfs Read Thread Concurrency Bucket: 0:80.46% 1:16.09% 2:3.448% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2856 BytesRead(1s000ms): 116.48 MB, 203.55 MB, 278.81 MB, 379.18 MB, 468.88 MB, 566.63 MB, 637.31 MB, 737.77 MB, 827.32 MB, 906.21 MB, 993.20 MB, 1.07 GB, 1.14 GB, 1.23 GB, 1.33 GB, 1.40 GB, 1.49 GB, 1.59 GB, 1.66 GB, 1.75 GB, 1.84 GB, 1.92 GB, 2.00 GB, 2.09 GB, 2.18 GB, 2.26 GB, 2.35 GB, 2.44 GB, 2.52 GB, 2.61 GB, 2.70 GB, 2.77 GB, 2.86 GB, 2.95 GB, 3.03 GB, 3.11 GB, 3.21 GB, 3.28 GB, 3.38 GB, 3.47 GB, 3.54 GB, 3.64 GB, 3.72 GB - FooterProcessingTime: (Avg: 14.814ms ; Min: 1.004ms ; Max: 93.986ms ; Number of samples: 714) - AverageHdfsReadThreadConcurrency: 0.23 - AverageScannerThreadConcurrency: 23.91 - BytesRead: 3.75 GB (4023848430) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.70 GB (3967528430) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.70 GB (3967528430) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 714 (714) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 444.73 MB (466331518) - PerReadThreadRawHdfsThroughput: 391.97 MB/sec - RemoteScanRanges: 550 (550) - RowBatchQueueGetWaitTime: 34s704ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1155570781) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 714 (714) - ScannerThreadsInvoluntaryContextSwitches: 395.43K (395433) - ScannerThreadsTotalWallClockTime: 17m24s - MaterializeTupleTime(*): 16m47s - ScannerThreadsSysTime: 30s732ms - ScannerThreadsUserTime: 14m33s - ScannerThreadsVoluntaryContextSwitches: 64.56K (64562) - TotalRawHdfsReadTime(*): 9s790ms - TotalReadThroughput: 88.21 MB/sec Instance 8d4071b24aee8531:393af00400000006 (host=foo15.boo:22000):(Total: 43s732ms, non-child: 4.306ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 15:17/2.11 GB 9:23/2.83 GB 21:31/3.86 GB 18:29/3.59 GB 1:27/3.38 GB 2:29/3.56 GB 8:26/3.17 GB 5:24/2.96 GB 22:35/4.35 GB 16:22/2.72 GB 7:34/4.21 GB 4:27/3.38 GB 12:29/3.59 GB 13:26/3.23 GB 23:31/3.86 GB 6:33/4.10 GB 10:35/4.33 GB 19:33/4.10 GB 20:40/4.95 GB 3:26/3.21 GB 11:27/3.32 GB 14:37/4.52 GB 17:33/4.08 GB 0:40/4.89 GB MemoryUsage(1s000ms): 316.55 MB, 309.46 MB, 350.39 MB, 340.23 MB, 366.19 MB, 348.55 MB, 382.07 MB, 365.60 MB, 349.62 MB, 365.89 MB, 387.70 MB, 367.30 MB, 399.28 MB, 352.44 MB, 365.26 MB, 373.90 MB, 386.52 MB, 379.41 MB, 362.02 MB, 402.35 MB, 396.69 MB, 391.71 MB, 368.30 MB, 374.95 MB, 345.91 MB, 390.21 MB, 346.03 MB, 367.04 MB, 375.84 MB, 374.65 MB, 372.85 MB, 387.84 MB, 355.57 MB, 388.32 MB, 387.95 MB, 363.00 MB, 386.47 MB, 368.34 MB, 371.25 MB, 370.15 MB, 348.32 MB, 363.02 MB, 369.54 MB ThreadUsage(1s000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25 - AverageThreadTokens: 24.89 - BloomFilterBytes: 0 - PeakMemoryUsage: 455.57 MB (477697445) - PerHostPeakMemUsage: 455.57 MB (477697445) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 542.541us - TotalStorageWaitTime: 32s450ms - TotalThreadsInvoluntaryContextSwitches: 387.67K (387669) - TotalThreadsTotalWallClockTime: 18m - TotalThreadsSysTime: 37s133ms - TotalThreadsUserTime: 14m26s - TotalThreadsVoluntaryContextSwitches: 69.42K (69416) Fragment Instance Lifecycle Timings: - ExecTime: 1.181ms - ExecTreeExecTime: 4.355us - OpenTime: 43s703ms - ExecTreeOpenTime: 43s585ms - PrepareTime: 26.987ms - ExecTreePrepareTime: 2.331ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 42.613us, non-child: 42.613us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 776.36 KB/sec - OverallThroughput: 366.67 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.491us - TransmitDataRPCTime: 20.126us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 139.717ms, non-child: 139.717ms, % non-child: 100.00%) - CodegenTime: 1.446ms - CompileTime: 25.752ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 91.148ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 22.273ms AGGREGATION_NODE (id=9):(Total: 43s588ms, non-child: 8s519ms, % non-child: 19.55%) ExecOption: Codegen Enabled - BuildTime: 8s501ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.64 MB (26885698) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 35s068ms, non-child: 39.454ms, % non-child: 0.11%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 151.075us, non-child: 95.462us, % non-child: 63.19%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 55.613us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 64.289us, non-child: 57.842us, % non-child: 89.97%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 6.447us, non-child: 6.447us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 12.046us, non-child: 12.046us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 35s028ms, non-child: 35s028ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 15:17/2.11 GB 9:23/2.83 GB 21:31/3.86 GB 18:29/3.59 GB 1:27/3.38 GB 2:29/3.56 GB 8:26/3.17 GB 5:24/2.96 GB 22:35/4.35 GB 16:22/2.72 GB 7:34/4.21 GB 4:27/3.38 GB 12:29/3.59 GB 13:26/3.23 GB 23:31/3.86 GB 6:33/4.10 GB 10:35/4.33 GB 19:33/4.10 GB 20:40/4.95 GB 3:26/3.21 GB 11:27/3.32 GB 14:37/4.52 GB 17:33/4.08 GB 0:40/4.89 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 714 out of 714 Hdfs Read Thread Concurrency Bucket: 0:78.16% 1:18.39% 2:3.448% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2856 BytesRead(1s000ms): 119.87 MB, 171.16 MB, 250.61 MB, 354.55 MB, 461.02 MB, 526.17 MB, 626.72 MB, 724.42 MB, 804.81 MB, 888.69 MB, 988.04 MB, 1.04 GB, 1.12 GB, 1.23 GB, 1.31 GB, 1.39 GB, 1.49 GB, 1.57 GB, 1.65 GB, 1.75 GB, 1.82 GB, 1.90 GB, 2.00 GB, 2.08 GB, 2.16 GB, 2.26 GB, 2.34 GB, 2.42 GB, 2.52 GB, 2.61 GB, 2.68 GB, 2.78 GB, 2.86 GB, 2.94 GB, 3.04 GB, 3.12 GB, 3.20 GB, 3.29 GB, 3.38 GB, 3.46 GB, 3.55 GB, 3.64 GB, 3.72 GB - FooterProcessingTime: (Avg: 14.348ms ; Min: 861.012us ; Max: 77.485ms ; Number of samples: 714) - AverageHdfsReadThreadConcurrency: 0.25 - AverageScannerThreadConcurrency: 23.89 - BytesRead: 3.75 GB (4027021647) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.70 GB (3972647247) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.70 GB (3972647247) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 714 (714) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 451.24 MB (473160616) - PerReadThreadRawHdfsThroughput: 437.83 MB/sec - RemoteScanRanges: 531 (531) - RowBatchQueueGetWaitTime: 34s748ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1156518429) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 714 (714) - ScannerThreadsInvoluntaryContextSwitches: 386.02K (386020) - ScannerThreadsTotalWallClockTime: 17m16s - MaterializeTupleTime(*): 16m40s - ScannerThreadsSysTime: 29s869ms - ScannerThreadsUserTime: 14m25s - ScannerThreadsVoluntaryContextSwitches: 64.36K (64360) - TotalRawHdfsReadTime(*): 8s771ms - TotalReadThroughput: 88.29 MB/sec Instance 8d4071b24aee8531:393af00400000007 (host=foo09.boo:22000):(Total: 43s550ms, non-child: 3.213ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 19:30/3.70 GB 6:30/3.69 GB 1:26/3.23 GB 9:25/3.08 GB 0:30/3.73 GB 20:24/2.96 GB 11:31/3.86 GB 17:24/2.94 GB 10:36/4.44 GB 12:43/5.32 GB 3:25/3.05 GB 21:39/4.75 GB 4:26/3.23 GB 22:23/2.81 GB 5:38/4.67 GB 8:30/3.71 GB 23:30/3.73 GB 14:29/3.59 GB 16:36/4.42 GB 13:23/2.88 GB 18:24/2.97 GB 2:41/5.08 GB 7:29/3.57 GB 15:22/2.73 GB MemoryUsage(1s000ms): 176.37 MB, 341.38 MB, 378.64 MB, 331.38 MB, 354.08 MB, 338.50 MB, 393.52 MB, 373.15 MB, 386.45 MB, 346.95 MB, 376.77 MB, 412.48 MB, 370.67 MB, 386.22 MB, 363.74 MB, 377.19 MB, 390.18 MB, 325.71 MB, 368.21 MB, 379.60 MB, 339.35 MB, 367.87 MB, 363.44 MB, 379.42 MB, 372.25 MB, 391.53 MB, 366.22 MB, 351.31 MB, 380.39 MB, 360.43 MB, 378.85 MB, 384.92 MB, 363.90 MB, 390.35 MB, 359.68 MB, 383.21 MB, 372.52 MB, 365.75 MB, 368.75 MB, 379.92 MB, 369.82 MB, 372.67 MB, 353.84 MB, 131.86 MB ThreadUsage(1s000ms): 13, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 9 - AverageThreadTokens: 24.38 - BloomFilterBytes: 0 - PeakMemoryUsage: 450.62 MB (472512755) - PerHostPeakMemUsage: 450.62 MB (472512755) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 217.400us - TotalStorageWaitTime: 32s808ms - TotalThreadsInvoluntaryContextSwitches: 427.62K (427618) - TotalThreadsTotalWallClockTime: 17m55s - TotalThreadsSysTime: 37s312ms - TotalThreadsUserTime: 14m22s - TotalThreadsVoluntaryContextSwitches: 69.83K (69826) Fragment Instance Lifecycle Timings: - ExecTime: 361.033us - ExecTreeExecTime: 3.652us - OpenTime: 43s524ms - ExecTreeOpenTime: 43s414ms - PrepareTime: 25.404ms - ExecTreePrepareTime: 1.944ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 41.824us, non-child: 41.824us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 834.13 KB/sec - OverallThroughput: 373.59 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 8.103us - TransmitDataRPCTime: 18.732us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 131.168ms, non-child: 131.168ms, % non-child: 100.00%) - CodegenTime: 1.181ms - CompileTime: 24.299ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 85.211ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 21.130ms AGGREGATION_NODE (id=9):(Total: 43s416ms, non-child: 8s791ms, % non-child: 20.25%) ExecOption: Codegen Enabled - BuildTime: 8s772ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 24.74 MB (25939564) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 34s624ms, non-child: 40.559ms, % non-child: 0.12%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 141.096us, non-child: 89.562us, % non-child: 63.48%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 51.534us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 63.770us, non-child: 55.440us, % non-child: 86.94%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 8.330us, non-child: 8.330us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 9.150us, non-child: 9.150us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 34s583ms, non-child: 34s583ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 19:30/3.70 GB 6:30/3.69 GB 1:26/3.23 GB 9:25/3.08 GB 0:30/3.73 GB 20:24/2.96 GB 11:31/3.86 GB 17:24/2.94 GB 10:36/4.44 GB 12:43/5.32 GB 3:25/3.05 GB 21:39/4.75 GB 4:26/3.23 GB 22:23/2.81 GB 5:38/4.67 GB 8:30/3.71 GB 23:30/3.73 GB 14:29/3.59 GB 16:36/4.42 GB 13:23/2.88 GB 18:24/2.97 GB 2:41/5.08 GB 7:29/3.57 GB 15:22/2.73 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 714 out of 714 Hdfs Read Thread Concurrency Bucket: 0:79.31% 1:18.39% 2:2.299% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2856 BytesRead(1s000ms): 58.57 MB, 152.77 MB, 237.97 MB, 340.36 MB, 420.75 MB, 513.14 MB, 606.72 MB, 701.80 MB, 779.32 MB, 871.39 MB, 976.23 MB, 1.02 GB, 1.11 GB, 1.21 GB, 1.29 GB, 1.37 GB, 1.48 GB, 1.54 GB, 1.63 GB, 1.73 GB, 1.80 GB, 1.90 GB, 1.99 GB, 2.07 GB, 2.15 GB, 2.25 GB, 2.32 GB, 2.42 GB, 2.50 GB, 2.59 GB, 2.68 GB, 2.76 GB, 2.86 GB, 2.94 GB, 3.03 GB, 3.11 GB, 3.20 GB, 3.29 GB, 3.37 GB, 3.45 GB, 3.54 GB, 3.63 GB, 3.71 GB, 3.74 GB - FooterProcessingTime: (Avg: 13.784ms ; Min: 2.444ms ; Max: 113.234ms ; Number of samples: 714) - AverageHdfsReadThreadConcurrency: 0.23 - AverageScannerThreadConcurrency: 23.64 - BytesRead: 3.74 GB (4020300373) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.69 GB (3966642773) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.69 GB (3966642773) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 714 (714) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 448.84 MB (470639192) - PerReadThreadRawHdfsThroughput: 434.67 MB/sec - RemoteScanRanges: 524 (524) - RowBatchQueueGetWaitTime: 34s365ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.15B (1154507445) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 714 (714) - ScannerThreadsInvoluntaryContextSwitches: 425.64K (425641) - ScannerThreadsTotalWallClockTime: 17m12s - MaterializeTupleTime(*): 16m35s - ScannerThreadsSysTime: 29s854ms - ScannerThreadsUserTime: 14m22s - ScannerThreadsVoluntaryContextSwitches: 64.75K (64750) - TotalRawHdfsReadTime(*): 8s820ms - TotalReadThroughput: 87.12 MB/sec Instance 8d4071b24aee8531:393af00400000011 (host=foo17.boo:22000):(Total: 43s499ms, non-child: 3.543ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 21:32/3.98 GB 6:37/4.56 GB 18:27/3.31 GB 16:23/2.85 GB 23:33/4.00 GB 7:25/3.10 GB 22:26/3.23 GB 4:30/3.69 GB 12:34/4.23 GB 15:32/3.91 GB 10:29/3.56 GB 5:28/3.41 GB 19:38/4.67 GB 2:26/3.25 GB 13:31/3.82 GB 8:35/4.36 GB 11:30/3.66 GB 20:28/3.46 GB 3:31/3.83 GB 17:31/3.86 GB 0:26/3.19 GB 14:27/3.30 GB 1:32/4.00 GB 9:24/2.93 GB MemoryUsage(1s000ms): 315.97 MB, 355.24 MB, 351.69 MB, 314.59 MB, 339.41 MB, 384.42 MB, 372.73 MB, 392.60 MB, 393.21 MB, 358.85 MB, 355.36 MB, 357.09 MB, 379.58 MB, 391.36 MB, 379.38 MB, 355.15 MB, 370.73 MB, 361.35 MB, 374.51 MB, 356.46 MB, 340.12 MB, 381.61 MB, 381.24 MB, 371.64 MB, 369.92 MB, 373.29 MB, 369.22 MB, 382.84 MB, 368.83 MB, 348.87 MB, 354.94 MB, 380.52 MB, 371.83 MB, 373.74 MB, 377.69 MB, 365.20 MB, 358.86 MB, 370.57 MB, 390.16 MB, 357.57 MB, 417.92 MB, 362.98 MB, 297.07 MB ThreadUsage(1s000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 20 - AverageThreadTokens: 24.69 - BloomFilterBytes: 0 - PeakMemoryUsage: 447.91 MB (469662973) - PerHostPeakMemUsage: 447.91 MB (469662973) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 531.101us - TotalStorageWaitTime: 34s306ms - TotalThreadsInvoluntaryContextSwitches: 328.92K (328921) - TotalThreadsTotalWallClockTime: 17m54s - TotalThreadsSysTime: 35s679ms - TotalThreadsUserTime: 14m32s - TotalThreadsVoluntaryContextSwitches: 68.61K (68607) Fragment Instance Lifecycle Timings: - ExecTime: 666.169us - ExecTreeExecTime: 4.445us - OpenTime: 43s472ms - ExecTreeOpenTime: 43s360ms - PrepareTime: 26.082ms - ExecTreePrepareTime: 2.089ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 45.238us, non-child: 45.238us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 773.25 KB/sec - OverallThroughput: 345.39 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 8.368us - TransmitDataRPCTime: 20.207us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 132.385ms, non-child: 132.385ms, % non-child: 100.00%) - CodegenTime: 1.210ms - CompileTime: 24.028ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 86.137ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 21.700ms AGGREGATION_NODE (id=9):(Total: 43s363ms, non-child: 8s598ms, % non-child: 19.83%) ExecOption: Codegen Enabled - BuildTime: 8s581ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.64 MB (26885740) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 34s764ms, non-child: 26.910ms, % non-child: 0.08%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 200.107us, non-child: 127.759us, % non-child: 63.85%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 72.348us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 81.144us, non-child: 75.321us, % non-child: 92.82%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 5.823us, non-child: 5.823us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 14.146us, non-child: 14.146us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 34s737ms, non-child: 34s737ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 21:32/3.98 GB 6:37/4.56 GB 18:27/3.31 GB 16:23/2.85 GB 23:33/4.00 GB 7:25/3.10 GB 22:26/3.23 GB 4:30/3.69 GB 12:34/4.23 GB 15:32/3.91 GB 10:29/3.56 GB 5:28/3.41 GB 19:38/4.67 GB 2:26/3.25 GB 13:31/3.82 GB 8:35/4.36 GB 11:30/3.66 GB 20:28/3.46 GB 3:31/3.83 GB 17:31/3.86 GB 0:26/3.19 GB 14:27/3.30 GB 1:32/4.00 GB 9:24/2.93 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 715 out of 715 Hdfs Read Thread Concurrency Bucket: 0:89.66% 1:9.195% 2:1.149% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2860 BytesRead(1s000ms): 116.02 MB, 184.96 MB, 269.04 MB, 360.42 MB, 461.45 MB, 549.64 MB, 632.69 MB, 734.37 MB, 818.59 MB, 901.25 MB, 997.88 MB, 1.07 GB, 1.15 GB, 1.24 GB, 1.33 GB, 1.40 GB, 1.50 GB, 1.59 GB, 1.67 GB, 1.76 GB, 1.84 GB, 1.93 GB, 2.01 GB, 2.10 GB, 2.19 GB, 2.26 GB, 2.36 GB, 2.45 GB, 2.53 GB, 2.62 GB, 2.70 GB, 2.80 GB, 2.89 GB, 2.96 GB, 3.06 GB, 3.14 GB, 3.23 GB, 3.32 GB, 3.41 GB, 3.49 GB, 3.58 GB, 3.67 GB, 3.74 GB - FooterProcessingTime: (Avg: 14.472ms ; Min: 2.510ms ; Max: 70.837ms ; Number of samples: 715) - AverageHdfsReadThreadConcurrency: 0.11 - AverageScannerThreadConcurrency: 23.69 - BytesRead: 3.75 GB (4023134395) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.69 GB (3967428795) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.69 GB (3967428795) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 715 (715) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 445.93 MB (467594882) - PerReadThreadRawHdfsThroughput: 433.43 MB/sec - RemoteScanRanges: 544 (544) - RowBatchQueueGetWaitTime: 34s498ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1155300728) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 715 (715) - ScannerThreadsInvoluntaryContextSwitches: 327.26K (327264) - ScannerThreadsTotalWallClockTime: 17m11s - MaterializeTupleTime(*): 16m33s - ScannerThreadsSysTime: 28s418ms - ScannerThreadsUserTime: 14m32s - ScannerThreadsVoluntaryContextSwitches: 63.51K (63510) - TotalRawHdfsReadTime(*): 8s852ms - TotalReadThroughput: 88.20 MB/sec Instance 8d4071b24aee8531:393af0040000000a (host=foo21.boo:22000):(Total: 43s315ms, non-child: 3.533ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 18:30/3.73 GB 6:32/3.95 GB 22:33/4.06 GB 10:31/3.76 GB 20:32/3.96 GB 3:19/2.36 GB 14:31/3.75 GB 5:29/3.59 GB 7:36/4.42 GB 13:33/4.07 GB 15:31/3.83 GB 9:27/3.31 GB 4:30/3.68 GB 21:27/3.35 GB 8:32/3.91 GB 11:40/4.90 GB 12:35/4.29 GB 1:32/3.89 GB 2:34/4.21 GB 19:38/4.69 GB 16:29/3.56 GB 17:26/3.21 GB 0:29/3.62 GB MemoryUsage(1s000ms): 320.35 MB, 295.91 MB, 324.29 MB, 330.88 MB, 327.96 MB, 348.45 MB, 353.06 MB, 352.67 MB, 376.92 MB, 356.32 MB, 363.31 MB, 372.38 MB, 373.30 MB, 380.37 MB, 373.16 MB, 363.31 MB, 379.30 MB, 346.01 MB, 366.96 MB, 374.57 MB, 348.58 MB, 390.53 MB, 376.06 MB, 362.53 MB, 374.50 MB, 372.07 MB, 365.19 MB, 377.18 MB, 352.93 MB, 377.98 MB, 346.84 MB, 374.25 MB, 351.61 MB, 356.78 MB, 362.29 MB, 370.39 MB, 361.09 MB, 395.74 MB, 392.97 MB, 385.04 MB, 369.84 MB, 358.60 MB, 311.40 MB ThreadUsage(1s000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 20 - AverageThreadTokens: 24.90 - BloomFilterBytes: 0 - PeakMemoryUsage: 435.80 MB (456970703) - PerHostPeakMemUsage: 435.80 MB (456970703) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 364.570us - TotalStorageWaitTime: 34s199ms - TotalThreadsInvoluntaryContextSwitches: 154.62K (154616) - TotalThreadsTotalWallClockTime: 17m51s - TotalThreadsSysTime: 34s885ms - TotalThreadsUserTime: 15m7s - TotalThreadsVoluntaryContextSwitches: 73.81K (73813) Fragment Instance Lifecycle Timings: - ExecTime: 676.024us - ExecTreeExecTime: 4.982us - OpenTime: 43s288ms - ExecTreeOpenTime: 43s178ms - PrepareTime: 26.052ms - ExecTreePrepareTime: 2.254ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 49.966us, non-child: 49.966us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 732.47 KB/sec - OverallThroughput: 312.71 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.944us - TransmitDataRPCTime: 21.332us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 130.539ms, non-child: 130.539ms, % non-child: 100.00%) - CodegenTime: 1.250ms - CompileTime: 23.955ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 84.671ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 21.385ms AGGREGATION_NODE (id=9):(Total: 43s180ms, non-child: 8s997ms, % non-child: 20.84%) ExecOption: Codegen Enabled - BuildTime: 8s980ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.12 MB (26345068) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 34s183ms, non-child: 49.008ms, % non-child: 0.14%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 151.176us, non-child: 97.046us, % non-child: 64.19%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 54.130us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 61.295us, non-child: 54.159us, % non-child: 88.36%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 7.136us, non-child: 7.136us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 12.692us, non-child: 12.692us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 34s133ms, non-child: 34s133ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 18:30/3.73 GB 6:32/3.95 GB 22:33/4.06 GB 10:31/3.76 GB 20:32/3.96 GB 3:19/2.36 GB 14:31/3.75 GB 5:29/3.59 GB 7:36/4.42 GB 13:33/4.07 GB 15:31/3.83 GB 9:27/3.31 GB 4:30/3.68 GB 21:27/3.35 GB 8:32/3.91 GB 11:40/4.90 GB 12:35/4.29 GB 1:32/3.89 GB 2:34/4.21 GB 19:38/4.69 GB 16:29/3.56 GB 17:26/3.21 GB 0:29/3.62 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 716 out of 716 Hdfs Read Thread Concurrency Bucket: 0:80.23% 1:18.6% 2:1.163% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2864 BytesRead(1s000ms): 115.04 MB, 177.68 MB, 270.42 MB, 357.63 MB, 460.32 MB, 555.41 MB, 631.43 MB, 731.99 MB, 821.70 MB, 896.50 MB, 992.79 MB, 1.05 GB, 1.14 GB, 1.23 GB, 1.32 GB, 1.40 GB, 1.49 GB, 1.58 GB, 1.67 GB, 1.75 GB, 1.84 GB, 1.93 GB, 2.01 GB, 2.11 GB, 2.18 GB, 2.27 GB, 2.36 GB, 2.44 GB, 2.53 GB, 2.62 GB, 2.70 GB, 2.79 GB, 2.89 GB, 2.97 GB, 3.06 GB, 3.14 GB, 3.24 GB, 3.32 GB, 3.41 GB, 3.50 GB, 3.58 GB, 3.67 GB, 3.74 GB - FooterProcessingTime: (Avg: 14.147ms ; Min: 1.104ms ; Max: 78.909ms ; Number of samples: 716) - AverageHdfsReadThreadConcurrency: 0.21 - AverageScannerThreadConcurrency: 23.90 - BytesRead: 3.74 GB (4020739371) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.69 GB (3967081771) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.69 GB (3967081771) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 24 (24) - NumRowGroups: 716 (716) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 430.54 MB (451453780) - PerReadThreadRawHdfsThroughput: 416.86 MB/sec - RemoteScanRanges: 524 (524) - RowBatchQueueGetWaitTime: 33s859ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.15B (1154518343) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 716 (716) - ScannerThreadsInvoluntaryContextSwitches: 153.67K (153672) - ScannerThreadsTotalWallClockTime: 17m8s - MaterializeTupleTime(*): 16m30s - ScannerThreadsSysTime: 27s096ms - ScannerThreadsUserTime: 15m6s - ScannerThreadsVoluntaryContextSwitches: 68.65K (68651) - TotalRawHdfsReadTime(*): 9s198ms - TotalReadThroughput: 89.17 MB/sec Instance 8d4071b24aee8531:393af00400000010 (host=foo19.boo:22000):(Total: 43s106ms, non-child: 3.255ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 16:29/3.61 GB 22:29/3.60 GB 17:27/3.33 GB 2:34/4.17 GB 12:29/3.60 GB 6:33/4.11 GB 10:21/2.62 GB 0:31/3.84 GB 15:42/5.15 GB 7:25/3.08 GB 5:43/5.33 GB 14:26/3.19 GB 8:37/4.55 GB 9:38/4.67 GB 4:29/3.57 GB 21:26/3.23 GB 11:36/4.44 GB 1:36/4.41 GB 18:30/3.71 GB 19:35/4.36 GB 13:26/3.18 GB 20:29/3.58 GB 3:23/2.83 GB MemoryUsage(1s000ms): 160.97 MB, 329.73 MB, 317.21 MB, 372.02 MB, 333.51 MB, 327.73 MB, 400.26 MB, 389.27 MB, 388.67 MB, 376.86 MB, 358.73 MB, 385.15 MB, 355.08 MB, 362.11 MB, 369.47 MB, 373.38 MB, 364.84 MB, 355.50 MB, 388.20 MB, 364.25 MB, 373.38 MB, 366.14 MB, 387.21 MB, 380.24 MB, 382.91 MB, 376.48 MB, 375.43 MB, 380.61 MB, 364.72 MB, 378.30 MB, 381.96 MB, 381.04 MB, 355.46 MB, 386.79 MB, 387.30 MB, 390.35 MB, 393.63 MB, 372.44 MB, 385.00 MB, 378.87 MB, 386.49 MB, 395.74 MB, 296.56 MB ThreadUsage(1s000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 20 - AverageThreadTokens: 24.90 - BloomFilterBytes: 0 - PeakMemoryUsage: 452.90 MB (474897861) - PerHostPeakMemUsage: 452.90 MB (474897861) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 172.616us - TotalStorageWaitTime: 32s321ms - TotalThreadsInvoluntaryContextSwitches: 310.09K (310094) - TotalThreadsTotalWallClockTime: 17m45s - TotalThreadsSysTime: 37s517ms - TotalThreadsUserTime: 14m27s - TotalThreadsVoluntaryContextSwitches: 71.94K (71944) Fragment Instance Lifecycle Timings: - ExecTime: 360.616us - ExecTreeExecTime: 3.900us - OpenTime: 43s079ms - ExecTreeOpenTime: 42s968ms - PrepareTime: 26.390ms - ExecTreePrepareTime: 1.974ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 44.923us, non-child: 44.923us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 765.63 KB/sec - OverallThroughput: 347.82 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 8.070us - TransmitDataRPCTime: 20.408us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 132.146ms, non-child: 132.146ms, % non-child: 100.00%) - CodegenTime: 1.607ms - CompileTime: 24.417ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 85.150ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 22.040ms AGGREGATION_NODE (id=9):(Total: 42s970ms, non-child: 8s969ms, % non-child: 20.87%) ExecOption: Codegen Enabled - BuildTime: 8s952ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 24.87 MB (26074732) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 34s001ms, non-child: 47.295ms, % non-child: 0.14%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 163.750us, non-child: 104.407us, % non-child: 63.76%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 59.343us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 68.406us, non-child: 62.758us, % non-child: 91.74%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 5.648us, non-child: 5.648us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 10.205us, non-child: 10.205us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 33s953ms, non-child: 33s953ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 16:29/3.61 GB 22:29/3.60 GB 17:27/3.33 GB 2:34/4.17 GB 12:29/3.60 GB 6:33/4.11 GB 10:21/2.62 GB 0:31/3.84 GB 15:42/5.15 GB 7:25/3.08 GB 5:43/5.33 GB 14:26/3.19 GB 8:37/4.55 GB 9:38/4.67 GB 4:29/3.57 GB 21:26/3.23 GB 11:36/4.44 GB 1:36/4.41 GB 18:30/3.71 GB 19:35/4.36 GB 13:26/3.18 GB 20:29/3.58 GB 3:23/2.83 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 714 out of 714 Hdfs Read Thread Concurrency Bucket: 0:87.21% 1:8.14% 2:3.488% 3:0% 4:0% 5:1.163% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2856 BytesRead(1s000ms): 60.31 MB, 160.39 MB, 240.92 MB, 351.44 MB, 445.54 MB, 524.47 MB, 627.50 MB, 714.43 MB, 806.80 MB, 896.51 MB, 982.19 MB, 1.05 GB, 1.13 GB, 1.23 GB, 1.31 GB, 1.40 GB, 1.49 GB, 1.57 GB, 1.66 GB, 1.75 GB, 1.83 GB, 1.92 GB, 2.01 GB, 2.10 GB, 2.18 GB, 2.27 GB, 2.36 GB, 2.45 GB, 2.54 GB, 2.61 GB, 2.71 GB, 2.80 GB, 2.88 GB, 2.98 GB, 3.06 GB, 3.15 GB, 3.24 GB, 3.32 GB, 3.42 GB, 3.50 GB, 3.59 GB, 3.67 GB, 3.75 GB - FooterProcessingTime: (Avg: 14.302ms ; Min: 720.218us ; Max: 78.327ms ; Number of samples: 714) - AverageHdfsReadThreadConcurrency: 0.21 - AverageScannerThreadConcurrency: 23.90 - BytesRead: 3.75 GB (4022124544) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.69 GB (3965394944) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.69 GB (3965394944) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 24 (24) - NumRowGroups: 714 (714) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 451.72 MB (473666346) - PerReadThreadRawHdfsThroughput: 432.78 MB/sec - RemoteScanRanges: 554 (554) - RowBatchQueueGetWaitTime: 33s720ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1155066031) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 714 (714) - ScannerThreadsInvoluntaryContextSwitches: 308.28K (308284) - ScannerThreadsTotalWallClockTime: 17m2s - MaterializeTupleTime(*): 16m26s - ScannerThreadsSysTime: 29s973ms - ScannerThreadsUserTime: 14m27s - ScannerThreadsVoluntaryContextSwitches: 66.86K (66862) - TotalRawHdfsReadTime(*): 8s863ms - TotalReadThroughput: 89.20 MB/sec Instance 8d4071b24aee8531:393af0040000000c (host=foo10.boo:22000):(Total: 43s106ms, non-child: 3.265ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 1:23/2.83 GB 23:31/3.82 GB 13:28/3.44 GB 17:36/4.47 GB 12:30/3.67 GB 4:23/2.85 GB 2:30/3.75 GB 20:32/4.00 GB 3:34/4.19 GB 22:25/3.11 GB 5:32/3.97 GB 8:35/4.33 GB 0:28/3.48 GB 18:25/3.11 GB 21:38/4.64 GB 15:30/3.71 GB 14:29/3.60 GB 11:35/4.32 GB 7:25/3.06 GB 19:35/4.29 GB 10:30/3.71 GB 6:19/2.33 GB 16:33/4.03 GB 9:28/3.45 GB MemoryUsage(1s000ms): 194.08 MB, 297.22 MB, 336.80 MB, 341.05 MB, 325.48 MB, 342.27 MB, 367.61 MB, 387.26 MB, 345.60 MB, 355.92 MB, 378.74 MB, 364.00 MB, 363.37 MB, 379.39 MB, 364.48 MB, 350.60 MB, 383.02 MB, 363.22 MB, 357.29 MB, 386.31 MB, 363.11 MB, 392.30 MB, 358.24 MB, 391.47 MB, 367.64 MB, 379.12 MB, 379.39 MB, 360.03 MB, 369.75 MB, 348.61 MB, 375.80 MB, 374.46 MB, 360.97 MB, 372.63 MB, 353.16 MB, 387.26 MB, 379.42 MB, 370.74 MB, 359.68 MB, 365.43 MB, 362.81 MB, 344.68 MB, 295.51 MB ThreadUsage(1s000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 19 - AverageThreadTokens: 24.86 - BloomFilterBytes: 0 - PeakMemoryUsage: 459.37 MB (481687107) - PerHostPeakMemUsage: 459.37 MB (481687107) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 225.055us - TotalStorageWaitTime: 32s853ms - TotalThreadsInvoluntaryContextSwitches: 369.88K (369875) - TotalThreadsTotalWallClockTime: 17m44s - TotalThreadsSysTime: 36s686ms - TotalThreadsUserTime: 14m13s - TotalThreadsVoluntaryContextSwitches: 70.78K (70780) Fragment Instance Lifecycle Timings: - ExecTime: 403.481us - ExecTreeExecTime: 4.038us - OpenTime: 43s079ms - ExecTreeOpenTime: 42s967ms - PrepareTime: 26.263ms - ExecTreePrepareTime: 2.095ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 52.950us, non-child: 52.950us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 533.35 KB/sec - OverallThroughput: 295.09 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 7.367us - TransmitDataRPCTime: 29.296us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 132.876ms, non-child: 132.876ms, % non-child: 100.00%) - CodegenTime: 1.228ms - CompileTime: 24.181ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 86.314ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 21.821ms AGGREGATION_NODE (id=9):(Total: 42s970ms, non-child: 8s787ms, % non-child: 20.45%) ExecOption: Codegen Enabled - BuildTime: 8s769ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 26.02 MB (27282432) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 34s182ms, non-child: 27.904ms, % non-child: 0.08%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 152.894us, non-child: 95.950us, % non-child: 62.76%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 56.944us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 72.358us, non-child: 58.986us, % non-child: 81.52%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 13.372us, non-child: 13.372us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 8.649us, non-child: 8.649us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 34s154ms, non-child: 34s154ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:23/2.83 GB 23:31/3.82 GB 13:28/3.44 GB 17:36/4.47 GB 12:30/3.67 GB 4:23/2.85 GB 2:30/3.75 GB 20:32/4.00 GB 3:34/4.19 GB 22:25/3.11 GB 5:32/3.97 GB 8:35/4.33 GB 0:28/3.48 GB 18:25/3.11 GB 21:38/4.64 GB 15:30/3.71 GB 14:29/3.60 GB 11:35/4.32 GB 7:25/3.06 GB 19:35/4.29 GB 10:30/3.71 GB 6:19/2.33 GB 16:33/4.03 GB 9:28/3.45 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 714 out of 714 Hdfs Read Thread Concurrency Bucket: 0:75.58% 1:18.6% 2:4.651% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:1.163% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2856 BytesRead(1s000ms): 63.02 MB, 167.35 MB, 253.10 MB, 354.14 MB, 454.42 MB, 535.93 MB, 622.85 MB, 717.98 MB, 811.50 MB, 895.71 MB, 992.30 MB, 1.06 GB, 1.14 GB, 1.23 GB, 1.33 GB, 1.41 GB, 1.49 GB, 1.59 GB, 1.67 GB, 1.76 GB, 1.85 GB, 1.93 GB, 2.02 GB, 2.12 GB, 2.20 GB, 2.28 GB, 2.37 GB, 2.46 GB, 2.54 GB, 2.63 GB, 2.71 GB, 2.79 GB, 2.90 GB, 2.99 GB, 3.07 GB, 3.16 GB, 3.25 GB, 3.34 GB, 3.42 GB, 3.50 GB, 3.59 GB, 3.69 GB, 3.74 GB - FooterProcessingTime: (Avg: 14.204ms ; Min: 852.084us ; Max: 70.577ms ; Number of samples: 714) - AverageHdfsReadThreadConcurrency: 0.40 - AverageScannerThreadConcurrency: 23.86 - BytesRead: 3.75 GB (4022962394) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.70 GB (3968792794) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.70 GB (3968792794) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 714 (714) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 453.82 MB (475865000) - PerReadThreadRawHdfsThroughput: 432.00 MB/sec - RemoteScanRanges: 529 (529) - RowBatchQueueGetWaitTime: 33s897ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1155312245) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 714 (714) - ScannerThreadsInvoluntaryContextSwitches: 367.92K (367921) - ScannerThreadsTotalWallClockTime: 17m1s - MaterializeTupleTime(*): 16m24s - ScannerThreadsSysTime: 29s278ms - ScannerThreadsUserTime: 14m13s - ScannerThreadsVoluntaryContextSwitches: 65.74K (65742) - TotalRawHdfsReadTime(*): 8s881ms - TotalReadThroughput: 89.22 MB/sec Instance 8d4071b24aee8531:393af00400000009 (host=foo18.boo:22000):(Total: 42s994ms, non-child: 3.217ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 20:26/3.23 GB 4:34/4.19 GB 5:27/3.38 GB 8:42/5.10 GB 9:32/3.98 GB 19:31/3.83 GB 23:35/4.35 GB 13:23/2.83 GB 12:28/3.41 GB 11:31/3.84 GB 22:30/3.66 GB 6:27/3.35 GB 18:27/3.31 GB 1:34/4.25 GB 17:37/4.55 GB 0:21/2.56 GB 3:28/3.48 GB 21:21/2.58 GB 16:23/2.83 GB 2:33/4.06 GB 15:31/3.86 GB 14:32/3.95 GB 10:34/4.21 GB 7:27/3.32 GB MemoryUsage(1s000ms): 375.46 MB, 312.31 MB, 332.57 MB, 314.29 MB, 356.37 MB, 358.14 MB, 374.93 MB, 392.29 MB, 376.52 MB, 373.50 MB, 367.12 MB, 398.76 MB, 346.58 MB, 367.61 MB, 373.62 MB, 352.54 MB, 384.32 MB, 363.57 MB, 387.17 MB, 361.58 MB, 376.58 MB, 369.20 MB, 366.60 MB, 361.69 MB, 377.70 MB, 385.73 MB, 382.75 MB, 370.57 MB, 373.21 MB, 386.58 MB, 345.41 MB, 345.47 MB, 378.13 MB, 366.29 MB, 387.56 MB, 362.02 MB, 371.39 MB, 386.65 MB, 356.71 MB, 385.67 MB, 374.93 MB, 367.88 MB ThreadUsage(1s000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25 - AverageThreadTokens: 24.92 - BloomFilterBytes: 0 - PeakMemoryUsage: 451.76 MB (473706749) - PerHostPeakMemUsage: 451.76 MB (473706749) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 232.270us - TotalStorageWaitTime: 33s129ms - TotalThreadsInvoluntaryContextSwitches: 379.37K (379370) - TotalThreadsTotalWallClockTime: 17m43s - TotalThreadsSysTime: 36s730ms - TotalThreadsUserTime: 14m10s - TotalThreadsVoluntaryContextSwitches: 70.32K (70318) Fragment Instance Lifecycle Timings: - ExecTime: 360.580us - ExecTreeExecTime: 4.191us - OpenTime: 42s968ms - ExecTreeOpenTime: 42s857ms - PrepareTime: 25.518ms - ExecTreePrepareTime: 1.974ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 62.973us, non-child: 62.973us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 395.18 KB/sec - OverallThroughput: 248.12 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 7.911us - TransmitDataRPCTime: 39.539us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 131.519ms, non-child: 131.519ms, % non-child: 100.00%) - CodegenTime: 1.167ms - CompileTime: 23.742ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 85.942ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 21.303ms AGGREGATION_NODE (id=9):(Total: 42s859ms, non-child: 8s606ms, % non-child: 20.08%) ExecOption: Codegen Enabled - BuildTime: 8s589ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.64 MB (26885740) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 34s253ms, non-child: 38.660ms, % non-child: 0.11%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 170.245us, non-child: 114.887us, % non-child: 67.48%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 55.358us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 67.041us, non-child: 59.640us, % non-child: 88.96%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 7.401us, non-child: 7.401us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 10.289us, non-child: 10.289us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 34s214ms, non-child: 34s214ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 20:26/3.23 GB 4:34/4.19 GB 5:27/3.38 GB 8:42/5.10 GB 9:32/3.98 GB 19:31/3.83 GB 23:35/4.35 GB 13:23/2.83 GB 12:28/3.41 GB 11:31/3.84 GB 22:30/3.66 GB 6:27/3.35 GB 18:27/3.31 GB 1:34/4.25 GB 17:37/4.55 GB 0:21/2.56 GB 3:28/3.48 GB 21:21/2.58 GB 16:23/2.83 GB 2:33/4.06 GB 15:31/3.86 GB 14:32/3.95 GB 10:34/4.21 GB 7:27/3.32 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 714 out of 714 Hdfs Read Thread Concurrency Bucket: 0:77.65% 1:18.82% 2:3.529% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2856 BytesRead(1s000ms): 118.53 MB, 204.35 MB, 284.64 MB, 377.52 MB, 479.09 MB, 576.92 MB, 654.40 MB, 747.96 MB, 843.04 MB, 921.77 MB, 1018.37 MB, 1.08 GB, 1.17 GB, 1.26 GB, 1.36 GB, 1.43 GB, 1.52 GB, 1.61 GB, 1.71 GB, 1.78 GB, 1.87 GB, 1.96 GB, 2.05 GB, 2.13 GB, 2.22 GB, 2.31 GB, 2.40 GB, 2.48 GB, 2.58 GB, 2.66 GB, 2.74 GB, 2.84 GB, 2.93 GB, 3.01 GB, 3.10 GB, 3.19 GB, 3.27 GB, 3.36 GB, 3.46 GB, 3.54 GB, 3.62 GB, 3.71 GB - FooterProcessingTime: (Avg: 14.051ms ; Min: 1.309ms ; Max: 86.382ms ; Number of samples: 714) - AverageHdfsReadThreadConcurrency: 0.26 - AverageScannerThreadConcurrency: 23.92 - BytesRead: 3.74 GB (4020253449) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.69 GB (3965981449) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.69 GB (3965981449) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 714 (714) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 450.95 MB (472855170) - PerReadThreadRawHdfsThroughput: 444.19 MB/sec - RemoteScanRanges: 530 (530) - RowBatchQueueGetWaitTime: 33s922ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.15B (1154518546) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 714 (714) - ScannerThreadsInvoluntaryContextSwitches: 377.42K (377423) - ScannerThreadsTotalWallClockTime: 17m - MaterializeTupleTime(*): 16m23s - ScannerThreadsSysTime: 29s426ms - ScannerThreadsUserTime: 14m10s - ScannerThreadsVoluntaryContextSwitches: 65.26K (65258) - TotalRawHdfsReadTime(*): 8s631ms - TotalReadThroughput: 90.21 MB/sec Instance 8d4071b24aee8531:393af00400000012 (host=foo03.boo:22000):(Total: 42s883ms, non-child: 3.598ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 23:28/3.48 GB 2:24/2.98 GB 10:24/2.95 GB 7:27/3.35 GB 14:22/2.73 GB 18:34/4.18 GB 15:25/3.09 GB 1:29/3.54 GB 9:38/4.67 GB 19:33/4.05 GB 13:33/4.08 GB 21:32/4.00 GB 4:26/3.19 GB 11:29/3.53 GB 6:38/4.71 GB 16:27/3.32 GB 3:27/3.34 GB 20:33/4.06 GB 8:31/3.86 GB 22:35/4.27 GB 5:31/3.77 GB 17:33/4.04 GB 0:31/3.83 GB 12:25/3.11 GB MemoryUsage(1s000ms): 140.36 MB, 341.57 MB, 347.91 MB, 322.40 MB, 356.37 MB, 373.59 MB, 385.82 MB, 367.12 MB, 369.40 MB, 383.49 MB, 369.75 MB, 375.76 MB, 359.83 MB, 380.49 MB, 373.03 MB, 361.89 MB, 409.62 MB, 369.66 MB, 341.93 MB, 365.27 MB, 382.45 MB, 370.84 MB, 374.41 MB, 353.64 MB, 349.27 MB, 380.08 MB, 372.70 MB, 383.51 MB, 361.84 MB, 371.62 MB, 404.03 MB, 340.36 MB, 386.29 MB, 349.99 MB, 378.48 MB, 369.77 MB, 365.71 MB, 373.46 MB, 375.49 MB, 379.97 MB, 355.91 MB, 385.67 MB, 265.73 MB ThreadUsage(1s000ms): 13, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 17 - AverageThreadTokens: 24.53 - BloomFilterBytes: 0 - PeakMemoryUsage: 445.80 MB (467453391) - PerHostPeakMemUsage: 445.80 MB (467453391) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 266.558us - TotalStorageWaitTime: 33s638ms - TotalThreadsInvoluntaryContextSwitches: 357.62K (357621) - TotalThreadsTotalWallClockTime: 17m38s - TotalThreadsSysTime: 36s622ms - TotalThreadsUserTime: 14m12s - TotalThreadsVoluntaryContextSwitches: 70.55K (70554) Fragment Instance Lifecycle Timings: - ExecTime: 420.846us - ExecTreeExecTime: 3.405us - OpenTime: 42s854ms - ExecTreeOpenTime: 42s732ms - PrepareTime: 28.202ms - ExecTreePrepareTime: 2.125ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 61.836us, non-child: 61.836us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 388.34 KB/sec - OverallThroughput: 252.68 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 8.676us - TransmitDataRPCTime: 40.235us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 144.954ms, non-child: 144.954ms, % non-child: 100.00%) - CodegenTime: 1.501ms - CompileTime: 26.065ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 94.962ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 23.271ms AGGREGATION_NODE (id=9):(Total: 42s734ms, non-child: 8s767ms, % non-child: 20.52%) ExecOption: Codegen Enabled - BuildTime: 8s748ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.12 MB (26345068) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 33s967ms, non-child: 22.995ms, % non-child: 0.07%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 156.899us, non-child: 101.819us, % non-child: 64.89%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 55.080us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 64.694us, non-child: 55.957us, % non-child: 86.49%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 8.737us, non-child: 8.737us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 10.142us, non-child: 10.142us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 33s943ms, non-child: 33s943ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 23:28/3.48 GB 2:24/2.98 GB 10:24/2.95 GB 7:27/3.35 GB 14:22/2.73 GB 18:34/4.18 GB 15:25/3.09 GB 1:29/3.54 GB 9:38/4.67 GB 19:33/4.05 GB 13:33/4.08 GB 21:32/4.00 GB 4:26/3.19 GB 11:29/3.53 GB 6:38/4.71 GB 16:27/3.32 GB 3:27/3.34 GB 20:33/4.06 GB 8:31/3.86 GB 22:35/4.27 GB 5:31/3.77 GB 17:33/4.04 GB 0:31/3.83 GB 12:25/3.11 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 715 out of 715 Hdfs Read Thread Concurrency Bucket: 0:83.53% 1:14.12% 2:2.353% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2860 BytesRead(1s000ms): 57.76 MB, 152.94 MB, 244.48 MB, 342.46 MB, 424.04 MB, 527.10 MB, 614.08 MB, 710.32 MB, 798.66 MB, 891.13 MB, 986.22 MB, 1.05 GB, 1.13 GB, 1.22 GB, 1.31 GB, 1.40 GB, 1.49 GB, 1.58 GB, 1.66 GB, 1.76 GB, 1.84 GB, 1.92 GB, 2.02 GB, 2.12 GB, 2.19 GB, 2.28 GB, 2.38 GB, 2.45 GB, 2.54 GB, 2.64 GB, 2.72 GB, 2.81 GB, 2.91 GB, 2.98 GB, 3.08 GB, 3.17 GB, 3.26 GB, 3.33 GB, 3.43 GB, 3.52 GB, 3.60 GB, 3.69 GB, 3.75 GB - FooterProcessingTime: (Avg: 14.448ms ; Min: 901.640us ; Max: 117.936ms ; Number of samples: 715) - AverageHdfsReadThreadConcurrency: 0.19 - AverageScannerThreadConcurrency: 23.81 - BytesRead: 3.75 GB (4022261959) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.70 GB (3969013959) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.70 GB (3969013959) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 715 (715) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 443.44 MB (464979796) - PerReadThreadRawHdfsThroughput: 420.52 MB/sec - RemoteScanRanges: 520 (520) - RowBatchQueueGetWaitTime: 33s662ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1155057076) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 715 (715) - ScannerThreadsInvoluntaryContextSwitches: 355.70K (355698) - ScannerThreadsTotalWallClockTime: 16m55s - MaterializeTupleTime(*): 16m18s - ScannerThreadsSysTime: 29s196ms - ScannerThreadsUserTime: 14m12s - ScannerThreadsVoluntaryContextSwitches: 65.49K (65486) - TotalRawHdfsReadTime(*): 9s121ms - TotalReadThroughput: 89.21 MB/sec Instance 8d4071b24aee8531:393af00400000001 (host=foo06.boo:22000):(Total: 42s724ms, non-child: 3.254ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 20:26/3.25 GB 12:27/3.27 GB 21:33/4.08 GB 22:30/3.65 GB 17:24/2.98 GB 1:26/3.19 GB 0:32/3.95 GB 9:37/4.49 GB 10:36/4.48 GB 16:30/3.68 GB 4:36/4.38 GB 18:30/3.72 GB 15:32/3.94 GB 3:21/2.57 GB 7:35/4.34 GB 23:23/2.86 GB 6:34/4.23 GB 19:27/3.35 GB 2:35/4.32 GB 13:30/3.73 GB 14:24/2.98 GB 8:29/3.59 GB 5:25/3.11 GB 11:33/4.06 GB MemoryUsage(1s000ms): 365.88 MB, 308.89 MB, 345.46 MB, 342.48 MB, 360.19 MB, 388.46 MB, 372.96 MB, 366.57 MB, 365.26 MB, 390.76 MB, 387.45 MB, 369.80 MB, 342.54 MB, 338.27 MB, 378.13 MB, 379.71 MB, 386.26 MB, 348.63 MB, 367.31 MB, 388.95 MB, 331.86 MB, 351.39 MB, 386.89 MB, 348.48 MB, 349.83 MB, 356.76 MB, 379.31 MB, 378.16 MB, 373.58 MB, 366.17 MB, 363.41 MB, 371.73 MB, 369.11 MB, 356.54 MB, 373.99 MB, 399.19 MB, 347.07 MB, 386.41 MB, 378.86 MB, 336.61 MB, 342.66 MB, 390.95 MB ThreadUsage(1s000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 24 - AverageThreadTokens: 24.80 - BloomFilterBytes: 0 - PeakMemoryUsage: 454.90 MB (477000999) - PerHostPeakMemUsage: 454.90 MB (477000999) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 172.208us - TotalStorageWaitTime: 32s810ms - TotalThreadsInvoluntaryContextSwitches: 334.79K (334790) - TotalThreadsTotalWallClockTime: 17m37s - TotalThreadsSysTime: 37s107ms - TotalThreadsUserTime: 14m16s - TotalThreadsVoluntaryContextSwitches: 69.34K (69345) Fragment Instance Lifecycle Timings: - ExecTime: 365.125us - ExecTreeExecTime: 3.992us - OpenTime: 42s697ms - ExecTreeOpenTime: 42s586ms - PrepareTime: 26.055ms - ExecTreePrepareTime: 2.300ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 49.772us, non-child: 49.772us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 666.99 KB/sec - OverallThroughput: 313.93 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 7.630us - TransmitDataRPCTime: 23.426us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 131.974ms, non-child: 131.974ms, % non-child: 100.00%) - CodegenTime: 1.162ms - CompileTime: 23.647ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 86.370ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 21.448ms AGGREGATION_NODE (id=9):(Total: 42s589ms, non-child: 8s613ms, % non-child: 20.22%) ExecOption: Codegen Enabled - BuildTime: 8s585ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.64 MB (26885740) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 33s975ms, non-child: 22.179ms, % non-child: 0.07%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 186.053us, non-child: 117.061us, % non-child: 62.92%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 68.992us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 93.282us, non-child: 83.166us, % non-child: 89.16%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 10.116us, non-child: 10.116us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 10.745us, non-child: 10.745us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 33s953ms, non-child: 33s953ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 20:26/3.25 GB 12:27/3.27 GB 21:33/4.08 GB 22:30/3.65 GB 17:24/2.98 GB 1:26/3.19 GB 0:32/3.95 GB 9:37/4.49 GB 10:36/4.48 GB 16:30/3.68 GB 4:36/4.38 GB 18:30/3.72 GB 15:32/3.94 GB 3:21/2.57 GB 7:35/4.34 GB 23:23/2.86 GB 6:34/4.23 GB 19:27/3.35 GB 2:35/4.32 GB 13:30/3.73 GB 14:24/2.98 GB 8:29/3.59 GB 5:25/3.11 GB 11:33/4.06 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 715 out of 715 Hdfs Read Thread Concurrency Bucket: 0:83.53% 1:14.12% 2:2.353% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2860 BytesRead(1s000ms): 115.13 MB, 192.84 MB, 277.05 MB, 365.93 MB, 471.98 MB, 564.38 MB, 643.16 MB, 746.35 MB, 841.45 MB, 927.22 MB, 1015.31 MB, 1.08 GB, 1.17 GB, 1.27 GB, 1.35 GB, 1.45 GB, 1.53 GB, 1.61 GB, 1.70 GB, 1.80 GB, 1.88 GB, 1.96 GB, 2.06 GB, 2.14 GB, 2.23 GB, 2.32 GB, 2.41 GB, 2.49 GB, 2.58 GB, 2.67 GB, 2.75 GB, 2.84 GB, 2.95 GB, 3.02 GB, 3.11 GB, 3.21 GB, 3.29 GB, 3.37 GB, 3.47 GB, 3.55 GB, 3.64 GB, 3.73 GB - FooterProcessingTime: (Avg: 14.924ms ; Min: 948.491us ; Max: 71.638ms ; Number of samples: 715) - AverageHdfsReadThreadConcurrency: 0.19 - AverageScannerThreadConcurrency: 23.80 - BytesRead: 3.75 GB (4024303024) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.70 GB (3968699824) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.70 GB (3968699824) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 715 (715) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 452.67 MB (474661548) - PerReadThreadRawHdfsThroughput: 403.10 MB/sec - RemoteScanRanges: 543 (543) - RowBatchQueueGetWaitTime: 33s641ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1155657499) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 715 (715) - ScannerThreadsInvoluntaryContextSwitches: 332.90K (332904) - ScannerThreadsTotalWallClockTime: 16m54s - MaterializeTupleTime(*): 16m17s - ScannerThreadsSysTime: 29s836ms - ScannerThreadsUserTime: 14m16s - ScannerThreadsVoluntaryContextSwitches: 64.20K (64201) - TotalRawHdfsReadTime(*): 9s520ms - TotalReadThroughput: 90.30 MB/sec Instance 8d4071b24aee8531:393af00400000008 (host=foo04.boo:22000):(Total: 42s721ms, non-child: 3.659ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 11:28/3.43 GB 3:31/3.79 GB 0:32/3.97 GB 8:19/2.38 GB 18:32/3.94 GB 2:29/3.60 GB 10:24/2.95 GB 12:36/4.45 GB 19:29/3.58 GB 15:32/3.98 GB 16:33/4.11 GB 21:30/3.75 GB 4:32/3.96 GB 6:34/4.21 GB 20:25/3.03 GB 9:38/4.67 GB 1:30/3.73 GB 14:37/4.56 GB 22:30/3.71 GB 5:40/4.92 GB 7:22/2.72 GB 17:35/4.28 GB 13:35/4.35 GB MemoryUsage(1s000ms): 321.77 MB, 343.38 MB, 320.83 MB, 339.96 MB, 351.00 MB, 375.72 MB, 376.63 MB, 374.54 MB, 374.07 MB, 358.03 MB, 399.82 MB, 386.13 MB, 357.97 MB, 388.78 MB, 378.94 MB, 372.57 MB, 378.76 MB, 393.32 MB, 372.36 MB, 380.84 MB, 379.80 MB, 369.89 MB, 364.75 MB, 348.30 MB, 369.26 MB, 349.07 MB, 378.12 MB, 371.97 MB, 364.08 MB, 358.94 MB, 369.95 MB, 373.88 MB, 366.29 MB, 367.13 MB, 371.50 MB, 369.99 MB, 349.50 MB, 351.73 MB, 354.01 MB, 367.26 MB, 370.07 MB, 339.05 MB ThreadUsage(1s000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 24 - AverageThreadTokens: 24.87 - BloomFilterBytes: 0 - PeakMemoryUsage: 462.15 MB (484594983) - PerHostPeakMemUsage: 462.15 MB (484594983) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 170.575us - TotalStorageWaitTime: 32s119ms - TotalThreadsInvoluntaryContextSwitches: 382.34K (382342) - TotalThreadsTotalWallClockTime: 17m35s - TotalThreadsSysTime: 37s072ms - TotalThreadsUserTime: 14m6s - TotalThreadsVoluntaryContextSwitches: 70.20K (70199) Fragment Instance Lifecycle Timings: - ExecTime: 636.924us - ExecTreeExecTime: 4.455us - OpenTime: 42s693ms - ExecTreeOpenTime: 42s580ms - PrepareTime: 26.708ms - ExecTreePrepareTime: 2.103ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 43.654us, non-child: 43.654us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 781.60 KB/sec - OverallThroughput: 357.93 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.696us - TransmitDataRPCTime: 19.991us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 134.974ms, non-child: 134.974ms, % non-child: 100.00%) - CodegenTime: 1.274ms - CompileTime: 24.661ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 87.588ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 22.153ms AGGREGATION_NODE (id=9):(Total: 42s582ms, non-child: 8s666ms, % non-child: 20.35%) ExecOption: Codegen Enabled - BuildTime: 8s644ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.64 MB (26885740) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 33s916ms, non-child: 34.837ms, % non-child: 0.10%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 140.585us, non-child: 85.812us, % non-child: 61.04%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 54.773us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 60.702us, non-child: 53.478us, % non-child: 88.10%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 7.224us, non-child: 7.224us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 13.382us, non-child: 13.382us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 33s881ms, non-child: 33s881ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 11:28/3.43 GB 3:31/3.79 GB 0:32/3.97 GB 8:19/2.38 GB 18:32/3.94 GB 2:29/3.60 GB 10:24/2.95 GB 12:36/4.45 GB 19:29/3.58 GB 15:32/3.98 GB 16:33/4.11 GB 21:30/3.75 GB 4:32/3.96 GB 6:34/4.21 GB 20:25/3.03 GB 9:38/4.67 GB 1:30/3.73 GB 14:37/4.56 GB 22:30/3.71 GB 5:40/4.92 GB 7:22/2.72 GB 17:35/4.28 GB 13:35/4.35 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 713 out of 713 Hdfs Read Thread Concurrency Bucket: 0:74.12% 1:24.71% 2:1.176% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2852 BytesRead(1s000ms): 113.18 MB, 180.51 MB, 263.24 MB, 352.90 MB, 475.04 MB, 551.10 MB, 629.93 MB, 743.97 MB, 831.09 MB, 907.01 MB, 1010.20 MB, 1.08 GB, 1.15 GB, 1.24 GB, 1.35 GB, 1.44 GB, 1.51 GB, 1.62 GB, 1.70 GB, 1.77 GB, 1.87 GB, 1.96 GB, 2.04 GB, 2.14 GB, 2.23 GB, 2.32 GB, 2.40 GB, 2.50 GB, 2.58 GB, 2.66 GB, 2.75 GB, 2.85 GB, 2.93 GB, 3.02 GB, 3.11 GB, 3.19 GB, 3.29 GB, 3.37 GB, 3.46 GB, 3.55 GB, 3.63 GB, 3.72 GB - FooterProcessingTime: (Avg: 13.797ms ; Min: 939.000us ; Max: 105.470ms ; Number of samples: 713) - AverageHdfsReadThreadConcurrency: 0.27 - AverageScannerThreadConcurrency: 23.87 - BytesRead: 3.74 GB (4018864498) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.69 GB (3963261298) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.69 GB (3963261298) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 24 (24) - NumRowGroups: 713 (713) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 459.33 MB (481646252) - PerReadThreadRawHdfsThroughput: 430.25 MB/sec - RemoteScanRanges: 543 (543) - RowBatchQueueGetWaitTime: 33s639ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.15B (1154173681) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 713 (713) - ScannerThreadsInvoluntaryContextSwitches: 380.30K (380300) - ScannerThreadsTotalWallClockTime: 16m53s - MaterializeTupleTime(*): 16m17s - ScannerThreadsSysTime: 29s702ms - ScannerThreadsUserTime: 14m6s - ScannerThreadsVoluntaryContextSwitches: 65.16K (65157) - TotalRawHdfsReadTime(*): 8s908ms - TotalReadThroughput: 90.18 MB/sec Instance 8d4071b24aee8531:393af0040000000d (host=foo02.boo:22000):(Total: 42s704ms, non-child: 3.181ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 20:28/3.48 GB 4:25/3.10 GB 18:30/3.67 GB 5:27/3.31 GB 16:25/3.08 GB 1:32/3.94 GB 17:36/4.44 GB 10:29/3.57 GB 3:32/3.96 GB 13:29/3.54 GB 8:29/3.56 GB 0:37/4.57 GB 14:33/4.06 GB 21:33/4.08 GB 22:34/4.17 GB 7:29/3.60 GB 23:25/3.11 GB 6:16/2.00 GB 15:29/3.61 GB 9:26/3.19 GB 12:37/4.57 GB 11:34/4.21 GB 19:26/3.16 GB 2:33/4.08 GB MemoryUsage(1s000ms): 175.38 MB, 321.60 MB, 296.05 MB, 344.40 MB, 322.10 MB, 406.72 MB, 360.40 MB, 352.79 MB, 375.79 MB, 378.49 MB, 374.41 MB, 366.37 MB, 381.15 MB, 380.27 MB, 348.17 MB, 372.26 MB, 364.05 MB, 350.48 MB, 375.53 MB, 355.00 MB, 357.75 MB, 388.96 MB, 357.14 MB, 380.33 MB, 384.43 MB, 363.40 MB, 374.93 MB, 360.99 MB, 352.56 MB, 363.18 MB, 347.69 MB, 361.36 MB, 375.27 MB, 366.89 MB, 375.14 MB, 388.67 MB, 374.84 MB, 354.84 MB, 342.59 MB, 381.38 MB, 366.15 MB, 406.65 MB, 154.65 MB ThreadUsage(1s000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 11 - AverageThreadTokens: 24.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 452.13 MB (474091001) - PerHostPeakMemUsage: 452.13 MB (474091001) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 211.444us - TotalStorageWaitTime: 32s960ms - TotalThreadsInvoluntaryContextSwitches: 315.53K (315529) - TotalThreadsTotalWallClockTime: 17m35s - TotalThreadsSysTime: 36s169ms - TotalThreadsUserTime: 14m15s - TotalThreadsVoluntaryContextSwitches: 71.23K (71229) Fragment Instance Lifecycle Timings: - ExecTime: 353.615us - ExecTreeExecTime: 3.443us - OpenTime: 42s678ms - ExecTreeOpenTime: 42s567ms - PrepareTime: 25.949ms - ExecTreePrepareTime: 1.986ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 42.258us, non-child: 42.258us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 831.47 KB/sec - OverallThroughput: 369.75 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 8.542us - TransmitDataRPCTime: 18.792us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 131.785ms, non-child: 131.785ms, % non-child: 100.00%) - CodegenTime: 1.173ms - CompileTime: 23.948ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 85.648ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 21.657ms AGGREGATION_NODE (id=9):(Total: 42s569ms, non-child: 8s730ms, % non-child: 20.51%) ExecOption: Codegen Enabled - BuildTime: 8s712ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.64 MB (26885740) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 33s838ms, non-child: 51.454ms, % non-child: 0.15%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 155.066us, non-child: 98.450us, % non-child: 63.49%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 56.616us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 72.475us, non-child: 60.450us, % non-child: 83.41%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 12.025us, non-child: 12.025us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 9.432us, non-child: 9.432us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 33s787ms, non-child: 33s787ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 20:28/3.48 GB 4:25/3.10 GB 18:30/3.67 GB 5:27/3.31 GB 16:25/3.08 GB 1:32/3.94 GB 17:36/4.44 GB 10:29/3.57 GB 3:32/3.96 GB 13:29/3.54 GB 8:29/3.56 GB 0:37/4.57 GB 14:33/4.06 GB 21:33/4.08 GB 22:34/4.17 GB 7:29/3.60 GB 23:25/3.11 GB 6:16/2.00 GB 15:29/3.61 GB 9:26/3.19 GB 12:37/4.57 GB 11:34/4.21 GB 19:26/3.16 GB 2:33/4.08 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 714 out of 714 Hdfs Read Thread Concurrency Bucket: 0:86.05% 1:12.79% 2:1.163% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2856 BytesRead(1s000ms): 60.29 MB, 164.98 MB, 249.78 MB, 350.02 MB, 446.32 MB, 540.46 MB, 619.24 MB, 733.34 MB, 813.24 MB, 900.51 MB, 1002.22 MB, 1.06 GB, 1.15 GB, 1.24 GB, 1.32 GB, 1.41 GB, 1.51 GB, 1.59 GB, 1.67 GB, 1.76 GB, 1.85 GB, 1.93 GB, 2.03 GB, 2.12 GB, 2.21 GB, 2.29 GB, 2.38 GB, 2.47 GB, 2.55 GB, 2.64 GB, 2.73 GB, 2.81 GB, 2.90 GB, 3.00 GB, 3.08 GB, 3.17 GB, 3.27 GB, 3.35 GB, 3.44 GB, 3.53 GB, 3.62 GB, 3.70 GB, 3.74 GB - FooterProcessingTime: (Avg: 14.820ms ; Min: 949.081us ; Max: 83.339ms ; Number of samples: 714) - AverageHdfsReadThreadConcurrency: 0.15 - AverageScannerThreadConcurrency: 23.67 - BytesRead: 3.74 GB (4019199635) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.69 GB (3965542035) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.69 GB (3965542035) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 714 (714) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 448.73 MB (470524798) - PerReadThreadRawHdfsThroughput: 424.36 MB/sec - RemoteScanRanges: 524 (524) - RowBatchQueueGetWaitTime: 33s485ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.15B (1154179778) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 714 (714) - ScannerThreadsInvoluntaryContextSwitches: 313.63K (313626) - ScannerThreadsTotalWallClockTime: 16m52s - MaterializeTupleTime(*): 16m16s - ScannerThreadsSysTime: 28s726ms - ScannerThreadsUserTime: 14m15s - ScannerThreadsVoluntaryContextSwitches: 66.15K (66152) - TotalRawHdfsReadTime(*): 9s032ms - TotalReadThroughput: 89.14 MB/sec Instance 8d4071b24aee8531:393af0040000000b (host=foo08.boo:22000):(Total: 42s629ms, non-child: 3.326ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 15:23/2.88 GB 3:30/3.72 GB 8:23/2.85 GB 11:27/3.35 GB 2:29/3.62 GB 14:23/2.83 GB 20:26/3.25 GB 12:35/4.31 GB 13:35/4.35 GB 16:34/4.21 GB 4:38/4.68 GB 21:30/3.69 GB 10:26/3.20 GB 1:29/3.56 GB 18:31/3.77 GB 23:35/4.30 GB 6:30/3.75 GB 0:31/3.84 GB 17:33/4.01 GB 5:32/3.90 GB 22:31/3.81 GB 19:29/3.59 GB 7:29/3.57 GB 9:26/3.16 GB MemoryUsage(1s000ms): 141.02 MB, 320.72 MB, 309.48 MB, 353.23 MB, 346.64 MB, 385.75 MB, 400.98 MB, 359.45 MB, 391.78 MB, 354.97 MB, 366.40 MB, 387.92 MB, 359.37 MB, 401.03 MB, 351.66 MB, 366.05 MB, 364.47 MB, 393.93 MB, 347.49 MB, 376.31 MB, 366.06 MB, 368.67 MB, 357.50 MB, 349.87 MB, 381.63 MB, 360.59 MB, 362.49 MB, 348.57 MB, 355.63 MB, 358.94 MB, 336.54 MB, 363.19 MB, 367.35 MB, 351.10 MB, 370.99 MB, 361.90 MB, 375.76 MB, 377.89 MB, 382.00 MB, 390.60 MB, 358.65 MB, 367.15 MB, 151.76 MB ThreadUsage(1s000ms): 13, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 11 - AverageThreadTokens: 24.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 457.80 MB (480039291) - PerHostPeakMemUsage: 457.80 MB (480039291) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 232.864us - TotalStorageWaitTime: 34s976ms - TotalThreadsInvoluntaryContextSwitches: 353.05K (353047) - TotalThreadsTotalWallClockTime: 17m34s - TotalThreadsSysTime: 36s897ms - TotalThreadsUserTime: 14m10s - TotalThreadsVoluntaryContextSwitches: 71.78K (71785) Fragment Instance Lifecycle Timings: - ExecTime: 408.987us - ExecTreeExecTime: 4.162us - OpenTime: 42s603ms - ExecTreeOpenTime: 42s491ms - PrepareTime: 26.228ms - ExecTreePrepareTime: 2.015ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 48.855us, non-child: 48.855us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 656.62 KB/sec - OverallThroughput: 319.82 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 8.535us - TransmitDataRPCTime: 23.796us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 132.487ms, non-child: 132.487ms, % non-child: 100.00%) - CodegenTime: 1.166ms - CompileTime: 23.965ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 85.996ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 21.995ms AGGREGATION_NODE (id=9):(Total: 42s493ms, non-child: 8s800ms, % non-child: 20.71%) ExecOption: Codegen Enabled - BuildTime: 8s782ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.64 MB (26885740) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 33s693ms, non-child: 30.024ms, % non-child: 0.09%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 148.255us, non-child: 90.281us, % non-child: 60.90%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 57.974us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 62.804us, non-child: 55.717us, % non-child: 88.72%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 7.087us, non-child: 7.087us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 12.040us, non-child: 12.040us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 33s663ms, non-child: 33s663ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 15:23/2.88 GB 3:30/3.72 GB 8:23/2.85 GB 11:27/3.35 GB 2:29/3.62 GB 14:23/2.83 GB 20:26/3.25 GB 12:35/4.31 GB 13:35/4.35 GB 16:34/4.21 GB 4:38/4.68 GB 21:30/3.69 GB 10:26/3.20 GB 1:29/3.56 GB 18:31/3.77 GB 23:35/4.30 GB 6:30/3.75 GB 0:31/3.84 GB 17:33/4.01 GB 5:32/3.90 GB 22:31/3.81 GB 19:29/3.59 GB 7:29/3.57 GB 9:26/3.16 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 715 out of 715 Hdfs Read Thread Concurrency Bucket: 0:77.65% 1:21.18% 2:1.176% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2860 BytesRead(1s000ms): 59.65 MB, 152.13 MB, 244.08 MB, 353.15 MB, 448.65 MB, 535.26 MB, 619.48 MB, 728.20 MB, 812.41 MB, 896.53 MB, 997.12 MB, 1.07 GB, 1.14 GB, 1.23 GB, 1.33 GB, 1.41 GB, 1.50 GB, 1.60 GB, 1.68 GB, 1.77 GB, 1.86 GB, 1.94 GB, 2.03 GB, 2.12 GB, 2.22 GB, 2.28 GB, 2.38 GB, 2.47 GB, 2.56 GB, 2.64 GB, 2.74 GB, 2.82 GB, 2.90 GB, 3.00 GB, 3.09 GB, 3.18 GB, 3.27 GB, 3.35 GB, 3.43 GB, 3.53 GB, 3.63 GB, 3.71 GB, 3.75 GB - FooterProcessingTime: (Avg: 14.306ms ; Min: 2.519ms ; Max: 89.723ms ; Number of samples: 715) - AverageHdfsReadThreadConcurrency: 0.24 - AverageScannerThreadConcurrency: 23.67 - BytesRead: 3.75 GB (4025203551) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.70 GB (3970009951) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.70 GB (3970009951) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 715 (715) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 449.66 MB (471503616) - PerReadThreadRawHdfsThroughput: 422.74 MB/sec - RemoteScanRanges: 539 (539) - RowBatchQueueGetWaitTime: 33s404ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1155891320) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 715 (715) - ScannerThreadsInvoluntaryContextSwitches: 351.24K (351243) - ScannerThreadsTotalWallClockTime: 16m51s - MaterializeTupleTime(*): 16m12s - ScannerThreadsSysTime: 29s469ms - ScannerThreadsUserTime: 14m9s - ScannerThreadsVoluntaryContextSwitches: 66.70K (66704) - TotalRawHdfsReadTime(*): 9s080ms - TotalReadThroughput: 89.26 MB/sec Instance 8d4071b24aee8531:393af0040000000f (host=foo07.boo:22000):(Total: 42s471ms, non-child: 3.508ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 20:25/3.02 GB 8:21/2.62 GB 23:21/2.61 GB 6:31/3.85 GB 7:36/4.34 GB 9:34/4.17 GB 21:28/3.50 GB 10:25/3.01 GB 11:34/4.17 GB 2:28/3.48 GB 19:32/3.94 GB 16:32/3.96 GB 22:28/3.47 GB 5:31/3.84 GB 3:23/2.78 GB 13:32/3.93 GB 14:32/3.98 GB 0:34/4.23 GB 17:30/3.75 GB 18:35/4.26 GB 1:32/3.98 GB 4:31/3.83 GB 15:27/3.36 GB 12:33/4.08 GB MemoryUsage(1s000ms): 391.13 MB, 342.03 MB, 333.23 MB, 361.28 MB, 340.69 MB, 373.07 MB, 386.41 MB, 382.20 MB, 354.14 MB, 344.81 MB, 367.57 MB, 354.86 MB, 354.73 MB, 373.42 MB, 352.98 MB, 346.19 MB, 357.32 MB, 377.19 MB, 329.57 MB, 379.26 MB, 359.15 MB, 367.74 MB, 336.52 MB, 394.42 MB, 387.71 MB, 378.68 MB, 338.27 MB, 356.93 MB, 367.53 MB, 373.29 MB, 354.46 MB, 383.71 MB, 378.15 MB, 355.09 MB, 389.82 MB, 377.92 MB, 354.00 MB, 382.84 MB, 356.84 MB, 361.81 MB, 370.01 MB, 343.47 MB ThreadUsage(1s000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 22 - AverageThreadTokens: 24.93 - BloomFilterBytes: 0 - PeakMemoryUsage: 460.93 MB (483325265) - PerHostPeakMemUsage: 460.93 MB (483325265) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 179.980us - TotalStorageWaitTime: 33s326ms - TotalThreadsInvoluntaryContextSwitches: 306.38K (306375) - TotalThreadsTotalWallClockTime: 17m30s - TotalThreadsSysTime: 37s020ms - TotalThreadsUserTime: 14m17s - TotalThreadsVoluntaryContextSwitches: 72.22K (72223) Fragment Instance Lifecycle Timings: - ExecTime: 328.336us - ExecTreeExecTime: 3.991us - OpenTime: 42s444ms - ExecTreeOpenTime: 42s334ms - PrepareTime: 26.093ms - ExecTreePrepareTime: 2.077ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 44.340us, non-child: 44.340us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 924.88 KB/sec - OverallThroughput: 352.39 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.026us - TransmitDataRPCTime: 16.894us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 131.531ms, non-child: 131.531ms, % non-child: 100.00%) - CodegenTime: 1.251ms - CompileTime: 24.062ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 85.324ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 21.628ms AGGREGATION_NODE (id=9):(Total: 42s336ms, non-child: 8s668ms, % non-child: 20.48%) ExecOption: Codegen Enabled - BuildTime: 8s651ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.64 MB (26885740) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 33s667ms, non-child: 33.407ms, % non-child: 0.10%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 160.173us, non-child: 102.404us, % non-child: 63.93%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 57.769us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 73.030us, non-child: 61.978us, % non-child: 84.87%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 11.052us, non-child: 11.052us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 9.725us, non-child: 9.725us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 33s633ms, non-child: 33s633ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 20:25/3.02 GB 8:21/2.62 GB 23:21/2.61 GB 6:31/3.85 GB 7:36/4.34 GB 9:34/4.17 GB 21:28/3.50 GB 10:25/3.01 GB 11:34/4.17 GB 2:28/3.48 GB 19:32/3.94 GB 16:32/3.96 GB 22:28/3.47 GB 5:31/3.84 GB 3:23/2.78 GB 13:32/3.93 GB 14:32/3.98 GB 0:34/4.23 GB 17:30/3.75 GB 18:35/4.26 GB 1:32/3.98 GB 4:31/3.83 GB 15:27/3.36 GB 12:33/4.08 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 715 out of 715 Hdfs Read Thread Concurrency Bucket: 0:82.14% 1:14.29% 2:3.571% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2860 BytesRead(1s000ms): 119.31 MB, 196.75 MB, 286.25 MB, 366.37 MB, 474.93 MB, 570.98 MB, 653.70 MB, 747.52 MB, 841.14 MB, 926.52 MB, 1020.51 MB, 1.10 GB, 1.18 GB, 1.26 GB, 1.36 GB, 1.43 GB, 1.54 GB, 1.62 GB, 1.72 GB, 1.80 GB, 1.88 GB, 1.97 GB, 2.07 GB, 2.15 GB, 2.24 GB, 2.34 GB, 2.42 GB, 2.51 GB, 2.60 GB, 2.69 GB, 2.77 GB, 2.86 GB, 2.97 GB, 3.04 GB, 3.12 GB, 3.23 GB, 3.31 GB, 3.40 GB, 3.49 GB, 3.59 GB, 3.67 GB, 3.74 GB - FooterProcessingTime: (Avg: 14.099ms ; Min: 923.335us ; Max: 76.507ms ; Number of samples: 715) - AverageHdfsReadThreadConcurrency: 0.21 - AverageScannerThreadConcurrency: 23.93 - BytesRead: 3.75 GB (4023998498) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.70 GB (3973515298) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.70 GB (3973515298) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 715 (715) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 457.83 MB (480073430) - PerReadThreadRawHdfsThroughput: 461.24 MB/sec - RemoteScanRanges: 493 (493) - RowBatchQueueGetWaitTime: 33s323ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1155561910) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 715 (715) - ScannerThreadsInvoluntaryContextSwitches: 304.63K (304629) - ScannerThreadsTotalWallClockTime: 16m48s - MaterializeTupleTime(*): 16m11s - ScannerThreadsSysTime: 29s571ms - ScannerThreadsUserTime: 14m16s - ScannerThreadsVoluntaryContextSwitches: 67.10K (67099) - TotalRawHdfsReadTime(*): 8s320ms - TotalReadThroughput: 91.36 MB/sec Instance 8d4071b24aee8531:393af00400000005 (host=foo05.boo:22000):(Total: 42s384ms, non-child: 3.401ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 14:26/3.17 GB 10:25/3.10 GB 19:29/3.61 GB 22:31/3.85 GB 4:26/3.23 GB 12:30/3.64 GB 6:33/4.10 GB 5:30/3.75 GB 16:30/3.73 GB 11:28/3.47 GB 9:29/3.57 GB 2:27/3.31 GB 23:26/3.20 GB 7:28/3.43 GB 13:27/3.34 GB 18:32/3.94 GB 1:41/5.03 GB 21:38/4.66 GB 8:30/3.67 GB 20:36/4.43 GB 3:25/3.10 GB 0:31/3.84 GB 17:27/3.35 GB 15:29/3.56 GB MemoryUsage(1s000ms): 160.18 MB, 332.25 MB, 311.51 MB, 355.14 MB, 340.11 MB, 387.92 MB, 354.66 MB, 373.26 MB, 350.02 MB, 363.63 MB, 366.80 MB, 410.41 MB, 375.32 MB, 357.55 MB, 378.62 MB, 330.60 MB, 385.01 MB, 351.38 MB, 357.47 MB, 380.08 MB, 381.04 MB, 366.34 MB, 399.21 MB, 359.72 MB, 346.12 MB, 350.29 MB, 340.08 MB, 375.14 MB, 373.37 MB, 341.81 MB, 359.82 MB, 387.86 MB, 385.92 MB, 407.90 MB, 352.57 MB, 377.50 MB, 369.55 MB, 390.99 MB, 394.15 MB, 365.60 MB, 352.87 MB, 358.37 MB ThreadUsage(1s000ms): 13, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 24 - AverageThreadTokens: 24.54 - BloomFilterBytes: 0 - PeakMemoryUsage: 450.74 MB (472635887) - PerHostPeakMemUsage: 450.74 MB (472635887) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 267.645us - TotalStorageWaitTime: 32s783ms - TotalThreadsInvoluntaryContextSwitches: 275.82K (275819) - TotalThreadsTotalWallClockTime: 17m27s - TotalThreadsSysTime: 37s239ms - TotalThreadsUserTime: 14m25s - TotalThreadsVoluntaryContextSwitches: 72.46K (72459) Fragment Instance Lifecycle Timings: - ExecTime: 465.735us - ExecTreeExecTime: 4.229us - OpenTime: 42s357ms - ExecTreeOpenTime: 42s246ms - PrepareTime: 26.740ms - ExecTreePrepareTime: 1.957ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 60.725us, non-child: 60.725us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 478.24 KB/sec - OverallThroughput: 257.31 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 8.800us - TransmitDataRPCTime: 32.671us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 132.733ms, non-child: 132.733ms, % non-child: 100.00%) - CodegenTime: 1.236ms - CompileTime: 23.954ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 85.943ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 22.310ms AGGREGATION_NODE (id=9):(Total: 42s248ms, non-child: 8s655ms, % non-child: 20.49%) ExecOption: Codegen Enabled - BuildTime: 8s637ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.64 MB (26885740) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 33s592ms, non-child: 27.619ms, % non-child: 0.08%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 149.537us, non-child: 93.230us, % non-child: 62.35%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 56.307us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 70.221us, non-child: 63.040us, % non-child: 89.77%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 7.181us, non-child: 7.181us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 9.326us, non-child: 9.326us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 33s564ms, non-child: 33s564ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 14:26/3.17 GB 10:25/3.10 GB 19:29/3.61 GB 22:31/3.85 GB 4:26/3.23 GB 12:30/3.64 GB 6:33/4.10 GB 5:30/3.75 GB 16:30/3.73 GB 11:28/3.47 GB 9:29/3.57 GB 2:27/3.31 GB 23:26/3.20 GB 7:28/3.43 GB 13:27/3.34 GB 18:32/3.94 GB 1:41/5.03 GB 21:38/4.66 GB 8:30/3.67 GB 20:36/4.43 GB 3:25/3.10 GB 0:31/3.84 GB 17:27/3.35 GB 15:29/3.56 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 714 out of 714 Hdfs Read Thread Concurrency Bucket: 0:77.38% 1:22.62% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2856 BytesRead(1s000ms): 58.63 MB, 153.29 MB, 247.54 MB, 345.70 MB, 442.67 MB, 530.76 MB, 621.96 MB, 722.08 MB, 812.17 MB, 883.98 MB, 994.14 MB, 1.06 GB, 1.13 GB, 1.23 GB, 1.33 GB, 1.41 GB, 1.50 GB, 1.59 GB, 1.68 GB, 1.76 GB, 1.86 GB, 1.94 GB, 2.03 GB, 2.12 GB, 2.21 GB, 2.29 GB, 2.38 GB, 2.47 GB, 2.57 GB, 2.65 GB, 2.75 GB, 2.83 GB, 2.93 GB, 3.01 GB, 3.10 GB, 3.20 GB, 3.28 GB, 3.37 GB, 3.46 GB, 3.55 GB, 3.65 GB, 3.72 GB - FooterProcessingTime: (Avg: 14.177ms ; Min: 2.333ms ; Max: 93.619ms ; Number of samples: 714) - AverageHdfsReadThreadConcurrency: 0.23 - AverageScannerThreadConcurrency: 23.82 - BytesRead: 3.74 GB (4019245899) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.69 GB (3963437899) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.69 GB (3963437899) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 714 (714) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 443.44 MB (464979880) - PerReadThreadRawHdfsThroughput: 417.55 MB/sec - RemoteScanRanges: 545 (545) - RowBatchQueueGetWaitTime: 33s305ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.15B (1154197424) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 714 (714) - ScannerThreadsInvoluntaryContextSwitches: 274.31K (274311) - ScannerThreadsTotalWallClockTime: 16m45s - MaterializeTupleTime(*): 16m8s - ScannerThreadsSysTime: 29s847ms - ScannerThreadsUserTime: 14m25s - ScannerThreadsVoluntaryContextSwitches: 67.41K (67407) - TotalRawHdfsReadTime(*): 9s179ms - TotalReadThroughput: 90.19 MB/sec Instance 8d4071b24aee8531:393af0040000000e (host=foo14.boo:22000):(Total: 42s326ms, non-child: 3.174ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 22:32/3.93 GB 13:23/2.88 GB 14:29/3.60 GB 6:23/2.85 GB 10:29/3.58 GB 19:23/2.82 GB 17:28/3.44 GB 7:25/3.06 GB 9:29/3.60 GB 12:33/4.04 GB 23:34/4.19 GB 15:27/3.31 GB 0:34/4.23 GB 11:30/3.75 GB 2:29/3.58 GB 16:29/3.56 GB 5:34/4.19 GB 4:30/3.70 GB 21:36/4.44 GB 3:28/3.47 GB 20:27/3.35 GB 1:39/4.81 GB 18:34/4.23 GB 8:28/3.48 GB MemoryUsage(1s000ms): 176.19 MB, 303.60 MB, 344.56 MB, 332.86 MB, 343.35 MB, 366.92 MB, 375.58 MB, 366.09 MB, 359.85 MB, 353.15 MB, 375.80 MB, 378.57 MB, 363.96 MB, 386.10 MB, 372.72 MB, 378.88 MB, 357.86 MB, 382.05 MB, 371.39 MB, 372.39 MB, 366.33 MB, 378.11 MB, 363.89 MB, 334.25 MB, 365.27 MB, 336.16 MB, 354.47 MB, 386.51 MB, 362.19 MB, 370.37 MB, 375.61 MB, 350.77 MB, 385.87 MB, 368.85 MB, 368.39 MB, 379.95 MB, 351.11 MB, 392.89 MB, 374.64 MB, 375.67 MB, 364.64 MB, 331.52 MB ThreadUsage(1s000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 22 - AverageThreadTokens: 24.79 - BloomFilterBytes: 0 - PeakMemoryUsage: 457.92 MB (480165317) - PerHostPeakMemUsage: 457.92 MB (480165317) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 211.555us - TotalStorageWaitTime: 33s299ms - TotalThreadsInvoluntaryContextSwitches: 344.59K (344588) - TotalThreadsTotalWallClockTime: 17m26s - TotalThreadsSysTime: 35s399ms - TotalThreadsUserTime: 14m16s - TotalThreadsVoluntaryContextSwitches: 68.93K (68934) Fragment Instance Lifecycle Timings: - ExecTime: 352.088us - ExecTreeExecTime: 4.552us - OpenTime: 42s300ms - ExecTreeOpenTime: 42s190ms - PrepareTime: 25.418ms - ExecTreePrepareTime: 1.860ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 44.595us, non-child: 44.595us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 830.63 KB/sec - OverallThroughput: 350.38 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.140us - TransmitDataRPCTime: 18.811us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 130.429ms, non-child: 130.429ms, % non-child: 100.00%) - CodegenTime: 1.160ms - CompileTime: 23.723ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 84.824ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 21.384ms AGGREGATION_NODE (id=9):(Total: 42s192ms, non-child: 8s225ms, % non-child: 19.50%) ExecOption: Codegen Enabled - BuildTime: 8s208ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.12 MB (26345068) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 33s967ms, non-child: 27.158ms, % non-child: 0.08%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 139.406us, non-child: 85.523us, % non-child: 61.35%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 53.883us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 60.930us, non-child: 53.120us, % non-child: 87.18%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 7.810us, non-child: 7.810us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 11.791us, non-child: 11.791us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 33s939ms, non-child: 33s939ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 22:32/3.93 GB 13:23/2.88 GB 14:29/3.60 GB 6:23/2.85 GB 10:29/3.58 GB 19:23/2.82 GB 17:28/3.44 GB 7:25/3.06 GB 9:29/3.60 GB 12:33/4.04 GB 23:34/4.19 GB 15:27/3.31 GB 0:34/4.23 GB 11:30/3.75 GB 2:29/3.58 GB 16:29/3.56 GB 5:34/4.19 GB 4:30/3.70 GB 21:36/4.44 GB 3:28/3.47 GB 20:27/3.35 GB 1:39/4.81 GB 18:34/4.23 GB 8:28/3.48 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 713 out of 713 Hdfs Read Thread Concurrency Bucket: 0:78.82% 1:18.82% 2:2.353% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2852 BytesRead(1s000ms): 59.25 MB, 170.35 MB, 253.46 MB, 354.91 MB, 455.04 MB, 540.52 MB, 630.17 MB, 733.51 MB, 828.49 MB, 907.38 MB, 1013.21 MB, 1.07 GB, 1.16 GB, 1.25 GB, 1.34 GB, 1.43 GB, 1.51 GB, 1.62 GB, 1.70 GB, 1.78 GB, 1.86 GB, 1.96 GB, 2.05 GB, 2.13 GB, 2.23 GB, 2.32 GB, 2.40 GB, 2.49 GB, 2.58 GB, 2.67 GB, 2.76 GB, 2.85 GB, 2.93 GB, 3.02 GB, 3.12 GB, 3.20 GB, 3.29 GB, 3.39 GB, 3.47 GB, 3.56 GB, 3.65 GB, 3.73 GB - FooterProcessingTime: (Avg: 13.781ms ; Min: 939.026us ; Max: 100.018ms ; Number of samples: 713) - AverageHdfsReadThreadConcurrency: 0.24 - AverageScannerThreadConcurrency: 23.79 - BytesRead: 3.74 GB (4019228609) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.69 GB (3964956609) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.69 GB (3964956609) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 713 (713) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 456.82 MB (479009578) - PerReadThreadRawHdfsThroughput: 446.62 MB/sec - RemoteScanRanges: 530 (530) - RowBatchQueueGetWaitTime: 33s635ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.15B (1154254500) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 713 (713) - ScannerThreadsInvoluntaryContextSwitches: 342.92K (342924) - ScannerThreadsTotalWallClockTime: 16m44s - MaterializeTupleTime(*): 16m7s - ScannerThreadsSysTime: 28s288ms - ScannerThreadsUserTime: 14m15s - ScannerThreadsVoluntaryContextSwitches: 63.95K (63946) - TotalRawHdfsReadTime(*): 8s582ms - TotalReadThroughput: 90.19 MB/sec Instance 8d4071b24aee8531:393af00400000002 (host=foo11.boo:22000):(Total: 42s276ms, non-child: 3.091ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 6:28/3.46 GB 11:27/3.38 GB 3:28/3.46 GB 0:33/4.09 GB 7:34/4.19 GB 4:26/3.22 GB 15:33/4.08 GB 21:34/4.21 GB 23:33/4.11 GB 1:30/3.66 GB 18:28/3.46 GB 19:31/3.85 GB 2:23/2.85 GB 8:32/3.94 GB 16:34/4.16 GB 9:28/3.46 GB 10:35/4.32 GB 17:30/3.68 GB 5:27/3.38 GB 22:38/4.73 GB 20:22/2.71 GB 14:21/2.56 GB 13:28/3.47 GB 12:30/3.70 GB MemoryUsage(1s000ms): 350.32 MB, 294.96 MB, 317.02 MB, 334.19 MB, 385.60 MB, 365.95 MB, 378.31 MB, 384.04 MB, 386.60 MB, 354.49 MB, 388.34 MB, 384.64 MB, 371.88 MB, 364.76 MB, 377.29 MB, 381.43 MB, 391.24 MB, 392.27 MB, 363.63 MB, 382.46 MB, 368.82 MB, 377.24 MB, 329.90 MB, 398.68 MB, 395.40 MB, 370.47 MB, 371.45 MB, 392.98 MB, 363.90 MB, 376.36 MB, 373.63 MB, 334.43 MB, 343.94 MB, 348.63 MB, 350.13 MB, 352.18 MB, 371.57 MB, 387.30 MB, 346.14 MB, 374.99 MB, 351.79 MB, 298.31 MB ThreadUsage(1s000ms): 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 19 - AverageThreadTokens: 24.86 - BloomFilterBytes: 0 - PeakMemoryUsage: 451.22 MB (473139621) - PerHostPeakMemUsage: 451.22 MB (473139621) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 181.715us - TotalStorageWaitTime: 32s131ms - TotalThreadsInvoluntaryContextSwitches: 364.13K (364127) - TotalThreadsTotalWallClockTime: 17m24s - TotalThreadsSysTime: 37s591ms - TotalThreadsUserTime: 14m - TotalThreadsVoluntaryContextSwitches: 70.04K (70041) Fragment Instance Lifecycle Timings: - ExecTime: 324.012us - ExecTreeExecTime: 3.930us - OpenTime: 42s249ms - ExecTreeOpenTime: 42s140ms - PrepareTime: 25.844ms - ExecTreePrepareTime: 1.992ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 46.283us, non-child: 46.283us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 825.02 KB/sec - OverallThroughput: 337.60 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 7.527us - TransmitDataRPCTime: 18.939us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 130.622ms, non-child: 130.622ms, % non-child: 100.00%) - CodegenTime: 1.155ms - CompileTime: 23.740ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 84.786ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 21.564ms AGGREGATION_NODE (id=9):(Total: 42s142ms, non-child: 8s693ms, % non-child: 20.63%) ExecOption: Codegen Enabled - BuildTime: 8s675ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.64 MB (26885740) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 33s448ms, non-child: 26.585ms, % non-child: 0.08%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 156.357us, non-child: 100.385us, % non-child: 64.20%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 55.972us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 64.601us, non-child: 54.693us, % non-child: 84.66%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 9.908us, non-child: 9.908us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 10.127us, non-child: 10.127us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 33s421ms, non-child: 33s421ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:28/3.46 GB 11:27/3.38 GB 3:28/3.46 GB 0:33/4.09 GB 7:34/4.19 GB 4:26/3.22 GB 15:33/4.08 GB 21:34/4.21 GB 23:33/4.11 GB 1:30/3.66 GB 18:28/3.46 GB 19:31/3.85 GB 2:23/2.85 GB 8:32/3.94 GB 16:34/4.16 GB 9:28/3.46 GB 10:35/4.32 GB 17:30/3.68 GB 5:27/3.38 GB 22:38/4.73 GB 20:22/2.71 GB 14:21/2.56 GB 13:28/3.47 GB 12:30/3.70 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 713 out of 713 Hdfs Read Thread Concurrency Bucket: 0:83.33% 1:15.48% 2:0% 3:1.19% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2852 BytesRead(1s000ms): 116.71 MB, 190.35 MB, 283.94 MB, 375.50 MB, 475.90 MB, 570.99 MB, 660.69 MB, 744.86 MB, 853.57 MB, 937.74 MB, 1013.81 MB, 1.09 GB, 1.19 GB, 1.26 GB, 1.37 GB, 1.45 GB, 1.54 GB, 1.63 GB, 1.72 GB, 1.81 GB, 1.89 GB, 1.99 GB, 2.07 GB, 2.16 GB, 2.25 GB, 2.34 GB, 2.42 GB, 2.52 GB, 2.60 GB, 2.69 GB, 2.79 GB, 2.87 GB, 2.96 GB, 3.06 GB, 3.14 GB, 3.23 GB, 3.32 GB, 3.41 GB, 3.50 GB, 3.58 GB, 3.68 GB, 3.74 GB - FooterProcessingTime: (Avg: 14.676ms ; Min: 1.738ms ; Max: 85.834ms ; Number of samples: 713) - AverageHdfsReadThreadConcurrency: 0.19 - AverageScannerThreadConcurrency: 23.86 - BytesRead: 3.74 GB (4019885917) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.69 GB (3964897117) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.69 GB (3964897117) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 713 (713) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 447.02 MB (468730666) - PerReadThreadRawHdfsThroughput: 414.75 MB/sec - RemoteScanRanges: 537 (537) - RowBatchQueueGetWaitTime: 33s180ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.15B (1154446832) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 713 (713) - ScannerThreadsInvoluntaryContextSwitches: 362.08K (362085) - ScannerThreadsTotalWallClockTime: 16m42s - MaterializeTupleTime(*): 16m6s - ScannerThreadsSysTime: 30s326ms - ScannerThreadsUserTime: 13m59s - ScannerThreadsVoluntaryContextSwitches: 64.92K (64919) - TotalRawHdfsReadTime(*): 9s243ms - TotalReadThroughput: 91.28 MB/sec Instance 8d4071b24aee8531:393af00400000014 (host=foo16.boo:22000):(Total: 42s038ms, non-child: 3.361ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 23:32/3.92 GB 17:24/2.98 GB 21:30/3.66 GB 20:25/3.11 GB 19:33/4.01 GB 7:24/2.98 GB 6:37/4.57 GB 4:29/3.61 GB 16:29/3.59 GB 9:29/3.59 GB 0:32/3.89 GB 10:37/4.54 GB 22:29/3.56 GB 5:32/3.97 GB 11:33/4.04 GB 2:36/4.46 GB 13:29/3.48 GB 12:31/3.79 GB 8:31/3.84 GB 18:22/2.66 GB 1:33/4.01 GB 15:25/3.11 GB 14:27/3.34 GB 3:28/3.48 GB MemoryUsage(1s000ms): 157.82 MB, 347.06 MB, 355.62 MB, 339.22 MB, 318.05 MB, 326.64 MB, 336.09 MB, 376.05 MB, 360.25 MB, 368.15 MB, 411.75 MB, 374.69 MB, 370.29 MB, 356.65 MB, 378.06 MB, 338.35 MB, 363.29 MB, 368.92 MB, 374.49 MB, 346.62 MB, 392.85 MB, 364.87 MB, 392.40 MB, 370.37 MB, 371.87 MB, 392.05 MB, 358.66 MB, 370.11 MB, 376.86 MB, 378.08 MB, 379.44 MB, 356.07 MB, 351.35 MB, 373.01 MB, 367.65 MB, 387.49 MB, 372.39 MB, 353.31 MB, 388.78 MB, 375.92 MB, 351.58 MB, 298.12 MB ThreadUsage(1s000ms): 13, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 20 - AverageThreadTokens: 24.61 - BloomFilterBytes: 0 - PeakMemoryUsage: 448.70 MB (470494503) - PerHostPeakMemUsage: 448.70 MB (470494503) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 197.422us - TotalStorageWaitTime: 34s001ms - TotalThreadsInvoluntaryContextSwitches: 295.51K (295510) - TotalThreadsTotalWallClockTime: 17m18s - TotalThreadsSysTime: 33s666ms - TotalThreadsUserTime: 14m21s - TotalThreadsVoluntaryContextSwitches: 65.78K (65778) Fragment Instance Lifecycle Timings: - ExecTime: 437.071us - ExecTreeExecTime: 4.120us - OpenTime: 42s012ms - ExecTreeOpenTime: 41s902ms - PrepareTime: 26.108ms - ExecTreePrepareTime: 2.074ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 204.80 GB (219902328832) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=10):(Total: 44.717us, non-child: 44.717us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 863.35 KB/sec - OverallThroughput: 349.42 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.875us - TransmitDataRPCTime: 18.098us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 130.575ms, non-child: 130.575ms, % non-child: 100.00%) - CodegenTime: 1.172ms - CompileTime: 23.633ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2073212) - NumFunctions: 52 (52) - NumInstructions: 854 (854) - OptimizationTime: 84.776ms - PeakMemoryUsage: 427.00 KB (437248) - PrepareTime: 21.661ms AGGREGATION_NODE (id=9):(Total: 41s904ms, non-child: 7s471ms, % non-child: 17.83%) ExecOption: Codegen Enabled - BuildTime: 7s453ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 25.64 MB (26885740) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) SUBPLAN_NODE (id=1):(Total: 34s433ms, non-child: 16.123ms, % non-child: 0.05%) - PeakMemoryUsage: 14.00 MB (14681088) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=8):(Total: 155.571us, non-child: 97.138us, % non-child: 62.44%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 24.00 KB (24576) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 SUBPLAN_NODE (id=4):(Total: 58.433us, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 8.00 KB (8192) - RowsReturned: 0 (0) - RowsReturnedRate: 0 NESTED_LOOP_JOIN_NODE (id=7):(Total: 65.830us, non-child: 58.789us, % non-child: 89.30%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - ProbeRows: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Nested Loop Join Builder: - PeakMemoryUsage: 0 SINGULAR_ROW_SRC_NODE (id=5): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=6):(Total: 7.041us, non-child: 7.041us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 UNNEST_NODE (id=3):(Total: 9.912us, non-child: 9.912us, % non-child: 100.00%) - AvgCollectionSize: 0.00 - MaxCollectionSize: 0 (0) - MinCollectionSize: 0 (0) - NumCollections: 0 (0) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 34s417ms, non-child: 34s417ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 23:32/3.92 GB 17:24/2.98 GB 21:30/3.66 GB 20:25/3.11 GB 19:33/4.01 GB 7:24/2.98 GB 6:37/4.57 GB 4:29/3.61 GB 16:29/3.59 GB 9:29/3.59 GB 0:32/3.89 GB 10:37/4.54 GB 22:29/3.56 GB 5:32/3.97 GB 11:33/4.04 GB 2:36/4.46 GB 13:29/3.48 GB 12:31/3.79 GB 8:31/3.84 GB 18:22/2.66 GB 1:33/4.01 GB 15:25/3.11 GB 14:27/3.34 GB 3:28/3.48 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 717 out of 717 Hdfs Read Thread Concurrency Bucket: 0:78.31% 1:19.28% 2:2.41% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% File Formats: PARQUET/NONE:2868 BytesRead(1s000ms): 59.70 MB, 159.20 MB, 243.71 MB, 346.90 MB, 444.74 MB, 524.30 MB, 621.79 MB, 729.29 MB, 821.66 MB, 901.61 MB, 995.53 MB, 1.07 GB, 1.16 GB, 1.25 GB, 1.35 GB, 1.43 GB, 1.51 GB, 1.61 GB, 1.71 GB, 1.79 GB, 1.88 GB, 1.97 GB, 2.06 GB, 2.14 GB, 2.24 GB, 2.33 GB, 2.41 GB, 2.51 GB, 2.59 GB, 2.68 GB, 2.77 GB, 2.86 GB, 2.96 GB, 3.05 GB, 3.13 GB, 3.24 GB, 3.31 GB, 3.41 GB, 3.50 GB, 3.59 GB, 3.68 GB, 3.75 GB - FooterProcessingTime: (Avg: 14.161ms ; Min: 986.347us ; Max: 72.313ms ; Number of samples: 717) - AverageHdfsReadThreadConcurrency: 0.24 - AverageScannerThreadConcurrency: 23.89 - BytesRead: 3.75 GB (4026325399) - BytesReadDataNodeCache: 0 - BytesReadLocal: 3.70 GB (3972974999) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 3.70 GB (3972974999) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 25 (25) - NumRowGroups: 717 (717) - NumScannerThreadsStarted: 24 (24) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 447.53 MB (469270188) - PerReadThreadRawHdfsThroughput: 423.56 MB/sec - RemoteScanRanges: 521 (521) - RowBatchQueueGetWaitTime: 34s164ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1156167015) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 717 (717) - ScannerThreadsInvoluntaryContextSwitches: 294.12K (294119) - ScannerThreadsTotalWallClockTime: 16m36s - MaterializeTupleTime(*): 15m59s - ScannerThreadsSysTime: 27s137ms - ScannerThreadsUserTime: 14m21s - ScannerThreadsVoluntaryContextSwitches: 60.65K (60651) - TotalRawHdfsReadTime(*): 9s065ms - TotalReadThroughput: 91.42 MB/sec