================================================================================ SUMMARY: +-----------------+--------+----------+---------+----------+---------+-----------+ | Operator | #Hosts | Avg Time | Std Dev | Max Time | #Rows | Est #Rows | +-----------------+--------+----------+---------+----------+---------+-----------+ | 06:AGGREGATE | 1 | 268.44us | 0ns | 268.44us | 1 | 1 | | 05:EXCHANGE | 1 | 18.22us | 0ns | 18.22us | 1 | 1 | | 03:AGGREGATE | 1 | 2.33ms | 0ns | 2.33ms | 1 | 1 | | 02:HASH JOIN | 1 | 66.96s | 0ns | 66.96s | 0 | 3 | | |--04:EXCHANGE | 1 | 7.61s | 0ns | 7.61s | 350.01M | 180.00M | | | 01:SCAN HDFS | 14 | 302.14ms | 0ns | 341.40ms | 350.01M | 180.00M | | 00:SCAN HDFS | 1 | 10.23ms | 0ns | 10.23ms | 0 | 3 | +-----------------+--------+----------+---------+----------+---------+-----------+ PROFILE: Query (id=7c4dade34ea955d6:dc885f5700000000): Summary: Session ID: 454fe290300cd340:bdebabf166e81cb2 Session Type: BEESWAX Start Time: 2017-02-17 04:36:07.015133000 End Time: Query Type: QUERY Query State: CREATED Query Status: OK Impala Version: impalad version 2.9.0-SNAPSHOT RELEASE (build 9842bfcdbff9dfbb2fb1a8da8eda98ed2d56e76a) User: jenkins Connected User: jenkins Delegated User: Network Address: 10.17.193.6:38793 Default Db: tpch_300_parquet Sql Statement: -- Description : Large broadcast join with no rows qualifying from the probe side. -- Target test case : Broadcast large amounts of data over the network while stressing the -- exchange operator along with HashTable build. SELECT /* +straight_join */ count(*) FROM nation JOIN /* +broadcast */ lineitem ON l_orderkey = n_nationkey WHERE l_partkey > 0 AND l_suppkey > 0 AND l_linenumber > 0 AND n_regionkey < 0 AND l_orderkey < 350000000 Coordinator: vb0204.halxg.cloudera.com:22000 Query Options (non default): NUM_SCANNER_THREADS=12,SYNC_DDL=1,MAX_NUM_RUNTIME_FILTERS=50 Plan: ---------------- Estimated Per-Host Requirements: Memory=6.24GB VCores=2 PLAN-ROOT SINK | 06:AGGREGATE [FINALIZE] | output: count:merge(*) | hosts=1 per-host-mem=unavailable | tuple-ids=2 row-size=8B cardinality=1 | 05:EXCHANGE [UNPARTITIONED] | hosts=1 per-host-mem=unavailable | tuple-ids=2 row-size=8B cardinality=1 | 03:AGGREGATE | output: count(*) | hosts=1 per-host-mem=10.00MB | tuple-ids=2 row-size=8B cardinality=1 | 02:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: n_nationkey = l_orderkey | runtime filters: RF000 <- l_orderkey | hosts=1 per-host-mem=5.90GB | tuple-ids=0,1 row-size=48B cardinality=3 | |--04:EXCHANGE [BROADCAST] | | hosts=14 per-host-mem=0B | | tuple-ids=1 row-size=32B cardinality=179998909 | | | 01:SCAN HDFS [tpch_300_parquet.lineitem, RANDOM] | partitions=1/1 files=264 size=64.36GB | predicates: l_linenumber > 0, l_orderkey < 350000000, l_partkey > 0, l_suppkey > 0 | table stats: 1799989091 rows total | column stats: all | hosts=14 per-host-mem=352.00MB | tuple-ids=1 row-size=32B cardinality=179998909 | 00:SCAN HDFS [tpch_300_parquet.nation, RANDOM] partitions=1/1 files=1 size=2.19KB predicates: n_regionkey < 0, tpch_300_parquet.nation.n_nationkey < 350000000 runtime filters: RF000 -> n_nationkey table stats: 25 rows total column stats: all hosts=1 per-host-mem=32.00MB tuple-ids=0 row-size=16B cardinality=3 ---------------- Estimated Per-Host Mem: 6705060455 Estimated Per-Host VCores: 2 Request Pool: root.jenkins Admission result: Admitted immediately Planner Timeline: 5.958ms - Analysis finished: 1.899ms (1.899ms) - Equivalence classes computed: 2.021ms (122.118us) - Single node plan created: 3.391ms (1.369ms) - Runtime filters computed: 3.505ms (113.656us) - Distributed plan created: 3.738ms (233.887us) - Lineage info computed: 3.864ms (126.038us) - Planning finished: 5.958ms (2.093ms) Query Timeline: 1m14s - Query submitted: 41.881us (41.881us) - Planning finished: 8.902ms (8.860ms) - Submit for admission: 10.325ms (1.423ms) - Completed admission: 10.447ms (121.647us) - Ready to start 16 fragment instances: 10.846ms (398.766us) - All 16 fragment instances started: 13.364ms (2.518ms) - Rows available: 1m14s (1m14s) - First row fetched: 1m14s (6.251ms) - ComputeScanRangeAssignmentTimer: 628.788us ImpalaServer: - ClientFetchWaitTimer: 6.259ms - RowMaterializationTimer: 3.917ms Execution Profile 7c4dade34ea955d6:dc885f5700000000:(Total: 1m14s, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 1 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed Enabled ---------------------------------------------------------------------------------------------------------------------------- 0 2 0 1 LOCAL false 0 (1) N/A N/A true Fragment instance start latencies: Count: 16, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 2ms Per Node Peak Memory Usage: vb0206.halxg.cloudera.com:22000(1.11 GB) vb0220.halxg.cloudera.com:22000(1.11 GB) vb0228.halxg.cloudera.com:22000(1.07 GB) vb0222.halxg.cloudera.com:22000(1.11 GB) vb0224.halxg.cloudera.com:22000(1.10 GB) vb0232.halxg.cloudera.com:22000(1.11 GB) vb0234.halxg.cloudera.com:22000(1.11 GB) vb0218.halxg.cloudera.com:22000(1.10 GB) vb0214.halxg.cloudera.com:22000(26.43 GB) vb0212.halxg.cloudera.com:22000(1.11 GB) vb0216.halxg.cloudera.com:22000(1.11 GB) vb0208.halxg.cloudera.com:22000(1.10 GB) vb0230.halxg.cloudera.com:22000(1.11 GB) vb0204.halxg.cloudera.com:22000(1.11 GB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Coordinator Fragment F02: Instance 7c4dade34ea955d6:dc885f5700000000 (host=vb0204.halxg.cloudera.com:22000):(Total: 1m14s, non-child: 7.154ms, % non-child: 0.01%) MemoryUsage(2s000ms): 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 28.01 KB (28680) - PerHostPeakMemUsage: 1.11 GB (1187442837) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1m14s - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 1m14s - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 1.000ms - TotalThreadsVoluntaryContextSwitches: 6 (6) Fragment Instance Lifecycle Timings: - ExecTime: 6.381ms - ExecTreeExecTime: 6.035us - OpenTime: 1m14s - ExecTreeOpenTime: 1m14s - PrepareTime: 23.188ms - ExecTreePrepareTime: 118.844us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 22.693ms, non-child: 22.693ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 22.146ms AGGREGATION_NODE (id=6):(Total: 1m14s, non-child: 268.442us, % non-child: 0.00%) - BuildTime: 8.851us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.00 KB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=5):(Total: 1m14s, non-child: 18.219us, % non-child: 0.00%) BytesReceived(2s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 - BytesReceived: 16.00 B (16) - ConvertRowBatchTime: 2.309us - DeserializeRowBatchTimer: 11.327us - FirstBatchArrivalWaitTime: 1m14s - PeakMemoryUsage: 0 - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 1m14s, non-child: 6.813ms, % non-child: 0.01%) split sizes: min: 2.19 KB, max: 2.19 KB, avg: 2.19 KB, stddev: 0 completion times: min:1m14s max:1m14s mean: 1m14s stddev:0.000ns execution rates: min:29.00 B/sec max:29.00 B/sec mean:29.00 B/sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 1.99 - BloomFilterBytes: 16.00 MB (16777216) - PeakMemoryUsage: 26.43 GB (28380916821) - PerHostPeakMemUsage: 26.43 GB (28380916821) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1s386ms - TotalNetworkSendTime: 458.393us - TotalStorageWaitTime: 8.999ms - TotalThreadsInvoluntaryContextSwitches: 7.77K (7772) - TotalThreadsTotalWallClockTime: 2m28s - TotalThreadsSysTime: 17s919ms - TotalThreadsUserTime: 56s438ms - TotalThreadsVoluntaryContextSwitches: 7.00K (7000) Fragment Instance Lifecycle Timings: - ExecTime: 789.697us - ExecTreeExecTime: 3.863us - OpenTime: 1m14s - ExecTreeOpenTime: 1m14s - PrepareTime: 30.186ms - ExecTreePrepareTime: 340.999us DataStreamSender (dst_id=5):(Total: 76.804us, non-child: 76.804us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 442.21 KB/sec - OverallThroughput: 203.44 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.254us - TransmitDataRPCTime: 35.334us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 352.164ms, non-child: 352.164ms, % non-child: 100.00%) - CodegenTime: 1.841ms - CompileTime: 85.506ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 90 (90) - NumInstructions: 2.93K (2928) - OptimizationTime: 241.930ms - PeakMemoryUsage: 1.43 MB (1499136) - PrepareTime: 24.188ms AGGREGATION_NODE (id=3):(Total: 1m14s, non-child: 2.334ms, % non-child: 0.00%) - BuildTime: 2.263ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 24.00 KB (24576) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=2):(Total: 1m14s, non-child: 1m6s, % non-child: 89.78%) - BuildRows: 350.01M (350008706) - BuildTime: 1m5s - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 26.42 GB (28364049664) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 1.671us - RowsReturned: 0 (0) - RowsReturnedRate: 0 Hash Join Builder (join_node_id=2): - BuildRowsPartitionTime: 38s775ms - BuildRowsPartitioned: 350.01M (350008706) - GetNewBlockTime: 9.540ms - HashBuckets: 536.87M (536870912) - HashCollisions: 0 (0) - HashTablesBuildTime: 26s908ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 26.42 GB (28364002432) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=4):(Total: 7s612ms, non-child: 7s612ms, % non-child: 100.00%) - BytesReceived: 6.99 GB (7501460242) - ConvertRowBatchTime: 5s887ms - DeserializeRowBatchTimer: 15s933ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 350.01M (350008706) - RowsReturnedRate: 45.98 M/sec - SendersBlockedTimer: 45s220ms - SendersBlockedTotalTimer(*): 8m39s HDFS_SCAN_NODE (id=0):(Total: 10.225ms, non-child: 10.225ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 2.44 KB (2497) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.44 KB (2497) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.44 KB (2497) - DecompressionTime: 7.061us - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDisksAccessed: 1 (1) - NumRowGroups: 1 (1) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 73.00 KB (74752) - PerReadThreadRawHdfsThroughput: 67.59 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 9.426ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 25 (25) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 9.494ms - MaterializeTupleTime(*): 24.730us - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 6 (6) - TotalRawHdfsReadTime(*): 35.231us - TotalReadThroughput: 0.00 /sec Filter 0 (16.00 MB): - Rows processed: 25 (25) - Rows rejected: 0 (0) - Rows total: 25 (25) Fragment F00: Instance 7c4dade34ea955d6:dc885f570000000f (host=vb0214.halxg.cloudera.com:22000):(Total: 1m14s, non-child: 6.813ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 6:1/2.19 KB Filter 0 arrival: 47s460ms MemoryUsage(2s000ms): 193.21 MB, 620.26 MB, 1.11 GB, 1.57 GB, 1.98 GB, 2.48 GB, 2.95 GB, 3.36 GB, 3.76 GB, 4.23 GB, 4.67 GB, 5.11 GB, 5.61 GB, 6.05 GB, 6.48 GB, 6.98 GB, 7.48 GB, 7.92 GB, 8.36 GB, 8.86 GB, 9.36 GB, 9.82 GB, 10.23 GB, 10.83 GB, 11.91 GB, 13.24 GB, 14.64 GB, 15.65 GB, 16.79 GB, 17.84 GB, 19.01 GB, 20.07 GB, 21.24 GB, 22.40 GB, 23.44 GB, 24.61 GB, 25.67 GB ThreadUsage(2s000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2 - AverageThreadTokens: 1.99 - BloomFilterBytes: 16.00 MB (16777216) - PeakMemoryUsage: 26.43 GB (28380916821) - PerHostPeakMemUsage: 26.43 GB (28380916821) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1s386ms - TotalNetworkSendTime: 458.393us - TotalStorageWaitTime: 8.999ms - TotalThreadsInvoluntaryContextSwitches: 7.77K (7772) - TotalThreadsTotalWallClockTime: 2m28s - TotalThreadsSysTime: 17s919ms - TotalThreadsUserTime: 56s438ms - TotalThreadsVoluntaryContextSwitches: 7.00K (7000) Fragment Instance Lifecycle Timings: - ExecTime: 789.697us - ExecTreeExecTime: 3.863us - OpenTime: 1m14s - ExecTreeOpenTime: 1m14s - PrepareTime: 30.186ms - ExecTreePrepareTime: 340.999us DataStreamSender (dst_id=5):(Total: 76.804us, non-child: 76.804us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 442.21 KB/sec - OverallThroughput: 203.44 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.254us - TransmitDataRPCTime: 35.334us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 352.164ms, non-child: 352.164ms, % non-child: 100.00%) - CodegenTime: 1.841ms - CompileTime: 85.506ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 90 (90) - NumInstructions: 2.93K (2928) - OptimizationTime: 241.930ms - PeakMemoryUsage: 1.43 MB (1499136) - PrepareTime: 24.188ms AGGREGATION_NODE (id=3):(Total: 1m14s, non-child: 2.334ms, % non-child: 0.00%) ExecOption: Codegen Enabled - BuildTime: 2.263ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 24.00 KB (24576) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=2):(Total: 1m14s, non-child: 1m6s, % non-child: 89.78%) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 350.01M (350008706) - BuildTime: 1m5s - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 26.42 GB (28364049664) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 1.671us - RowsReturned: 0 (0) - RowsReturnedRate: 0 Hash Join Builder (join_node_id=2): ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 0 of 1 Runtime Filter Published, 1 Disabled - BuildRowsPartitionTime: 38s775ms - BuildRowsPartitioned: 350.01M (350008706) - GetNewBlockTime: 9.540ms - HashBuckets: 536.87M (536870912) - HashCollisions: 0 (0) - HashTablesBuildTime: 26s908ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 26.42 GB (28364002432) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=4):(Total: 7s612ms, non-child: 7s612ms, % non-child: 100.00%) BytesReceived(2s000ms): 78.52 MB, 366.16 MB, 668.57 MB, 976.07 MB, 1.25 GB, 1.55 GB, 1.85 GB, 2.12 GB, 2.40 GB, 2.69 GB, 2.98 GB, 3.27 GB, 3.57 GB, 3.87 GB, 4.18 GB, 4.48 GB, 4.79 GB, 5.09 GB, 5.40 GB, 5.70 GB, 6.01 GB, 6.31 GB, 6.61 GB, 6.89 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB - BytesReceived: 6.99 GB (7501460242) - ConvertRowBatchTime: 5s887ms - DeserializeRowBatchTimer: 15s933ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 350.01M (350008706) - RowsReturnedRate: 45.98 M/sec - SendersBlockedTimer: 45s220ms - SendersBlockedTotalTimer(*): 8m39s HDFS_SCAN_NODE (id=0):(Total: 10.225ms, non-child: 10.225ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:1/2.19 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:2 BytesRead(2s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 - FooterProcessingTime: (Avg: 6.143ms ; Min: 6.143ms ; Max: 6.143ms ; Number of samples: 1) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 2.44 KB (2497) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.44 KB (2497) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.44 KB (2497) - DecompressionTime: 7.061us - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDisksAccessed: 1 (1) - NumRowGroups: 1 (1) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 73.00 KB (74752) - PerReadThreadRawHdfsThroughput: 67.59 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 9.426ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 25 (25) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 9.494ms - MaterializeTupleTime(*): 24.730us - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 6 (6) - TotalRawHdfsReadTime(*): 35.231us - TotalReadThroughput: 0.00 /sec Filter 0 (16.00 MB): - Rows processed: 25 (25) - Rows rejected: 0 (0) - Rows total: 25 (25) Averaged Fragment F01:(Total: 45s955ms, non-child: 36s958ms, % non-child: 80.42%) split sizes: min: 4.45 GB, max: 4.70 GB, avg: 4.60 GB, stddev: 110.56 MB completion times: min:42s861ms max:47s401ms mean: 45s956ms stddev:1s224ms execution rates: min:98.89 MB/sec max:106.54 MB/sec mean:102.46 MB/sec stddev:2.13 MB/sec num instances: 14 - AverageThreadTokens: 11.51 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1185264982) - PerHostPeakMemUsage: 1.81 GB (1942344823) - RowsProduced: 25.00M (25000621) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 37s801ms - TotalStorageWaitTime: 581.017ms - TotalThreadsInvoluntaryContextSwitches: 14.93K (14927) - TotalThreadsTotalWallClockTime: 8m48s - TotalThreadsSysTime: 2s132ms - TotalThreadsUserTime: 28s170ms - TotalThreadsVoluntaryContextSwitches: 47.91K (47910) Fragment Instance Lifecycle Timings: - ExecTime: 45s897ms - ExecTreeExecTime: 365.287ms - OpenTime: 33.376ms - ExecTreeOpenTime: 48.398us - PrepareTime: 24.549ms - ExecTreePrepareTime: 343.701us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 108 (108) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 846.77 MB (887901892) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s638ms, non-child: 8s638ms, % non-child: 100.00%) - BytesSent: 511.00 MB (535818588) - NetworkThroughput(*): 290.20 MB/sec - OverallThroughput: 59.18 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 25.00M (25000621) - SerializeBatchTime: 6s862ms - TransmitDataRPCTime: 1s767ms - UncompressedRowBatchSize: 882.26 MB (925120683) CodeGen:(Total: 55.442ms, non-child: 55.442ms, % non-child: 100.00%) - CodegenTime: 1.410ms - CompileTime: 9.934ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.663ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 22.281ms HDFS_SCAN_NODE (id=1):(Total: 302.143ms, non-child: 302.143ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 10.52 - BytesRead: 1.52 GB (1627133960) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.52 GB (1627133960) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.52 GB (1627133960) - DecompressionTime: 9s153ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1184952376) - PerReadThreadRawHdfsThroughput: 1.39 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 30.973ms - RowBatchQueuePutWaitTime: 7m41s - RowsRead: 128.57M (128570649) - RowsReturned: 25.00M (25000621) - RowsReturnedRate: 82.95 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 10.80K (10803) - ScannerThreadsTotalWallClockTime: 8m2s - MaterializeTupleTime(*): 11s025ms - ScannerThreadsSysTime: 1s249ms - ScannerThreadsUserTime: 20s234ms - ScannerThreadsVoluntaryContextSwitches: 24.49K (24494) - TotalRawHdfsReadTime(*): 1s088ms - TotalReadThroughput: 33.82 MB/sec Fragment F01: Instance 7c4dade34ea955d6:dc885f5700000004 (host=vb0214.halxg.cloudera.com:22000):(Total: 47s400ms, non-child: 38s762ms, % non-child: 81.78%) Hdfs split stats (:<# splits>/): 7:1/253.39 MB 4:2/506.80 MB 0:1/253.39 MB 8:2/506.80 MB 9:2/506.77 MB 3:1/253.37 MB 5:3/760.16 MB 2:4/959.93 MB 1:3/760.19 MB MemoryUsage(1s000ms): 559.99 MB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.07 GB, 1.02 GB, 998.43 MB, 960.60 MB, 938.52 MB, 926.63 MB, 911.52 MB, 880.88 MB, 861.29 MB, 850.53 MB, 817.55 MB, 788.51 MB, 773.12 MB, 735.20 MB, 685.14 MB, 651.02 MB, 618.64 MB, 587.61 MB, 551.27 MB, 588.50 MB, 614.18 MB, 600.92 MB, 580.22 MB, 585.92 MB, 610.10 MB, 773.35 MB, 785.82 MB, 789.54 MB, 773.71 MB, 746.31 MB, 725.11 MB, 684.28 MB, 631.42 MB, 558.03 MB, 536.68 MB, 499.88 MB, 460.72 MB, 424.10 MB, 405.05 MB, 363.90 MB, 256.03 MB ThreadUsage(1s000ms): 7, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 11, 9, 9, 8, 8, 8, 8, 7, 6 - AverageThreadTokens: 11.89 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1187690579) - PerHostPeakMemUsage: 10.98 GB (11786791961) - RowsProduced: 25.73M (25731516) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 39s697ms - TotalStorageWaitTime: 499.884ms - TotalThreadsInvoluntaryContextSwitches: 13.31K (13311) - TotalThreadsTotalWallClockTime: 9m27s - TotalThreadsSysTime: 1s594ms - TotalThreadsUserTime: 24s410ms - TotalThreadsVoluntaryContextSwitches: 49.31K (49310) Fragment Instance Lifecycle Timings: - ExecTime: 47s341ms - ExecTreeExecTime: 371.948ms - OpenTime: 33.342ms - ExecTreeOpenTime: 63.860us - PrepareTime: 25.112ms - ExecTreePrepareTime: 354.820us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 1.41K (1408) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 10.75 GB (11542724608) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s260ms, non-child: 8s260ms, % non-child: 100.00%) - BytesSent: 525.93 MB (551479392) - NetworkThroughput(*): 324.78 MB/sec - OverallThroughput: 63.67 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 25.73M (25731516) - SerializeBatchTime: 6s633ms - TransmitDataRPCTime: 1s619ms - UncompressedRowBatchSize: 908.06 MB (952166628) CodeGen:(Total: 55.907ms, non-child: 55.907ms, % non-child: 100.00%) - CodegenTime: 1.448ms - CompileTime: 9.848ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.713ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 22.820ms HDFS_SCAN_NODE (id=1):(Total: 321.226ms, non-child: 321.226ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:1/253.39 MB 4:2/506.80 MB 0:1/253.39 MB 8:2/506.80 MB 9:2/506.77 MB 3:1/253.37 MB 5:3/760.16 MB 2:4/959.93 MB 1:3/760.19 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:97.87% 1:2.128% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 501.14 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1002.29 MB, 1.06 GB, 1.13 GB, 1.14 GB, 1.14 GB, 1.18 GB, 1.25 GB, 1.47 GB, 1.50 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB - FooterProcessingTime: (Avg: 3.318ms ; Min: 1.478ms ; Max: 4.616ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 11.01 - BytesRead: 1.53 GB (1645560524) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.53 GB (1645560524) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.53 GB (1645560524) - DecompressionTime: 7s657ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1187143232) - PerReadThreadRawHdfsThroughput: 1.45 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 29.320ms - RowBatchQueuePutWaitTime: 8m22s - RowsRead: 130.03M (130033379) - RowsReturned: 25.73M (25731516) - RowsReturnedRate: 80.10 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 9.15K (9147) - ScannerThreadsTotalWallClockTime: 8m40s - MaterializeTupleTime(*): 9s263ms - ScannerThreadsSysTime: 1s021ms - ScannerThreadsUserTime: 16s921ms - ScannerThreadsVoluntaryContextSwitches: 25.12K (25123) - TotalRawHdfsReadTime(*): 1s058ms - TotalReadThroughput: 33.04 MB/sec Instance 7c4dade34ea955d6:dc885f5700000006 (host=vb0218.halxg.cloudera.com:22000):(Total: 47s400ms, non-child: 37s391ms, % non-child: 78.89%) Hdfs split stats (:<# splits>/): 8:1/253.38 MB 1:1/253.39 MB 9:1/253.38 MB 10:1/253.40 MB 5:2/506.79 MB 2:5/1.24 GB 7:2/506.80 MB 6:2/506.81 MB 3:3/760.16 MB 0:1/253.39 MB MemoryUsage(1s000ms): 616.72 MB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.06 GB, 1.02 GB, 976.67 MB, 941.01 MB, 923.36 MB, 905.83 MB, 895.35 MB, 894.58 MB, 853.14 MB, 823.87 MB, 782.63 MB, 741.97 MB, 709.53 MB, 695.84 MB, 682.11 MB, 651.96 MB, 630.73 MB, 676.70 MB, 629.47 MB, 671.16 MB, 636.51 MB, 596.51 MB, 560.99 MB, 522.16 MB, 497.31 MB, 476.73 MB, 441.98 MB, 605.39 MB, 683.73 MB, 709.84 MB, 683.35 MB, 648.97 MB, 612.86 MB, 540.38 MB, 519.18 MB, 503.41 MB, 463.88 MB, 427.78 MB, 402.98 MB, 349.16 MB, 272.82 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 11, 10, 9, 8, 8, 8, 8, 8, 8, 8, 7, 7 - AverageThreadTokens: 11.78 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1185292325) - PerHostPeakMemUsage: 1.10 GB (1185292325) - RowsProduced: 27.36M (27356338) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 38s367ms - TotalStorageWaitTime: 551.254ms - TotalThreadsInvoluntaryContextSwitches: 16.79K (16791) - TotalThreadsTotalWallClockTime: 9m16s - TotalThreadsSysTime: 2s441ms - TotalThreadsUserTime: 29s938ms - TotalThreadsVoluntaryContextSwitches: 52.12K (52115) Fragment Instance Lifecycle Timings: - ExecTime: 47s340ms - ExecTreeExecTime: 402.789ms - OpenTime: 33.422ms - ExecTreeOpenTime: 46.518us - PrepareTime: 26.196ms - ExecTreePrepareTime: 571.166us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 9s617ms, non-child: 9s617ms, % non-child: 100.00%) - BytesSent: 559.15 MB (586311676) - NetworkThroughput(*): 281.83 MB/sec - OverallThroughput: 58.14 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 27.36M (27356338) - SerializeBatchTime: 7s623ms - TransmitDataRPCTime: 1s984ms - UncompressedRowBatchSize: 965.40 MB (1012291378) CodeGen:(Total: 57.169ms, non-child: 57.169ms, % non-child: 100.00%) - CodegenTime: 1.288ms - CompileTime: 9.910ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.794ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 23.848ms HDFS_SCAN_NODE (id=1):(Total: 334.177ms, non-child: 334.177ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 8:1/253.38 MB 1:1/253.39 MB 9:1/253.38 MB 10:1/253.40 MB 5:2/506.79 MB 2:5/1.24 GB 7:2/506.80 MB 6:2/506.81 MB 3:3/760.16 MB 0:1/253.39 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:98.95% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:1.053% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 517.76 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1.06 GB, 1.06 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.35 GB, 1.47 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB - FooterProcessingTime: (Avg: 3.081ms ; Min: 2.236ms ; Max: 5.685ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.08 - AverageScannerThreadConcurrency: 10.78 - BytesRead: 1.55 GB (1664143743) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.55 GB (1664143743) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.55 GB (1664143743) - DecompressionTime: 9s554ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 10 (10) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1185045026) - PerReadThreadRawHdfsThroughput: 1.33 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 28.420ms - RowBatchQueuePutWaitTime: 8m6s - RowsRead: 131.50M (131502589) - RowsReturned: 27.36M (27356338) - RowsReturnedRate: 81.86 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 12.53K (12534) - ScannerThreadsTotalWallClockTime: 8m28s - MaterializeTupleTime(*): 11s639ms - ScannerThreadsSysTime: 1s341ms - ScannerThreadsUserTime: 21s278ms - ScannerThreadsVoluntaryContextSwitches: 26.63K (26632) - TotalRawHdfsReadTime(*): 1s169ms - TotalReadThroughput: 33.41 MB/sec Instance 7c4dade34ea955d6:dc885f5700000005 (host=vb0204.halxg.cloudera.com:22000):(Total: 47s309ms, non-child: 37s653ms, % non-child: 79.59%) Hdfs split stats (:<# splits>/): 5:1/253.39 MB 2:1/253.41 MB 7:2/506.80 MB 6:1/253.41 MB 1:2/506.79 MB 8:2/506.78 MB 3:5/1.24 GB 4:4/1013.63 MB 0:1/253.40 MB MemoryUsage(1s000ms): 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.08 GB, 1.07 GB, 1.02 GB, 986.43 MB, 946.02 MB, 924.57 MB, 908.47 MB, 907.26 MB, 863.42 MB, 848.00 MB, 788.27 MB, 760.36 MB, 718.89 MB, 698.04 MB, 652.16 MB, 638.54 MB, 700.25 MB, 687.60 MB, 666.82 MB, 646.15 MB, 624.58 MB, 582.38 MB, 553.25 MB, 522.89 MB, 503.16 MB, 742.92 MB, 775.88 MB, 770.61 MB, 771.14 MB, 755.02 MB, 696.55 MB, 613.02 MB, 542.07 MB, 525.77 MB, 522.57 MB, 494.86 MB, 466.23 MB, 430.99 MB, 398.88 MB, 386.76 MB, 339.43 MB, 302.44 MB, 188.92 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 11, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 6 - AverageThreadTokens: 11.65 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1187426453) - PerHostPeakMemUsage: 1.11 GB (1187442837) - RowsProduced: 27.05M (27045946) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 38s622ms - TotalStorageWaitTime: 545.506ms - TotalThreadsInvoluntaryContextSwitches: 16.50K (16496) - TotalThreadsTotalWallClockTime: 9m9s - TotalThreadsSysTime: 2s112ms - TotalThreadsUserTime: 29s491ms - TotalThreadsVoluntaryContextSwitches: 51.47K (51469) Fragment Instance Lifecycle Timings: - ExecTime: 47s251ms - ExecTreeExecTime: 405.654ms - OpenTime: 33.417ms - ExecTreeOpenTime: 47.894us - PrepareTime: 24.502ms - ExecTreePrepareTime: 302.082us DataStreamSender (dst_id=4):(Total: 9s259ms, non-child: 9s259ms, % non-child: 100.00%) - BytesSent: 552.79 MB (579639503) - NetworkThroughput(*): 289.61 MB/sec - OverallThroughput: 59.70 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 27.05M (27045946) - SerializeBatchTime: 7s341ms - TransmitDataRPCTime: 1s908ms - UncompressedRowBatchSize: 954.44 MB (1000805662) CodeGen:(Total: 55.660ms, non-child: 55.660ms, % non-child: 100.00%) - CodegenTime: 1.441ms - CompileTime: 9.725ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.852ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 22.512ms HDFS_SCAN_NODE (id=1):(Total: 341.400ms, non-child: 341.400ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:1/253.39 MB 2:1/253.41 MB 7:2/506.80 MB 6:1/253.41 MB 1:2/506.79 MB 8:2/506.78 MB 3:5/1.24 GB 4:4/1013.63 MB 0:1/253.40 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:98.94% 1:1.064% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1002.38 MB, 1.06 GB, 1.10 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.18 GB, 1.49 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB - FooterProcessingTime: (Avg: 4.125ms ; Min: 2.646ms ; Max: 6.854ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.01 - AverageScannerThreadConcurrency: 10.65 - BytesRead: 1.55 GB (1664181733) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.55 GB (1664181733) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.55 GB (1664181733) - DecompressionTime: 9s373ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1187181194) - PerReadThreadRawHdfsThroughput: 1.45 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 27.904ms - RowBatchQueuePutWaitTime: 8m - RowsRead: 131.50M (131502331) - RowsReturned: 27.05M (27045946) - RowsReturnedRate: 79.22 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 12.24K (12243) - ScannerThreadsTotalWallClockTime: 8m22s - MaterializeTupleTime(*): 11s337ms - ScannerThreadsSysTime: 1s178ms - ScannerThreadsUserTime: 20s994ms - ScannerThreadsVoluntaryContextSwitches: 26.43K (26430) - TotalRawHdfsReadTime(*): 1s071ms - TotalReadThroughput: 33.77 MB/sec Instance 7c4dade34ea955d6:dc885f5700000002 (host=vb0216.halxg.cloudera.com:22000):(Total: 47s198ms, non-child: 37s682ms, % non-child: 79.84%) Hdfs split stats (:<# splits>/): 1:1/253.38 MB 3:2/506.78 MB 10:1/253.39 MB 7:2/506.78 MB 9:5/1.24 GB 6:1/253.41 MB 5:2/506.78 MB 2:1/253.41 MB 0:4/1013.59 MB MemoryUsage(1s000ms): 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.08 GB, 1.02 GB, 997.63 MB, 953.03 MB, 930.97 MB, 913.15 MB, 902.12 MB, 862.23 MB, 847.02 MB, 815.92 MB, 778.61 MB, 739.69 MB, 718.87 MB, 680.30 MB, 643.55 MB, 696.97 MB, 652.41 MB, 625.47 MB, 609.43 MB, 561.54 MB, 590.95 MB, 645.98 MB, 622.24 MB, 606.39 MB, 714.71 MB, 742.11 MB, 727.41 MB, 730.17 MB, 726.71 MB, 695.71 MB, 646.87 MB, 606.71 MB, 552.15 MB, 508.25 MB, 494.33 MB, 460.85 MB, 437.37 MB, 381.65 MB, 362.63 MB, 317.90 MB, 274.08 MB, 161.62 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 11, 10, 10, 8, 8, 8, 8, 8, 8, 8, 8, 7, 5 - AverageThreadTokens: 11.68 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1188127745) - PerHostPeakMemUsage: 1.11 GB (1188127745) - RowsProduced: 26.59M (26594701) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 38s590ms - TotalStorageWaitTime: 603.314ms - TotalThreadsInvoluntaryContextSwitches: 13.91K (13915) - TotalThreadsTotalWallClockTime: 9m9s - TotalThreadsSysTime: 2s100ms - TotalThreadsUserTime: 29s466ms - TotalThreadsVoluntaryContextSwitches: 50.99K (50989) Fragment Instance Lifecycle Timings: - ExecTime: 47s139ms - ExecTreeExecTime: 357.305ms - OpenTime: 33.823ms - ExecTreeOpenTime: 50.981us - PrepareTime: 24.305ms - ExecTreePrepareTime: 319.284us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 9s171ms, non-child: 9s171ms, % non-child: 100.00%) - BytesSent: 543.60 MB (570003508) - NetworkThroughput(*): 294.96 MB/sec - OverallThroughput: 59.27 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 26.59M (26594701) - SerializeBatchTime: 7s318ms - TransmitDataRPCTime: 1s842ms - UncompressedRowBatchSize: 938.52 MB (984107841) CodeGen:(Total: 55.567ms, non-child: 55.567ms, % non-child: 100.00%) - CodegenTime: 1.487ms - CompileTime: 10.260ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.791ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 21.933ms HDFS_SCAN_NODE (id=1):(Total: 288.599ms, non-child: 288.599ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:1/253.38 MB 3:2/506.78 MB 10:1/253.39 MB 7:2/506.78 MB 9:5/1.24 GB 6:1/253.41 MB 5:2/506.78 MB 2:1/253.41 MB 0:4/1013.59 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.14 GB, 1.22 GB, 1.22 GB, 1.26 GB, 1.43 GB, 1.47 GB, 1.47 GB, 1.51 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB - FooterProcessingTime: (Avg: 4.069ms ; Min: 2.822ms ; Max: 6.522ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.68 - BytesRead: 1.55 GB (1664130351) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.55 GB (1664130351) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.55 GB (1664130351) - DecompressionTime: 9s478ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1187623528) - PerReadThreadRawHdfsThroughput: 1.39 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 32.143ms - RowBatchQueuePutWaitTime: 7m59s - RowsRead: 131.50M (131502389) - RowsReturned: 26.59M (26594701) - RowsReturnedRate: 92.15 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 9.68K (9684) - ScannerThreadsTotalWallClockTime: 8m22s - MaterializeTupleTime(*): 11s331ms - ScannerThreadsSysTime: 1s270ms - ScannerThreadsUserTime: 20s953ms - ScannerThreadsVoluntaryContextSwitches: 26.01K (26014) - TotalRawHdfsReadTime(*): 1s116ms - TotalReadThroughput: 33.77 MB/sec Instance 7c4dade34ea955d6:dc885f5700000009 (host=vb0224.halxg.cloudera.com:22000):(Total: 46s459ms, non-child: 37s232ms, % non-child: 80.14%) Hdfs split stats (:<# splits>/): 3:1/253.39 MB 1:1/253.41 MB 10:1/253.38 MB 9:1/253.40 MB 5:2/506.78 MB 0:2/506.77 MB 4:2/506.76 MB 8:2/506.78 MB 7:3/760.18 MB 6:2/481.03 MB 2:2/506.80 MB MemoryUsage(1s000ms): 1.09 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.06 GB, 1.00 GB, 978.05 MB, 946.53 MB, 915.68 MB, 902.55 MB, 891.56 MB, 850.36 MB, 831.50 MB, 789.18 MB, 761.31 MB, 727.33 MB, 699.39 MB, 665.12 MB, 624.85 MB, 640.81 MB, 635.36 MB, 593.89 MB, 632.96 MB, 595.92 MB, 562.84 MB, 654.97 MB, 652.02 MB, 645.95 MB, 769.96 MB, 767.22 MB, 736.43 MB, 714.15 MB, 675.53 MB, 650.72 MB, 614.00 MB, 539.23 MB, 494.65 MB, 475.26 MB, 446.91 MB, 395.05 MB, 355.70 MB, 285.33 MB, 237.89 MB, 209.77 MB, 113.01 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 12, 11, 11, 10, 10, 9, 8, 8, 8, 8, 8, 7, 6, 6, 3 - AverageThreadTokens: 11.37 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1179367335) - PerHostPeakMemUsage: 1.10 GB (1179367335) - RowsProduced: 25.26M (25256825) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 38s235ms - TotalStorageWaitTime: 582.423ms - TotalThreadsInvoluntaryContextSwitches: 14.22K (14225) - TotalThreadsTotalWallClockTime: 8m50s - TotalThreadsSysTime: 2s157ms - TotalThreadsUserTime: 28s516ms - TotalThreadsVoluntaryContextSwitches: 48.60K (48597) Fragment Instance Lifecycle Timings: - ExecTime: 46s401ms - ExecTreeExecTime: 349.755ms - OpenTime: 33.840ms - ExecTreeOpenTime: 51.607us - PrepareTime: 23.867ms - ExecTreePrepareTime: 452.405us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s884ms, non-child: 8s884ms, % non-child: 100.00%) - BytesSent: 516.24 MB (541321888) - NetworkThroughput(*): 269.33 MB/sec - OverallThroughput: 58.10 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 25.26M (25256825) - SerializeBatchTime: 6s959ms - TransmitDataRPCTime: 1s916ms - UncompressedRowBatchSize: 891.31 MB (934601201) CodeGen:(Total: 54.905ms, non-child: 54.905ms, % non-child: 100.00%) - CodegenTime: 1.441ms - CompileTime: 10.167ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.881ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 21.274ms HDFS_SCAN_NODE (id=1):(Total: 286.735ms, non-child: 286.735ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:1/253.39 MB 1:1/253.41 MB 10:1/253.38 MB 9:1/253.40 MB 5:2/506.78 MB 0:2/506.77 MB 4:2/506.76 MB 8:2/506.78 MB 7:3/760.18 MB 6:2/481.03 MB 2:2/506.80 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 993.81 MB, 1.01 GB, 1.05 GB, 1.05 GB, 1.13 GB, 1.13 GB, 1.13 GB, 1.26 GB, 1.30 GB, 1.34 GB, 1.50 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB - FooterProcessingTime: (Avg: 3.904ms ; Min: 2.864ms ; Max: 6.868ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.37 - BytesRead: 1.54 GB (1655172689) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.54 GB (1655172689) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.54 GB (1655172689) - DecompressionTime: 9s334ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 11 (11) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1179122076) - PerReadThreadRawHdfsThroughput: 1.25 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 31.420ms - RowBatchQueuePutWaitTime: 7m42s - RowsRead: 130.80M (130797981) - RowsReturned: 25.26M (25256825) - RowsReturnedRate: 88.08 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.07K (10072) - ScannerThreadsTotalWallClockTime: 8m4s - MaterializeTupleTime(*): 11s185ms - ScannerThreadsSysTime: 1s298ms - ScannerThreadsUserTime: 20s606ms - ScannerThreadsVoluntaryContextSwitches: 24.79K (24792) - TotalRawHdfsReadTime(*): 1s231ms - TotalReadThroughput: 33.95 MB/sec Instance 7c4dade34ea955d6:dc885f5700000001 (host=vb0230.halxg.cloudera.com:22000):(Total: 46s371ms, non-child: 37s462ms, % non-child: 80.79%) Hdfs split stats (:<# splits>/): 9:1/253.41 MB 2:1/253.39 MB 5:2/506.78 MB 3:2/506.81 MB 6:1/253.38 MB 7:2/506.78 MB 4:2/506.81 MB 1:3/760.19 MB 0:5/1.24 GB MemoryUsage(1s000ms): 1.10 GB, 1.10 GB, 1.11 GB, 1.11 GB, 1.08 GB, 1.04 GB, 1.02 GB, 1003.37 MB, 948.06 MB, 906.75 MB, 882.50 MB, 873.88 MB, 842.46 MB, 828.82 MB, 799.53 MB, 751.39 MB, 698.20 MB, 666.30 MB, 618.69 MB, 600.13 MB, 678.50 MB, 677.99 MB, 733.19 MB, 713.69 MB, 677.54 MB, 799.34 MB, 758.64 MB, 712.93 MB, 708.05 MB, 726.57 MB, 711.73 MB, 682.76 MB, 648.27 MB, 655.16 MB, 670.92 MB, 602.77 MB, 566.06 MB, 490.70 MB, 461.09 MB, 416.95 MB, 380.99 MB, 329.73 MB, 307.97 MB, 223.57 MB, 171.91 MB, 97.29 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 11, 10, 8, 8, 8, 8, 7, 7, 5, 4, 3 - AverageThreadTokens: 11.65 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1191990227) - PerHostPeakMemUsage: 1.11 GB (1191990227) - RowsProduced: 25.08M (25076265) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 38s279ms - TotalStorageWaitTime: 625.907ms - TotalThreadsInvoluntaryContextSwitches: 14.46K (14462) - TotalThreadsTotalWallClockTime: 8m57s - TotalThreadsSysTime: 2s138ms - TotalThreadsUserTime: 28s592ms - TotalThreadsVoluntaryContextSwitches: 48.13K (48128) Fragment Instance Lifecycle Timings: - ExecTime: 46s314ms - ExecTreeExecTime: 343.578ms - OpenTime: 33.652ms - ExecTreeOpenTime: 46.234us - PrepareTime: 23.391ms - ExecTreePrepareTime: 277.818us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s573ms, non-child: 8s573ms, % non-child: 100.00%) - BytesSent: 512.53 MB (537431466) - NetworkThroughput(*): 292.42 MB/sec - OverallThroughput: 59.78 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 25.08M (25076265) - SerializeBatchTime: 6s812ms - TransmitDataRPCTime: 1s752ms - UncompressedRowBatchSize: 884.93 MB (927919769) CodeGen:(Total: 54.648ms, non-child: 54.648ms, % non-child: 100.00%) - CodegenTime: 1.367ms - CompileTime: 10.150ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.655ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 21.288ms HDFS_SCAN_NODE (id=1):(Total: 280.514ms, non-child: 280.514ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 9:1/253.41 MB 2:1/253.39 MB 5:2/506.78 MB 3:2/506.81 MB 6:1/253.38 MB 7:2/506.78 MB 4:2/506.81 MB 1:3/760.19 MB 0:5/1.24 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:97.83% 1:2.174% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1020.46 MB, 1.10 GB, 1.14 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.39 GB, 1.39 GB, 1.39 GB, 1.43 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.51 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB - FooterProcessingTime: (Avg: 4.573ms ; Min: 2.919ms ; Max: 7.019ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 10.65 - BytesRead: 1.55 GB (1664143238) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.55 GB (1664143238) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.55 GB (1664143238) - DecompressionTime: 9s395ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1191476672) - PerReadThreadRawHdfsThroughput: 1.40 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 33.607ms - RowBatchQueuePutWaitTime: 7m48s - RowsRead: 131.50M (131502184) - RowsReturned: 25.08M (25076265) - RowsReturnedRate: 89.39 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.33K (10327) - ScannerThreadsTotalWallClockTime: 8m10s - MaterializeTupleTime(*): 11s248ms - ScannerThreadsSysTime: 1s250ms - ScannerThreadsUserTime: 20s707ms - ScannerThreadsVoluntaryContextSwitches: 24.52K (24522) - TotalRawHdfsReadTime(*): 1s109ms - TotalReadThroughput: 34.50 MB/sec Instance 7c4dade34ea955d6:dc885f5700000008 (host=vb0232.halxg.cloudera.com:22000):(Total: 46s195ms, non-child: 36s885ms, % non-child: 79.85%) Hdfs split stats (:<# splits>/): 6:2/506.80 MB 9:3/760.17 MB 7:1/253.39 MB 8:5/1.24 GB 1:2/260.77 MB 5:3/760.18 MB 4:1/253.40 MB 2:1/253.40 MB 0:1/253.38 MB MemoryUsage(1s000ms): 1.06 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.07 GB, 1.01 GB, 998.97 MB, 970.64 MB, 947.17 MB, 922.79 MB, 885.83 MB, 853.13 MB, 839.40 MB, 793.36 MB, 779.38 MB, 748.15 MB, 713.64 MB, 669.28 MB, 631.90 MB, 618.39 MB, 568.71 MB, 552.67 MB, 640.26 MB, 602.57 MB, 656.98 MB, 637.00 MB, 676.62 MB, 667.86 MB, 739.29 MB, 718.40 MB, 687.65 MB, 656.52 MB, 631.33 MB, 586.53 MB, 505.45 MB, 476.64 MB, 465.37 MB, 407.94 MB, 368.16 MB, 338.67 MB, 292.94 MB, 270.89 MB, 206.58 MB, 161.92 MB, 119.95 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 12, 11, 11, 10, 8, 8, 8, 7, 7, 7, 6, 6, 5, 5, 4 - AverageThreadTokens: 11.23 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1187849367) - PerHostPeakMemUsage: 1.11 GB (1187849367) - RowsProduced: 24.90M (24904227) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 37s787ms - TotalStorageWaitTime: 618.610ms - TotalThreadsInvoluntaryContextSwitches: 14.35K (14351) - TotalThreadsTotalWallClockTime: 8m37s - TotalThreadsSysTime: 2s218ms - TotalThreadsUserTime: 28s381ms - TotalThreadsVoluntaryContextSwitches: 47.76K (47758) Fragment Instance Lifecycle Timings: - ExecTime: 46s138ms - ExecTreeExecTime: 395.746ms - OpenTime: 33.449ms - ExecTreeOpenTime: 42.458us - PrepareTime: 23.597ms - ExecTreePrepareTime: 298.660us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s925ms, non-child: 8s925ms, % non-child: 100.00%) - BytesSent: 509.00 MB (533725492) - NetworkThroughput(*): 273.97 MB/sec - OverallThroughput: 57.03 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.90M (24904227) - SerializeBatchTime: 7s057ms - TransmitDataRPCTime: 1s857ms - UncompressedRowBatchSize: 878.86 MB (921553699) CodeGen:(Total: 54.553ms, non-child: 54.553ms, % non-child: 100.00%) - CodegenTime: 1.424ms - CompileTime: 9.656ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 23.013ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 21.327ms HDFS_SCAN_NODE (id=1):(Total: 330.432ms, non-child: 330.432ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:2/506.80 MB 9:3/760.17 MB 7:1/253.39 MB 8:5/1.24 GB 1:2/260.77 MB 5:3/760.18 MB 4:1/253.40 MB 2:1/253.40 MB 0:1/253.38 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 963.23 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1.02 GB, 1.14 GB, 1.14 GB, 1.23 GB, 1.23 GB, 1.31 GB, 1.35 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 4.142ms ; Min: 2.278ms ; Max: 7.866ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.23 - BytesRead: 1.47 GB (1579317980) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1579317980) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1579317980) - DecompressionTime: 9s115ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1187667612) - PerReadThreadRawHdfsThroughput: 1.37 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 33.115ms - RowBatchQueuePutWaitTime: 7m29s - RowsRead: 124.77M (124770123) - RowsReturned: 24.90M (24904227) - RowsReturnedRate: 75.37 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.12K (10123) - ScannerThreadsTotalWallClockTime: 7m51s - MaterializeTupleTime(*): 11s069ms - ScannerThreadsSysTime: 1s284ms - ScannerThreadsUserTime: 20s252ms - ScannerThreadsVoluntaryContextSwitches: 24.43K (24426) - TotalRawHdfsReadTime(*): 1s072ms - TotalReadThroughput: 32.74 MB/sec Instance 7c4dade34ea955d6:dc885f570000000b (host=vb0222.halxg.cloudera.com:22000):(Total: 45s656ms, non-child: 36s966ms, % non-child: 80.97%) Hdfs split stats (:<# splits>/): 6:1/253.38 MB 3:3/514.23 MB 10:1/253.41 MB 4:3/760.21 MB 9:1/253.37 MB 8:2/506.80 MB 7:2/506.80 MB 5:2/506.79 MB 1:1/253.40 MB 2:3/760.16 MB MemoryUsage(1s000ms): 686.51 MB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.06 GB, 1.04 GB, 1020.89 MB, 964.77 MB, 917.24 MB, 889.19 MB, 883.90 MB, 869.23 MB, 839.01 MB, 808.90 MB, 765.15 MB, 708.49 MB, 665.84 MB, 644.52 MB, 613.45 MB, 666.58 MB, 751.74 MB, 746.91 MB, 735.90 MB, 714.48 MB, 706.87 MB, 715.63 MB, 665.87 MB, 631.33 MB, 584.95 MB, 592.92 MB, 624.73 MB, 647.35 MB, 661.47 MB, 618.61 MB, 512.96 MB, 477.13 MB, 428.38 MB, 407.95 MB, 368.70 MB, 349.29 MB, 301.82 MB, 230.78 MB, 206.93 MB, 145.69 MB, 35.83 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 8, 7, 7, 7, 7, 7, 7, 6, 6, 4, 1 - AverageThreadTokens: 11.37 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1190790125) - PerHostPeakMemUsage: 1.11 GB (1190790125) - RowsProduced: 24.52M (24523519) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 37s660ms - TotalStorageWaitTime: 581.774ms - TotalThreadsInvoluntaryContextSwitches: 15.26K (15264) - TotalThreadsTotalWallClockTime: 8m40s - TotalThreadsSysTime: 2s078ms - TotalThreadsUserTime: 27s904ms - TotalThreadsVoluntaryContextSwitches: 46.99K (46986) Fragment Instance Lifecycle Timings: - ExecTime: 45s600ms - ExecTreeExecTime: 349.047ms - OpenTime: 32.504ms - ExecTreeOpenTime: 39.856us - PrepareTime: 23.434ms - ExecTreePrepareTime: 304.676us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s351ms, non-child: 8s351ms, % non-child: 100.00%) - BytesSent: 501.25 MB (525595582) - NetworkThroughput(*): 306.02 MB/sec - OverallThroughput: 60.02 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.52M (24523519) - SerializeBatchTime: 6s705ms - TransmitDataRPCTime: 1s637ms - UncompressedRowBatchSize: 865.43 MB (907466011) CodeGen:(Total: 53.443ms, non-child: 53.443ms, % non-child: 100.00%) - CodegenTime: 1.427ms - CompileTime: 9.559ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.183ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 21.193ms HDFS_SCAN_NODE (id=1):(Total: 284.952ms, non-child: 284.952ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:1/253.38 MB 3:3/514.23 MB 10:1/253.41 MB 4:3/760.21 MB 9:1/253.37 MB 8:2/506.80 MB 7:2/506.80 MB 5:2/506.79 MB 1:1/253.40 MB 2:3/760.16 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:98.91% 1:0% 2:0% 3:0% 4:1.087% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 602.01 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1.06 GB, 1.19 GB, 1.23 GB, 1.23 GB, 1.23 GB, 1.27 GB, 1.31 GB, 1.31 GB, 1.31 GB, 1.31 GB, 1.35 GB, 1.39 GB, 1.43 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 5.423ms ; Min: 2.837ms ; Max: 21.655ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.05 - AverageScannerThreadConcurrency: 10.37 - BytesRead: 1.47 GB (1579318713) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1579318713) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1579318713) - DecompressionTime: 9s022ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 10 (10) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1190511164) - PerReadThreadRawHdfsThroughput: 1.38 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 27.956ms - RowBatchQueuePutWaitTime: 7m33s - RowsRead: 124.77M (124773003) - RowsReturned: 24.52M (24523519) - RowsReturnedRate: 86.06 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 11.21K (11212) - ScannerThreadsTotalWallClockTime: 7m55s - MaterializeTupleTime(*): 10s868ms - ScannerThreadsSysTime: 1s221ms - ScannerThreadsUserTime: 19s999ms - ScannerThreadsVoluntaryContextSwitches: 24.02K (24025) - TotalRawHdfsReadTime(*): 1s065ms - TotalReadThroughput: 32.74 MB/sec Instance 7c4dade34ea955d6:dc885f570000000c (host=vb0220.halxg.cloudera.com:22000):(Total: 45s637ms, non-child: 37s006ms, % non-child: 81.09%) Hdfs split stats (:<# splits>/): 8:2/506.79 MB 5:2/506.80 MB 10:1/253.39 MB 9:2/506.77 MB 7:1/253.41 MB 2:3/732.64 MB 3:4/1013.58 MB 0:3/760.18 MB 1:1/253.41 MB MemoryUsage(1s000ms): 1.10 GB, 1.11 GB, 1.10 GB, 1.11 GB, 1.09 GB, 1.03 GB, 1013.29 MB, 990.45 MB, 957.86 MB, 920.64 MB, 892.88 MB, 858.27 MB, 819.11 MB, 805.25 MB, 761.17 MB, 729.86 MB, 695.76 MB, 643.03 MB, 649.74 MB, 660.88 MB, 683.62 MB, 648.74 MB, 657.42 MB, 673.26 MB, 669.87 MB, 760.55 MB, 721.31 MB, 757.36 MB, 755.07 MB, 749.62 MB, 723.98 MB, 708.65 MB, 689.32 MB, 659.21 MB, 606.58 MB, 550.49 MB, 509.83 MB, 462.88 MB, 429.41 MB, 377.34 MB, 335.76 MB, 289.03 MB, 264.54 MB, 195.16 MB, 104.52 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 12, 12, 12, 12, 12, 10, 9, 8, 8, 8, 8, 7, 7, 5, 3 - AverageThreadTokens: 11.51 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1193023678) - PerHostPeakMemUsage: 1.11 GB (1193023678) - RowsProduced: 24.28M (24277670) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 37s716ms - TotalStorageWaitTime: 622.149ms - TotalThreadsInvoluntaryContextSwitches: 14.84K (14842) - TotalThreadsTotalWallClockTime: 8m44s - TotalThreadsSysTime: 2s192ms - TotalThreadsUserTime: 28s352ms - TotalThreadsVoluntaryContextSwitches: 46.60K (46603) Fragment Instance Lifecycle Timings: - ExecTime: 45s579ms - ExecTreeExecTime: 354.434ms - OpenTime: 33.497ms - ExecTreeOpenTime: 52.453us - PrepareTime: 24.348ms - ExecTreePrepareTime: 301.151us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s284ms, non-child: 8s284ms, % non-child: 100.00%) - BytesSent: 496.22 MB (520319489) - NetworkThroughput(*): 304.19 MB/sec - OverallThroughput: 59.89 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.28M (24277670) - SerializeBatchTime: 6s644ms - TransmitDataRPCTime: 1s631ms - UncompressedRowBatchSize: 856.75 MB (898368646) CodeGen:(Total: 55.310ms, non-child: 55.310ms, % non-child: 100.00%) - CodegenTime: 1.496ms - CompileTime: 10.104ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.611ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 22.042ms HDFS_SCAN_NODE (id=1):(Total: 290.394ms, non-child: 290.394ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 8:2/506.79 MB 5:2/506.80 MB 10:1/253.39 MB 9:2/506.77 MB 7:1/253.41 MB 2:3/732.64 MB 3:4/1013.58 MB 0:3/760.18 MB 1:1/253.41 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:98.9% 1:1.099% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1.04 GB, 1.10 GB, 1.14 GB, 1.14 GB, 1.18 GB, 1.22 GB, 1.26 GB, 1.38 GB, 1.38 GB, 1.46 GB, 1.50 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB - FooterProcessingTime: (Avg: 3.667ms ; Min: 2.766ms ; Max: 6.151ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.01 - AverageScannerThreadConcurrency: 10.51 - BytesRead: 1.54 GB (1654662021) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.54 GB (1654662021) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.54 GB (1654662021) - DecompressionTime: 9s388ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1192812211) - PerReadThreadRawHdfsThroughput: 1.39 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 31.643ms - RowBatchQueuePutWaitTime: 7m37s - RowsRead: 130.75M (130748945) - RowsReturned: 24.28M (24277670) - RowsReturnedRate: 83.60 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.78K (10782) - ScannerThreadsTotalWallClockTime: 7m59s - MaterializeTupleTime(*): 11s194ms - ScannerThreadsSysTime: 1s276ms - ScannerThreadsUserTime: 20s596ms - ScannerThreadsVoluntaryContextSwitches: 23.84K (23839) - TotalRawHdfsReadTime(*): 1s110ms - TotalReadThroughput: 34.68 MB/sec Instance 7c4dade34ea955d6:dc885f570000000e (host=vb0206.halxg.cloudera.com:22000):(Total: 45s547ms, non-child: 36s689ms, % non-child: 80.55%) Hdfs split stats (:<# splits>/): 2:2/506.78 MB 9:3/760.18 MB 6:2/506.81 MB 7:1/253.39 MB 5:2/506.80 MB 4:3/760.17 MB 1:2/506.77 MB 0:3/760.20 MB MemoryUsage(1s000ms): 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.08 GB, 1.05 GB, 999.75 MB, 938.63 MB, 924.99 MB, 912.33 MB, 906.61 MB, 899.68 MB, 874.53 MB, 837.97 MB, 779.90 MB, 755.80 MB, 709.28 MB, 689.44 MB, 664.34 MB, 648.80 MB, 710.36 MB, 650.05 MB, 653.20 MB, 673.21 MB, 642.01 MB, 586.85 MB, 550.33 MB, 524.80 MB, 491.00 MB, 513.11 MB, 645.41 MB, 639.78 MB, 653.73 MB, 672.10 MB, 563.42 MB, 498.71 MB, 458.24 MB, 442.93 MB, 404.89 MB, 363.29 MB, 316.74 MB, 268.85 MB, 230.86 MB, 115.32 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 8, 7, 7, 7, 7, 7, 7, 7, 7, 3 - AverageThreadTokens: 11.51 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1187857226) - PerHostPeakMemUsage: 1.11 GB (1187857226) - RowsProduced: 24.21M (24210513) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 37s504ms - TotalStorageWaitTime: 580.516ms - TotalThreadsInvoluntaryContextSwitches: 15.23K (15232) - TotalThreadsTotalWallClockTime: 8m45s - TotalThreadsSysTime: 2s210ms - TotalThreadsUserTime: 27s691ms - TotalThreadsVoluntaryContextSwitches: 46.46K (46464) Fragment Instance Lifecycle Timings: - ExecTime: 45s488ms - ExecTreeExecTime: 366.130ms - OpenTime: 33.791ms - ExecTreeOpenTime: 49.614us - PrepareTime: 24.439ms - ExecTreePrepareTime: 294.205us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s496ms, non-child: 8s496ms, % non-child: 100.00%) - BytesSent: 494.84 MB (518880607) - NetworkThroughput(*): 277.17 MB/sec - OverallThroughput: 58.24 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.21M (24210513) - SerializeBatchTime: 6s702ms - TransmitDataRPCTime: 1s785ms - UncompressedRowBatchSize: 854.38 MB (895883569) CodeGen:(Total: 55.891ms, non-child: 55.891ms, % non-child: 100.00%) - CodegenTime: 1.361ms - CompileTime: 10.164ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.770ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 22.247ms HDFS_SCAN_NODE (id=1):(Total: 305.594ms, non-child: 305.594ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:2/506.78 MB 9:3/760.18 MB 6:2/506.81 MB 7:1/253.39 MB 5:2/506.80 MB 4:3/760.17 MB 1:2/506.77 MB 0:3/760.20 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 18 out of 18 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:72 BytesRead(1s000ms): 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1.06 GB, 1.06 GB, 1.10 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.22 GB, 1.39 GB, 1.39 GB, 1.43 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 3.978ms ; Min: 2.588ms ; Max: 5.531ms ; Number of samples: 18) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.51 - BytesRead: 1.47 GB (1576537765) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1576537765) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1576537765) - DecompressionTime: 9s037ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 8 (8) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1187578175) - PerReadThreadRawHdfsThroughput: 1.45 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 33.013ms - RowBatchQueuePutWaitTime: 7m38s - RowsRead: 124.58M (124580613) - RowsReturned: 24.21M (24210513) - RowsReturnedRate: 79.22 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 11.12K (11118) - ScannerThreadsTotalWallClockTime: 8m - MaterializeTupleTime(*): 10s952ms - ScannerThreadsSysTime: 1s290ms - ScannerThreadsUserTime: 19s928ms - ScannerThreadsVoluntaryContextSwitches: 23.71K (23709) - TotalRawHdfsReadTime(*): 1s015ms - TotalReadThroughput: 33.04 MB/sec Instance 7c4dade34ea955d6:dc885f570000000a (host=vb0228.halxg.cloudera.com:22000):(Total: 45s535ms, non-child: 36s636ms, % non-child: 80.46%) Hdfs split stats (:<# splits>/): 6:1/253.41 MB 4:2/506.76 MB 2:1/253.42 MB 10:2/445.01 MB 8:3/760.18 MB 3:1/187.22 MB 7:2/506.79 MB 5:4/1013.55 MB 0:3/760.18 MB MemoryUsage(1s000ms): 542.70 MB, 1.06 GB, 1.07 GB, 1.06 GB, 1.05 GB, 1.02 GB, 1017.78 MB, 962.08 MB, 896.72 MB, 883.33 MB, 873.48 MB, 861.91 MB, 832.03 MB, 813.22 MB, 758.34 MB, 750.11 MB, 766.81 MB, 733.89 MB, 714.75 MB, 672.28 MB, 664.69 MB, 626.90 MB, 567.22 MB, 602.39 MB, 552.80 MB, 670.88 MB, 656.46 MB, 656.07 MB, 722.87 MB, 761.19 MB, 733.04 MB, 714.45 MB, 660.42 MB, 601.61 MB, 547.78 MB, 516.35 MB, 483.98 MB, 453.37 MB, 398.21 MB, 376.84 MB, 308.82 MB, 282.32 MB, 229.73 MB, 176.51 MB, 147.22 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 12, 11, 10, 9, 8, 8, 8, 8, 8, 6, 6, 5, 5, 5 - AverageThreadTokens: 11.27 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.07 GB (1147109520) - PerHostPeakMemUsage: 1.07 GB (1147109520) - RowsProduced: 24.20M (24199028) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 37s530ms - TotalStorageWaitTime: 543.778ms - TotalThreadsInvoluntaryContextSwitches: 14.89K (14889) - TotalThreadsTotalWallClockTime: 8m30s - TotalThreadsSysTime: 2s158ms - TotalThreadsUserTime: 28s129ms - TotalThreadsVoluntaryContextSwitches: 46.60K (46598) Fragment Instance Lifecycle Timings: - ExecTime: 45s477ms - ExecTreeExecTime: 355.862ms - OpenTime: 33.609ms - ExecTreeOpenTime: 49.293us - PrepareTime: 24.514ms - ExecTreePrepareTime: 347.157us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s551ms, non-child: 8s551ms, % non-child: 100.00%) - BytesSent: 494.62 MB (518648130) - NetworkThroughput(*): 273.82 MB/sec - OverallThroughput: 57.84 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.20M (24199028) - SerializeBatchTime: 6s736ms - TransmitDataRPCTime: 1s806ms - UncompressedRowBatchSize: 853.98 MB (895458584) CodeGen:(Total: 55.646ms, non-child: 55.646ms, % non-child: 100.00%) - CodegenTime: 1.454ms - CompileTime: 10.136ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.790ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 22.169ms HDFS_SCAN_NODE (id=1):(Total: 291.865ms, non-child: 291.865ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:1/253.41 MB 4:2/506.76 MB 2:1/253.42 MB 10:2/445.01 MB 8:3/760.18 MB 3:1/187.22 MB 7:2/506.79 MB 5:4/1013.55 MB 0:3/760.18 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 480.12 MB, 960.25 MB, 960.25 MB, 960.25 MB, 960.25 MB, 960.25 MB, 960.25 MB, 960.25 MB, 960.25 MB, 960.25 MB, 960.25 MB, 960.25 MB, 960.25 MB, 960.25 MB, 960.25 MB, 1002.01 MB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.10 GB, 1.10 GB, 1.26 GB, 1.26 GB, 1.30 GB, 1.43 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB - FooterProcessingTime: (Avg: 3.919ms ; Min: 2.402ms ; Max: 11.234ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.27 - BytesRead: 1.51 GB (1619983151) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.51 GB (1619983151) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.51 GB (1619983151) - DecompressionTime: 9s240ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.07 GB (1146896013) - PerReadThreadRawHdfsThroughput: 1.46 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 30.634ms - RowBatchQueuePutWaitTime: 7m23s - RowsRead: 128.00M (128001189) - RowsReturned: 24.20M (24199028) - RowsReturnedRate: 82.91 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.69K (10687) - ScannerThreadsTotalWallClockTime: 7m44s - MaterializeTupleTime(*): 11s112ms - ScannerThreadsSysTime: 1s270ms - ScannerThreadsUserTime: 20s337ms - ScannerThreadsVoluntaryContextSwitches: 23.82K (23822) - TotalRawHdfsReadTime(*): 1s031ms - TotalReadThroughput: 33.95 MB/sec Instance 7c4dade34ea955d6:dc885f570000000d (host=vb0208.halxg.cloudera.com:22000):(Total: 45s458ms, non-child: 36s690ms, % non-child: 80.71%) Hdfs split stats (:<# splits>/): 9:1/253.39 MB 7:1/234.34 MB 5:2/506.78 MB 4:3/760.21 MB 8:3/760.15 MB 1:2/506.82 MB 3:1/253.41 MB 2:4/1013.57 MB 0:2/506.80 MB MemoryUsage(1s000ms): 999.82 MB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.05 GB, 1.00 GB, 991.16 MB, 949.01 MB, 914.47 MB, 894.85 MB, 874.82 MB, 838.92 MB, 822.31 MB, 780.69 MB, 737.53 MB, 712.21 MB, 666.94 MB, 632.73 MB, 591.07 MB, 645.08 MB, 615.50 MB, 563.05 MB, 670.86 MB, 657.00 MB, 659.62 MB, 741.04 MB, 706.66 MB, 676.42 MB, 789.32 MB, 728.05 MB, 701.84 MB, 689.38 MB, 665.80 MB, 619.05 MB, 551.16 MB, 497.67 MB, 453.36 MB, 424.62 MB, 392.93 MB, 352.45 MB, 281.04 MB, 211.51 MB, 163.49 MB, 95.63 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 12, 12, 11, 11, 8, 8, 8, 8, 8, 8, 7, 5, 4, 3 - AverageThreadTokens: 11.35 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1185040755) - PerHostPeakMemUsage: 1.10 GB (1185040755) - RowsProduced: 24.00M (24004509) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 37s535ms - TotalStorageWaitTime: 573.408ms - TotalThreadsInvoluntaryContextSwitches: 14.18K (14176) - TotalThreadsTotalWallClockTime: 8m34s - TotalThreadsSysTime: 2s287ms - TotalThreadsUserTime: 28s371ms - TotalThreadsVoluntaryContextSwitches: 46.22K (46216) Fragment Instance Lifecycle Timings: - ExecTime: 45s398ms - ExecTreeExecTime: 357.329ms - OpenTime: 33.653ms - ExecTreeOpenTime: 48.370us - PrepareTime: 25.717ms - ExecTreePrepareTime: 353.083us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s418ms, non-child: 8s418ms, % non-child: 100.00%) - BytesSent: 490.64 MB (514476323) - NetworkThroughput(*): 269.66 MB/sec - OverallThroughput: 58.28 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.00M (24004509) - SerializeBatchTime: 6s590ms - TransmitDataRPCTime: 1s819ms - UncompressedRowBatchSize: 847.11 MB (888260613) CodeGen:(Total: 56.846ms, non-child: 56.846ms, % non-child: 100.00%) - CodegenTime: 1.441ms - CompileTime: 10.271ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.624ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 23.395ms HDFS_SCAN_NODE (id=1):(Total: 291.807ms, non-child: 291.807ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 9:1/253.39 MB 7:1/234.34 MB 5:2/506.78 MB 4:3/760.21 MB 8:3/760.15 MB 1:2/506.82 MB 3:1/253.41 MB 2:4/1013.57 MB 0:2/506.80 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:97.8% 1:1.099% 2:0% 3:1.099% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 874.14 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 1.05 GB, 1.05 GB, 1.06 GB, 1.22 GB, 1.22 GB, 1.26 GB, 1.38 GB, 1.38 GB, 1.38 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB - FooterProcessingTime: (Avg: 3.486ms ; Min: 1.838ms ; Max: 6.136ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.04 - AverageScannerThreadConcurrency: 10.35 - BytesRead: 1.54 GB (1657536250) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.54 GB (1657536250) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.54 GB (1657536250) - DecompressionTime: 9s439ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1184658288) - PerReadThreadRawHdfsThroughput: 1.40 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 31.905ms - RowBatchQueuePutWaitTime: 7m26s - RowsRead: 130.98M (130980854) - RowsReturned: 24.00M (24004509) - RowsReturnedRate: 82.26 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.05K (10048) - ScannerThreadsTotalWallClockTime: 7m49s - MaterializeTupleTime(*): 11s382ms - ScannerThreadsSysTime: 1s349ms - ScannerThreadsUserTime: 20s752ms - ScannerThreadsVoluntaryContextSwitches: 23.62K (23622) - TotalRawHdfsReadTime(*): 1s100ms - TotalReadThroughput: 34.74 MB/sec Instance 7c4dade34ea955d6:dc885f5700000003 (host=vb0212.halxg.cloudera.com:22000):(Total: 44s340ms, non-child: 35s752ms, % non-child: 80.63%) Hdfs split stats (:<# splits>/): 10:1/253.38 MB 1:3/760.18 MB 5:1/253.39 MB 4:2/506.79 MB 7:3/760.20 MB 0:1/253.40 MB 6:2/506.82 MB 3:2/506.82 MB 2:3/760.21 MB MemoryUsage(1s000ms): 1.01 GB, 1.11 GB, 1.11 GB, 1.11 GB, 1.09 GB, 1.04 GB, 1.03 GB, 989.47 MB, 946.73 MB, 905.38 MB, 880.72 MB, 876.87 MB, 855.73 MB, 826.86 MB, 791.65 MB, 733.95 MB, 701.47 MB, 669.55 MB, 635.82 MB, 619.53 MB, 609.74 MB, 701.36 MB, 747.05 MB, 711.13 MB, 683.39 MB, 656.17 MB, 627.06 MB, 574.20 MB, 566.65 MB, 677.81 MB, 695.38 MB, 669.74 MB, 643.02 MB, 613.97 MB, 536.10 MB, 472.34 MB, 435.26 MB, 407.79 MB, 356.51 MB, 326.77 MB, 292.34 MB, 243.27 MB, 178.39 MB, 100.19 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 11, 11, 9, 8, 7, 7, 7, 7, 7, 6, 5, 3 - AverageThreadTokens: 11.37 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1193057291) - PerHostPeakMemUsage: 1.11 GB (1193057291) - RowsProduced: 23.47M (23465800) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 36s565ms - TotalStorageWaitTime: 581.469ms - TotalThreadsInvoluntaryContextSwitches: 15.65K (15650) - TotalThreadsTotalWallClockTime: 8m23s - TotalThreadsSysTime: 2s009ms - TotalThreadsUserTime: 27s581ms - TotalThreadsVoluntaryContextSwitches: 45.07K (45071) Fragment Instance Lifecycle Timings: - ExecTime: 44s282ms - ExecTreeExecTime: 363.682ms - OpenTime: 32.687ms - ExecTreeOpenTime: 45.290us - PrepareTime: 25.300ms - ExecTreePrepareTime: 309.822us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s229ms, non-child: 8s229ms, % non-child: 100.00%) - BytesSent: 479.63 MB (502926019) - NetworkThroughput(*): 278.38 MB/sec - OverallThroughput: 58.28 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 23.47M (23465800) - SerializeBatchTime: 6s497ms - TransmitDataRPCTime: 1s722ms - UncompressedRowBatchSize: 828.10 MB (868326276) CodeGen:(Total: 55.524ms, non-child: 55.524ms, % non-child: 100.00%) - CodegenTime: 1.275ms - CompileTime: 9.544ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.355ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 23.111ms HDFS_SCAN_NODE (id=1):(Total: 303.559ms, non-child: 303.559ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 10:1/253.38 MB 1:3/760.18 MB 5:1/253.39 MB 4:2/506.79 MB 7:3/760.20 MB 0:1/253.40 MB 6:2/506.82 MB 3:2/506.82 MB 2:3/760.21 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 18 out of 18 Hdfs Read Thread Concurrency Bucket: 0:97.75% 1:1.124% 2:0% 3:1.124% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:72 BytesRead(1s000ms): 912.46 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1.02 GB, 1.14 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.26 GB, 1.42 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 4.209ms ; Min: 2.868ms ; Max: 6.931ms ; Number of samples: 18) - AverageHdfsReadThreadConcurrency: 0.04 - AverageScannerThreadConcurrency: 10.37 - BytesRead: 1.47 GB (1576589528) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1576589528) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1576589528) - DecompressionTime: 9s021ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1192710656) - PerReadThreadRawHdfsThroughput: 1.43 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 28.650ms - RowBatchQueuePutWaitTime: 7m17s - RowsRead: 124.58M (124581157) - RowsReturned: 23.47M (23465800) - RowsReturnedRate: 77.30 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 11.68K (11677) - ScannerThreadsTotalWallClockTime: 7m39s - MaterializeTupleTime(*): 10s842ms - ScannerThreadsSysTime: 1s168ms - ScannerThreadsUserTime: 19s963ms - ScannerThreadsVoluntaryContextSwitches: 23.04K (23039) - TotalRawHdfsReadTime(*): 1s028ms - TotalReadThroughput: 33.79 MB/sec Instance 7c4dade34ea955d6:dc885f5700000007 (host=vb0234.halxg.cloudera.com:22000):(Total: 42s860ms, non-child: 34s606ms, % non-child: 80.74%) Hdfs split stats (:<# splits>/): 4:1/253.41 MB 5:2/506.79 MB 9:2/506.77 MB 1:1/253.39 MB 7:3/760.20 MB 6:3/760.17 MB 3:3/760.18 MB 2:2/258.65 MB 0:2/506.79 MB MemoryUsage(1s000ms): 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.08 GB, 1.05 GB, 1.01 GB, 989.77 MB, 944.62 MB, 906.09 MB, 891.69 MB, 883.53 MB, 859.28 MB, 825.42 MB, 782.30 MB, 748.14 MB, 702.56 MB, 671.71 MB, 632.86 MB, 652.41 MB, 656.70 MB, 725.23 MB, 729.27 MB, 718.96 MB, 681.60 MB, 633.27 MB, 676.89 MB, 629.30 MB, 664.62 MB, 709.00 MB, 684.56 MB, 651.15 MB, 610.19 MB, 553.10 MB, 497.98 MB, 444.36 MB, 393.06 MB, 349.21 MB, 302.09 MB, 257.19 MB, 218.13 MB, 156.42 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 11, 10, 9, 8, 7, 6, 6, 6, 6, 5 - AverageThreadTokens: 11.55 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1189087133) - PerHostPeakMemUsage: 1.11 GB (1189087133) - RowsProduced: 23.36M (23361849) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 35s131ms - TotalStorageWaitTime: 624.243ms - TotalThreadsInvoluntaryContextSwitches: 15.37K (15374) - TotalThreadsTotalWallClockTime: 8m13s - TotalThreadsSysTime: 2s155ms - TotalThreadsUserTime: 27s554ms - TotalThreadsVoluntaryContextSwitches: 44.45K (44447) Fragment Instance Lifecycle Timings: - ExecTime: 42s802ms - ExecTreeExecTime: 340.753ms - OpenTime: 32.581ms - ExecTreeOpenTime: 43.153us - PrepareTime: 24.959ms - ExecTreePrepareTime: 325.487us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 7s919ms, non-child: 7s919ms, % non-child: 100.00%) - BytesSent: 477.51 MB (500701167) - NetworkThroughput(*): 326.60 MB/sec - OverallThroughput: 60.29 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 23.36M (23361849) - SerializeBatchTime: 6s449ms - TransmitDataRPCTime: 1s462ms - UncompressedRowBatchSize: 824.43 MB (864479689) CodeGen:(Total: 55.114ms, non-child: 55.114ms, % non-child: 100.00%) - CodegenTime: 1.391ms - CompileTime: 9.576ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.245ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 22.781ms HDFS_SCAN_NODE (id=1):(Total: 278.752ms, non-child: 278.752ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:1/253.41 MB 5:2/506.79 MB 9:2/506.77 MB 1:1/253.39 MB 7:3/760.20 MB 6:3/760.17 MB 3:3/760.18 MB 2:2/258.65 MB 0:2/506.79 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1.02 GB, 1.06 GB, 1.18 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.31 GB, 1.31 GB, 1.39 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 4.464ms ; Min: 2.629ms ; Max: 7.619ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.55 - BytesRead: 1.47 GB (1578597766) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1578597766) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1578597766) - DecompressionTime: 9s088ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1188907418) - PerReadThreadRawHdfsThroughput: 1.39 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 33.888ms - RowBatchQueuePutWaitTime: 7m8s - RowsRead: 124.71M (124712354) - RowsReturned: 23.36M (23361849) - RowsReturnedRate: 83.81 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 11.60K (11595) - ScannerThreadsTotalWallClockTime: 7m30s - MaterializeTupleTime(*): 10s930ms - ScannerThreadsSysTime: 1s270ms - ScannerThreadsUserTime: 19s983ms - ScannerThreadsVoluntaryContextSwitches: 22.92K (22923) - TotalRawHdfsReadTime(*): 1s058ms - TotalReadThroughput: 35.42 MB/sec ================================================================================ ================================================================================ SUMMARY: +-----------------+--------+----------+---------+----------+---------+-----------+ | Operator | #Hosts | Avg Time | Std Dev | Max Time | #Rows | Est #Rows | +-----------------+--------+----------+---------+----------+---------+-----------+ | 06:AGGREGATE | 1 | 314.51us | 0ns | 314.51us | 1 | 1 | | 05:EXCHANGE | 1 | 18.41us | 0ns | 18.41us | 1 | 1 | | 03:AGGREGATE | 1 | 2.17ms | 0ns | 2.17ms | 1 | 1 | | 02:HASH JOIN | 1 | 67.54s | 0ns | 67.54s | 0 | 3 | | |--04:EXCHANGE | 1 | 8.56s | 0ns | 8.56s | 350.01M | 180.00M | | | 01:SCAN HDFS | 14 | 304.02ms | 0ns | 349.32ms | 350.01M | 180.00M | | 00:SCAN HDFS | 1 | 7.32ms | 0ns | 7.32ms | 0 | 3 | +-----------------+--------+----------+---------+----------+---------+-----------+ PROFILE: Query (id=5c46a32203f83096:fcd5b82100000000): Summary: Session ID: 9455fa8a8a4d669:e1f20f88b52d35ac Session Type: BEESWAX Start Time: 2017-02-17 04:37:22.041950000 End Time: Query Type: QUERY Query State: CREATED Query Status: OK Impala Version: impalad version 2.9.0-SNAPSHOT RELEASE (build 9842bfcdbff9dfbb2fb1a8da8eda98ed2d56e76a) User: jenkins Connected User: jenkins Delegated User: Network Address: 10.17.193.6:38890 Default Db: tpch_300_parquet Sql Statement: -- Description : Large broadcast join with no rows qualifying from the probe side. -- Target test case : Broadcast large amounts of data over the network while stressing the -- exchange operator along with HashTable build. SELECT /* +straight_join */ count(*) FROM nation JOIN /* +broadcast */ lineitem ON l_orderkey = n_nationkey WHERE l_partkey > 0 AND l_suppkey > 0 AND l_linenumber > 0 AND n_regionkey < 0 AND l_orderkey < 350000000 Coordinator: vb0204.halxg.cloudera.com:22000 Query Options (non default): NUM_SCANNER_THREADS=12,SYNC_DDL=1,MAX_NUM_RUNTIME_FILTERS=50 Plan: ---------------- Estimated Per-Host Requirements: Memory=6.24GB VCores=2 PLAN-ROOT SINK | 06:AGGREGATE [FINALIZE] | output: count:merge(*) | hosts=1 per-host-mem=unavailable | tuple-ids=2 row-size=8B cardinality=1 | 05:EXCHANGE [UNPARTITIONED] | hosts=1 per-host-mem=unavailable | tuple-ids=2 row-size=8B cardinality=1 | 03:AGGREGATE | output: count(*) | hosts=1 per-host-mem=10.00MB | tuple-ids=2 row-size=8B cardinality=1 | 02:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: n_nationkey = l_orderkey | runtime filters: RF000 <- l_orderkey | hosts=1 per-host-mem=5.90GB | tuple-ids=0,1 row-size=48B cardinality=3 | |--04:EXCHANGE [BROADCAST] | | hosts=14 per-host-mem=0B | | tuple-ids=1 row-size=32B cardinality=179998909 | | | 01:SCAN HDFS [tpch_300_parquet.lineitem, RANDOM] | partitions=1/1 files=264 size=64.36GB | predicates: l_linenumber > 0, l_orderkey < 350000000, l_partkey > 0, l_suppkey > 0 | table stats: 1799989091 rows total | column stats: all | hosts=14 per-host-mem=352.00MB | tuple-ids=1 row-size=32B cardinality=179998909 | 00:SCAN HDFS [tpch_300_parquet.nation, RANDOM] partitions=1/1 files=1 size=2.19KB predicates: n_regionkey < 0, tpch_300_parquet.nation.n_nationkey < 350000000 runtime filters: RF000 -> n_nationkey table stats: 25 rows total column stats: all hosts=1 per-host-mem=32.00MB tuple-ids=0 row-size=16B cardinality=3 ---------------- Estimated Per-Host Mem: 6705060455 Estimated Per-Host VCores: 2 Request Pool: root.jenkins Admission result: Admitted immediately Planner Timeline: 5.951ms - Analysis finished: 1.916ms (1.916ms) - Equivalence classes computed: 2.032ms (115.654us) - Single node plan created: 3.412ms (1.380ms) - Runtime filters computed: 3.526ms (113.939us) - Distributed plan created: 3.757ms (230.527us) - Lineage info computed: 3.882ms (125.658us) - Planning finished: 5.951ms (2.068ms) Query Timeline: 1m19s - Query submitted: 45.072us (45.072us) - Planning finished: 8.532ms (8.487ms) - Submit for admission: 9.781ms (1.248ms) - Completed admission: 9.896ms (114.663us) - Ready to start 16 fragment instances: 10.211ms (314.865us) - All 16 fragment instances started: 3s314ms (3s303ms) - Rows available: 1m19s (1m16s) - First row fetched: 1m19s (22.626ms) - ComputeScanRangeAssignmentTimer: 584.954us ImpalaServer: - ClientFetchWaitTimer: 22.636ms - RowMaterializationTimer: 3.463ms Execution Profile 5c46a32203f83096:fcd5b82100000000:(Total: 1m19s, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 1 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed Enabled ---------------------------------------------------------------------------------------------------------------------------- 0 2 0 1 LOCAL false 0 (1) N/A N/A true Fragment instance start latencies: Count: 16, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 2ms, 95th %-ile: 3s302ms, 99.9th %-ile: 3s302ms Per Node Peak Memory Usage: vb0206.halxg.cloudera.com:22000(1.11 GB) vb0220.halxg.cloudera.com:22000(1.10 GB) vb0228.halxg.cloudera.com:22000(1.09 GB) vb0222.halxg.cloudera.com:22000(1.10 GB) vb0224.halxg.cloudera.com:22000(1.10 GB) vb0232.halxg.cloudera.com:22000(1.11 GB) vb0234.halxg.cloudera.com:22000(1.11 GB) vb0218.halxg.cloudera.com:22000(1.10 GB) vb0214.halxg.cloudera.com:22000(26.43 GB) vb0212.halxg.cloudera.com:22000(1.11 GB) vb0216.halxg.cloudera.com:22000(1.11 GB) vb0208.halxg.cloudera.com:22000(1.10 GB) vb0230.halxg.cloudera.com:22000(1.11 GB) vb0204.halxg.cloudera.com:22000(1.10 GB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Coordinator Fragment F02: Instance 5c46a32203f83096:fcd5b82100000000 (host=vb0204.halxg.cloudera.com:22000):(Total: 1m19s, non-child: 23.278ms, % non-child: 0.03%) MemoryUsage(2s000ms): 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 28.01 KB (28680) - PerHostPeakMemUsage: 1.10 GB (1183630989) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1m19s - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 1m19s - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 1.000ms - TotalThreadsVoluntaryContextSwitches: 6 (6) Fragment Instance Lifecycle Timings: - ExecTime: 22.709ms - ExecTreeExecTime: 6.419us - OpenTime: 1m19s - ExecTreeOpenTime: 1m19s - PrepareTime: 24.340ms - ExecTreePrepareTime: 117.534us PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 23.955ms, non-child: 23.955ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 23.424ms AGGREGATION_NODE (id=6):(Total: 1m19s, non-child: 314.508us, % non-child: 0.00%) - BuildTime: 8.053us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.00 KB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=5):(Total: 1m19s, non-child: 18.407us, % non-child: 0.00%) BytesReceived(2s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 - BytesReceived: 16.00 B (16) - ConvertRowBatchTime: 1.748us - DeserializeRowBatchTimer: 13.058us - FirstBatchArrivalWaitTime: 1m19s - PeakMemoryUsage: 0 - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 1m16s, non-child: 7.137ms, % non-child: 0.01%) split sizes: min: 2.19 KB, max: 2.19 KB, avg: 2.19 KB, stddev: 0 completion times: min:1m16s max:1m16s mean: 1m16s stddev:0.000ns execution rates: min:29.00 B/sec max:29.00 B/sec mean:29.00 B/sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 1.99 - BloomFilterBytes: 16.00 MB (16777216) - PeakMemoryUsage: 26.43 GB (28380916821) - PerHostPeakMemUsage: 26.43 GB (28380916821) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 2s081ms - TotalNetworkSendTime: 444.327us - TotalStorageWaitTime: 6.006ms - TotalThreadsInvoluntaryContextSwitches: 8.17K (8174) - TotalThreadsTotalWallClockTime: 2m32s - TotalThreadsSysTime: 16s637ms - TotalThreadsUserTime: 59s241ms - TotalThreadsVoluntaryContextSwitches: 15.94K (15944) Fragment Instance Lifecycle Timings: - ExecTime: 880.291us - ExecTreeExecTime: 4.317us - OpenTime: 1m16s - ExecTreeOpenTime: 1m16s - PrepareTime: 32.377ms - ExecTreePrepareTime: 400.076us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 2.42K (2419) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 26.40 GB (28345106432) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=5):(Total: 112.180us, non-child: 112.180us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 265.34 KB/sec - OverallThroughput: 139.28 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.517us - TransmitDataRPCTime: 58.887us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 360.159ms, non-child: 360.159ms, % non-child: 100.00%) - CodegenTime: 1.860ms - CompileTime: 85.777ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 90 (90) - NumInstructions: 2.93K (2928) - OptimizationTime: 247.747ms - PeakMemoryUsage: 1.43 MB (1499136) - PrepareTime: 26.108ms AGGREGATION_NODE (id=3):(Total: 1m16s, non-child: 2.167ms, % non-child: 0.00%) - BuildTime: 2.144ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 24.00 KB (24576) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=2):(Total: 1m16s, non-child: 1m7s, % non-child: 88.74%) - BuildRows: 350.01M (350008706) - BuildTime: 1m6s - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 26.42 GB (28364049664) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 1.360us - RowsReturned: 0 (0) - RowsReturnedRate: 0 Hash Join Builder (join_node_id=2): - BuildRowsPartitionTime: 41s555ms - BuildRowsPartitioned: 350.01M (350008706) - GetNewBlockTime: 10.489ms - HashBuckets: 536.87M (536870912) - HashCollisions: 0 (0) - HashTablesBuildTime: 24s775ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 26.42 GB (28364002432) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=4):(Total: 8s563ms, non-child: 8s563ms, % non-child: 100.00%) - BytesReceived: 6.99 GB (7501460242) - ConvertRowBatchTime: 6s095ms - DeserializeRowBatchTimer: 21s812ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 350.01M (350008706) - RowsReturnedRate: 40.87 M/sec - SendersBlockedTimer: 48s154ms - SendersBlockedTotalTimer(*): 9m13s HDFS_SCAN_NODE (id=0):(Total: 7.316ms, non-child: 7.316ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 2.44 KB (2497) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.44 KB (2497) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.44 KB (2497) - DecompressionTime: 6.640us - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDisksAccessed: 1 (1) - NumRowGroups: 1 (1) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 73.00 KB (74752) - PerReadThreadRawHdfsThroughput: 59.30 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 6.477ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 25 (25) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 6.549ms - MaterializeTupleTime(*): 34.163us - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 6 (6) - TotalRawHdfsReadTime(*): 40.157us - TotalReadThroughput: 0.00 /sec Filter 0 (16.00 MB): - Rows processed: 25 (25) - Rows rejected: 0 (0) - Rows total: 25 (25) Fragment F00: Instance 5c46a32203f83096:fcd5b8210000000f (host=vb0214.halxg.cloudera.com:22000):(Total: 1m16s, non-child: 7.137ms, % non-child: 0.01%) Hdfs split stats (:<# splits>/): 6:1/2.19 KB Filter 0 arrival: 51s225ms MemoryUsage(2s000ms): 230.34 MB, 652.26 MB, 1.11 GB, 1.51 GB, 1.98 GB, 2.42 GB, 2.86 GB, 3.36 GB, 3.76 GB, 4.23 GB, 4.67 GB, 5.11 GB, 5.51 GB, 5.98 GB, 6.36 GB, 6.76 GB, 7.17 GB, 7.57 GB, 7.98 GB, 8.36 GB, 8.78 GB, 9.20 GB, 9.61 GB, 9.98 GB, 10.42 GB, 10.90 GB, 12.04 GB, 13.28 GB, 14.63 GB, 15.93 GB, 17.51 GB, 18.81 GB, 20.15 GB, 21.45 GB, 22.62 GB, 23.68 GB, 24.87 GB, 25.95 GB ThreadUsage(2s000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2 - AverageThreadTokens: 1.99 - BloomFilterBytes: 16.00 MB (16777216) - PeakMemoryUsage: 26.43 GB (28380916821) - PerHostPeakMemUsage: 26.43 GB (28380916821) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 2s081ms - TotalNetworkSendTime: 444.327us - TotalStorageWaitTime: 6.006ms - TotalThreadsInvoluntaryContextSwitches: 8.17K (8174) - TotalThreadsTotalWallClockTime: 2m32s - TotalThreadsSysTime: 16s637ms - TotalThreadsUserTime: 59s241ms - TotalThreadsVoluntaryContextSwitches: 15.94K (15944) Fragment Instance Lifecycle Timings: - ExecTime: 880.291us - ExecTreeExecTime: 4.317us - OpenTime: 1m16s - ExecTreeOpenTime: 1m16s - PrepareTime: 32.377ms - ExecTreePrepareTime: 400.076us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 2.42K (2419) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 26.40 GB (28345106432) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=5):(Total: 112.180us, non-child: 112.180us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 265.34 KB/sec - OverallThroughput: 139.28 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.517us - TransmitDataRPCTime: 58.887us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 360.159ms, non-child: 360.159ms, % non-child: 100.00%) - CodegenTime: 1.860ms - CompileTime: 85.777ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 90 (90) - NumInstructions: 2.93K (2928) - OptimizationTime: 247.747ms - PeakMemoryUsage: 1.43 MB (1499136) - PrepareTime: 26.108ms AGGREGATION_NODE (id=3):(Total: 1m16s, non-child: 2.167ms, % non-child: 0.00%) ExecOption: Codegen Enabled - BuildTime: 2.144ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 24.00 KB (24576) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=2):(Total: 1m16s, non-child: 1m7s, % non-child: 88.74%) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 350.01M (350008706) - BuildTime: 1m6s - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 26.42 GB (28364049664) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 1.360us - RowsReturned: 0 (0) - RowsReturnedRate: 0 Hash Join Builder (join_node_id=2): ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 0 of 1 Runtime Filter Published, 1 Disabled - BuildRowsPartitionTime: 41s555ms - BuildRowsPartitioned: 350.01M (350008706) - GetNewBlockTime: 10.489ms - HashBuckets: 536.87M (536870912) - HashCollisions: 0 (0) - HashTablesBuildTime: 24s775ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 26.42 GB (28364002432) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=4):(Total: 8s563ms, non-child: 8s563ms, % non-child: 100.00%) BytesReceived(2s000ms): 98.89 MB, 380.96 MB, 673.06 MB, 961.32 MB, 1.23 GB, 1.52 GB, 1.81 GB, 2.10 GB, 2.40 GB, 2.69 GB, 2.98 GB, 3.26 GB, 3.53 GB, 3.82 GB, 4.08 GB, 4.34 GB, 4.60 GB, 4.86 GB, 5.12 GB, 5.39 GB, 5.65 GB, 5.91 GB, 6.17 GB, 6.44 GB, 6.71 GB, 6.95 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB - BytesReceived: 6.99 GB (7501460242) - ConvertRowBatchTime: 6s095ms - DeserializeRowBatchTimer: 21s812ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 350.01M (350008706) - RowsReturnedRate: 40.87 M/sec - SendersBlockedTimer: 48s154ms - SendersBlockedTotalTimer(*): 9m13s HDFS_SCAN_NODE (id=0):(Total: 7.316ms, non-child: 7.316ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:1/2.19 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:2 BytesRead(2s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 - FooterProcessingTime: (Avg: 3.764ms ; Min: 3.764ms ; Max: 3.764ms ; Number of samples: 1) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 2.44 KB (2497) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.44 KB (2497) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.44 KB (2497) - DecompressionTime: 6.640us - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDisksAccessed: 1 (1) - NumRowGroups: 1 (1) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 73.00 KB (74752) - PerReadThreadRawHdfsThroughput: 59.30 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 6.477ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 25 (25) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 6.549ms - MaterializeTupleTime(*): 34.163us - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 6 (6) - TotalRawHdfsReadTime(*): 40.157us - TotalReadThroughput: 0.00 /sec Filter 0 (16.00 MB): - Rows processed: 25 (25) - Rows rejected: 0 (0) - Rows total: 25 (25) Averaged Fragment F01:(Total: 52s639ms, non-child: 43s682ms, % non-child: 82.98%) split sizes: min: 4.45 GB, max: 4.70 GB, avg: 4.60 GB, stddev: 110.56 MB completion times: min:50s546ms max:54s491ms mean: 52s641ms stddev:1s251ms execution rates: min:85.87 MB/sec max:94.17 MB/sec mean:89.45 MB/sec stddev:2.15 MB/sec num instances: 14 - AverageThreadTokens: 11.47 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1185106167) - PerHostPeakMemUsage: 1.80 GB (1933578209) - RowsProduced: 25.00M (25000621) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 44s529ms - TotalStorageWaitTime: 583.331ms - TotalThreadsInvoluntaryContextSwitches: 14.86K (14855) - TotalThreadsTotalWallClockTime: 10m3s - TotalThreadsSysTime: 2s131ms - TotalThreadsUserTime: 28s063ms - TotalThreadsVoluntaryContextSwitches: 48.06K (48057) Fragment Instance Lifecycle Timings: - ExecTime: 52s580ms - ExecTreeExecTime: 366.780ms - OpenTime: 33.401ms - ExecTreeOpenTime: 48.960us - PrepareTime: 26.061ms - ExecTreePrepareTime: 325.795us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s595ms, non-child: 8s595ms, % non-child: 100.00%) - BytesSent: 511.00 MB (535818588) - NetworkThroughput(*): 290.71 MB/sec - OverallThroughput: 59.64 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 25.00M (25000621) - SerializeBatchTime: 6s815ms - TransmitDataRPCTime: 1s771ms - UncompressedRowBatchSize: 882.26 MB (925120683) CodeGen:(Total: 56.939ms, non-child: 56.939ms, % non-child: 100.00%) - CodegenTime: 1.415ms - CompileTime: 10.010ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.613ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 23.770ms HDFS_SCAN_NODE (id=1):(Total: 304.018ms, non-child: 304.018ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 10.47 - BytesRead: 1.52 GB (1627133960) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.52 GB (1627133960) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.52 GB (1627133960) - DecompressionTime: 9s111ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1184775314) - PerReadThreadRawHdfsThroughput: 1.43 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 31.550ms - RowBatchQueuePutWaitTime: 8m48s - RowsRead: 128.57M (128570649) - RowsReturned: 25.00M (25000621) - RowsReturnedRate: 82.53 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 10.88K (10877) - ScannerThreadsTotalWallClockTime: 9m10s - MaterializeTupleTime(*): 10s979ms - ScannerThreadsSysTime: 1s238ms - ScannerThreadsUserTime: 20s177ms - ScannerThreadsVoluntaryContextSwitches: 24.50K (24502) - TotalRawHdfsReadTime(*): 1s060ms - TotalReadThroughput: 29.53 MB/sec Fragment F01: Instance 5c46a32203f83096:fcd5b82100000006 (host=vb0218.halxg.cloudera.com:22000):(Total: 54s490ms, non-child: 44s390ms, % non-child: 81.47%) Hdfs split stats (:<# splits>/): 8:1/253.38 MB 1:1/253.39 MB 9:1/253.38 MB 10:1/253.40 MB 5:2/506.79 MB 2:5/1.24 GB 7:2/506.80 MB 6:2/506.81 MB 3:3/760.16 MB 0:1/253.39 MB MemoryUsage(1s000ms): 556.77 MB, 1.09 GB, 1.09 GB, 1.09 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.10 GB, 1.08 GB, 1.06 GB, 1.01 GB, 963.16 MB, 933.96 MB, 922.32 MB, 919.67 MB, 902.32 MB, 899.05 MB, 859.20 MB, 824.22 MB, 791.89 MB, 754.50 MB, 722.43 MB, 702.01 MB, 697.67 MB, 666.00 MB, 637.20 MB, 605.90 MB, 609.03 MB, 612.68 MB, 584.24 MB, 542.82 MB, 513.67 MB, 500.81 MB, 525.46 MB, 569.33 MB, 543.21 MB, 533.01 MB, 556.23 MB, 577.75 MB, 709.71 MB, 738.69 MB, 690.82 MB, 635.32 MB, 606.79 MB, 569.94 MB, 543.04 MB, 516.90 MB, 496.76 MB, 461.54 MB, 428.93 MB, 394.83 MB, 366.59 MB, 283.71 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 11, 9, 8, 8, 8, 8, 8, 8, 8, 8, 8, 7 - AverageThreadTokens: 11.88 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1179953799) - PerHostPeakMemUsage: 1.10 GB (1179953799) - RowsProduced: 27.36M (27356338) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 45s441ms - TotalStorageWaitTime: 603.556ms - TotalThreadsInvoluntaryContextSwitches: 16.17K (16173) - TotalThreadsTotalWallClockTime: 10m44s - TotalThreadsSysTime: 2s338ms - TotalThreadsUserTime: 30s006ms - TotalThreadsVoluntaryContextSwitches: 52.12K (52121) Fragment Instance Lifecycle Timings: - ExecTime: 54s433ms - ExecTreeExecTime: 404.848ms - OpenTime: 33.254ms - ExecTreeOpenTime: 42.636us - PrepareTime: 23.232ms - ExecTreePrepareTime: 300.060us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 9s710ms, non-child: 9s710ms, % non-child: 100.00%) - BytesSent: 559.15 MB (586311676) - NetworkThroughput(*): 270.78 MB/sec - OverallThroughput: 57.58 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 27.36M (27356338) - SerializeBatchTime: 7s636ms - TransmitDataRPCTime: 2s064ms - UncompressedRowBatchSize: 965.40 MB (1012291378) CodeGen:(Total: 54.042ms, non-child: 54.042ms, % non-child: 100.00%) - CodegenTime: 1.267ms - CompileTime: 10.052ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.473ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 20.993ms HDFS_SCAN_NODE (id=1):(Total: 334.414ms, non-child: 334.414ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 8:1/253.38 MB 1:1/253.39 MB 9:1/253.38 MB 10:1/253.40 MB 5:2/506.79 MB 2:5/1.24 GB 7:2/506.80 MB 6:2/506.81 MB 3:3/760.16 MB 0:1/253.39 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 500.61 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.21 MB, 1001.78 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1.02 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.10 GB, 1.14 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.26 GB, 1.31 GB, 1.47 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB - FooterProcessingTime: (Avg: 5.105ms ; Min: 2.663ms ; Max: 7.890ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.88 - BytesRead: 1.55 GB (1664143743) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.55 GB (1664143743) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.55 GB (1664143743) - DecompressionTime: 9s521ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 10 (10) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1179675018) - PerReadThreadRawHdfsThroughput: 1.32 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 31.991ms - RowBatchQueuePutWaitTime: 9m27s - RowsRead: 131.50M (131502589) - RowsReturned: 27.36M (27356338) - RowsReturnedRate: 81.80 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 12.02K (12015) - ScannerThreadsTotalWallClockTime: 9m50s - MaterializeTupleTime(*): 11s520ms - ScannerThreadsSysTime: 1s321ms - ScannerThreadsUserTime: 21s248ms - ScannerThreadsVoluntaryContextSwitches: 26.68K (26677) - TotalRawHdfsReadTime(*): 1s174ms - TotalReadThroughput: 29.12 MB/sec Instance 5c46a32203f83096:fcd5b82100000005 (host=vb0204.halxg.cloudera.com:22000):(Total: 54s339ms, non-child: 44s832ms, % non-child: 82.51%) Hdfs split stats (:<# splits>/): 5:1/253.39 MB 2:1/253.41 MB 7:2/506.80 MB 6:1/253.41 MB 1:2/506.79 MB 8:2/506.78 MB 3:5/1.24 GB 4:4/1013.63 MB 0:1/253.40 MB MemoryUsage(1s000ms): 1.07 GB, 1.07 GB, 1.07 GB, 1.08 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.07 GB, 1.03 GB, 1000.19 MB, 955.52 MB, 931.60 MB, 919.78 MB, 915.83 MB, 890.50 MB, 866.39 MB, 827.23 MB, 796.35 MB, 759.27 MB, 731.53 MB, 699.69 MB, 679.00 MB, 667.90 MB, 671.39 MB, 660.90 MB, 635.85 MB, 616.40 MB, 576.53 MB, 525.51 MB, 488.02 MB, 468.77 MB, 542.18 MB, 579.28 MB, 630.52 MB, 655.34 MB, 654.62 MB, 635.52 MB, 643.61 MB, 714.43 MB, 648.86 MB, 602.64 MB, 573.12 MB, 548.70 MB, 511.64 MB, 455.42 MB, 442.58 MB, 425.82 MB, 399.34 MB, 372.90 MB, 326.09 MB, 295.29 MB, 193.01 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 10, 9, 9, 8, 7, 7, 7, 7, 7, 7, 7, 7, 5 - AverageThreadTokens: 11.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1183614605) - PerHostPeakMemUsage: 1.10 GB (1183630989) - RowsProduced: 27.05M (27045946) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 45s744ms - TotalStorageWaitTime: 623.493ms - TotalThreadsInvoluntaryContextSwitches: 17.28K (17280) - TotalThreadsTotalWallClockTime: 10m32s - TotalThreadsSysTime: 2s119ms - TotalThreadsUserTime: 29s539ms - TotalThreadsVoluntaryContextSwitches: 51.76K (51760) Fragment Instance Lifecycle Timings: - ExecTime: 54s280ms - ExecTreeExecTime: 416.917ms - OpenTime: 32.857ms - ExecTreeOpenTime: 54.840us - PrepareTime: 25.841ms - ExecTreePrepareTime: 291.654us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 9s100ms, non-child: 9s100ms, % non-child: 100.00%) - BytesSent: 552.79 MB (579639503) - NetworkThroughput(*): 297.42 MB/sec - OverallThroughput: 60.74 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 27.05M (27045946) - SerializeBatchTime: 7s232ms - TransmitDataRPCTime: 1s858ms - UncompressedRowBatchSize: 954.44 MB (1000805662) CodeGen:(Total: 56.486ms, non-child: 56.486ms, % non-child: 100.00%) - CodegenTime: 1.252ms - CompileTime: 9.642ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.345ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 23.962ms HDFS_SCAN_NODE (id=1):(Total: 349.317ms, non-child: 349.317ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:1/253.39 MB 2:1/253.41 MB 7:2/506.80 MB 6:1/253.41 MB 1:2/506.79 MB 8:2/506.78 MB 3:5/1.24 GB 4:4/1013.63 MB 0:1/253.40 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1.02 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.18 GB, 1.26 GB, 1.35 GB, 1.39 GB, 1.39 GB, 1.39 GB, 1.43 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB - FooterProcessingTime: (Avg: 4.483ms ; Min: 2.897ms ; Max: 7.418ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.67 - BytesRead: 1.55 GB (1664181733) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.55 GB (1664181733) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.55 GB (1664181733) - DecompressionTime: 9s379ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1183369346) - PerReadThreadRawHdfsThroughput: 1.45 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 35.162ms - RowBatchQueuePutWaitTime: 9m15s - RowsRead: 131.50M (131502331) - RowsReturned: 27.05M (27045946) - RowsReturnedRate: 77.43 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 13.09K (13086) - ScannerThreadsTotalWallClockTime: 9m37s - MaterializeTupleTime(*): 11s412ms - ScannerThreadsSysTime: 1s163ms - ScannerThreadsUserTime: 21s075ms - ScannerThreadsVoluntaryContextSwitches: 26.39K (26390) - TotalRawHdfsReadTime(*): 1s070ms - TotalReadThroughput: 29.39 MB/sec Instance 5c46a32203f83096:fcd5b82100000002 (host=vb0216.halxg.cloudera.com:22000):(Total: 54s177ms, non-child: 44s742ms, % non-child: 82.59%) Hdfs split stats (:<# splits>/): 1:1/253.38 MB 3:2/506.78 MB 10:1/253.39 MB 7:2/506.78 MB 9:5/1.24 GB 6:1/253.41 MB 5:2/506.78 MB 2:1/253.41 MB 0:4/1013.59 MB MemoryUsage(1s000ms): 1.09 GB, 1.09 GB, 1.09 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.07 GB, 1.04 GB, 1022.94 MB, 980.34 MB, 945.87 MB, 916.54 MB, 895.23 MB, 874.33 MB, 851.00 MB, 829.95 MB, 797.33 MB, 752.77 MB, 723.74 MB, 673.12 MB, 637.44 MB, 607.97 MB, 649.25 MB, 618.26 MB, 597.44 MB, 570.22 MB, 533.56 MB, 678.02 MB, 694.52 MB, 694.07 MB, 737.79 MB, 757.32 MB, 737.08 MB, 728.62 MB, 712.16 MB, 712.42 MB, 712.92 MB, 680.29 MB, 611.45 MB, 585.90 MB, 555.95 MB, 536.46 MB, 509.88 MB, 472.40 MB, 451.08 MB, 431.44 MB, 407.84 MB, 380.59 MB, 340.44 MB, 303.75 MB, 248.84 MB, 155.95 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 12, 9, 9, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 7, 4 - AverageThreadTokens: 11.63 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1190625515) - PerHostPeakMemUsage: 1.11 GB (1190625515) - RowsProduced: 26.59M (26594701) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 45s610ms - TotalStorageWaitTime: 632.597ms - TotalThreadsInvoluntaryContextSwitches: 15.13K (15131) - TotalThreadsTotalWallClockTime: 10m28s - TotalThreadsSysTime: 2s151ms - TotalThreadsUserTime: 29s481ms - TotalThreadsVoluntaryContextSwitches: 50.97K (50966) Fragment Instance Lifecycle Timings: - ExecTime: 54s117ms - ExecTreeExecTime: 380.208ms - OpenTime: 33.805ms - ExecTreeOpenTime: 50.419us - PrepareTime: 26.278ms - ExecTreePrepareTime: 336.451us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 9s067ms, non-child: 9s067ms, % non-child: 100.00%) - BytesSent: 543.60 MB (570003508) - NetworkThroughput(*): 294.28 MB/sec - OverallThroughput: 59.95 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 26.59M (26594701) - SerializeBatchTime: 7s211ms - TransmitDataRPCTime: 1s847ms - UncompressedRowBatchSize: 938.52 MB (984107841) CodeGen:(Total: 57.507ms, non-child: 57.507ms, % non-child: 100.00%) - CodegenTime: 1.474ms - CompileTime: 10.253ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.782ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 23.911ms HDFS_SCAN_NODE (id=1):(Total: 309.457ms, non-child: 309.457ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:1/253.38 MB 3:2/506.78 MB 10:1/253.39 MB 7:2/506.78 MB 9:5/1.24 GB 6:1/253.41 MB 5:2/506.78 MB 2:1/253.41 MB 0:4/1013.59 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.26 GB, 1.31 GB, 1.35 GB, 1.43 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.51 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB - FooterProcessingTime: (Avg: 4.892ms ; Min: 2.314ms ; Max: 8.599ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.63 - BytesRead: 1.55 GB (1664130351) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.55 GB (1664130351) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.55 GB (1664130351) - DecompressionTime: 9s493ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1190213336) - PerReadThreadRawHdfsThroughput: 1.42 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 34.843ms - RowBatchQueuePutWaitTime: 9m11s - RowsRead: 131.50M (131502389) - RowsReturned: 26.59M (26594701) - RowsReturnedRate: 85.94 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 11.12K (11125) - ScannerThreadsTotalWallClockTime: 9m34s - MaterializeTupleTime(*): 11s392ms - ScannerThreadsSysTime: 1s253ms - ScannerThreadsUserTime: 21s050ms - ScannerThreadsVoluntaryContextSwitches: 25.82K (25815) - TotalRawHdfsReadTime(*): 1s089ms - TotalReadThroughput: 29.39 MB/sec Instance 5c46a32203f83096:fcd5b82100000009 (host=vb0224.halxg.cloudera.com:22000):(Total: 53s595ms, non-child: 44s610ms, % non-child: 83.24%) Hdfs split stats (:<# splits>/): 3:1/253.39 MB 1:1/253.41 MB 10:1/253.38 MB 9:1/253.40 MB 5:2/506.78 MB 0:2/506.77 MB 4:2/506.76 MB 8:2/506.78 MB 7:3/760.18 MB 6:2/481.03 MB 2:2/506.80 MB MemoryUsage(1s000ms): 1.08 GB, 1.08 GB, 1.08 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.07 GB, 1.05 GB, 1.01 GB, 986.30 MB, 937.90 MB, 902.64 MB, 893.31 MB, 871.33 MB, 861.29 MB, 831.29 MB, 798.28 MB, 752.14 MB, 710.68 MB, 663.49 MB, 643.62 MB, 620.04 MB, 665.87 MB, 640.38 MB, 680.64 MB, 720.91 MB, 697.73 MB, 678.79 MB, 654.77 MB, 654.46 MB, 645.83 MB, 654.27 MB, 746.56 MB, 726.40 MB, 704.38 MB, 696.17 MB, 701.25 MB, 712.25 MB, 687.13 MB, 642.87 MB, 591.40 MB, 548.66 MB, 516.85 MB, 473.64 MB, 429.97 MB, 398.98 MB, 365.31 MB, 350.06 MB, 258.63 MB, 195.40 MB, 158.36 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 10, 9, 8, 8, 8, 8, 8, 8, 6, 5, 4 - AverageThreadTokens: 11.75 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1180132221) - PerHostPeakMemUsage: 1.10 GB (1180132221) - RowsProduced: 25.26M (25256825) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 45s410ms - TotalStorageWaitTime: 555.717ms - TotalThreadsInvoluntaryContextSwitches: 13.64K (13641) - TotalThreadsTotalWallClockTime: 10m29s - TotalThreadsSysTime: 2s133ms - TotalThreadsUserTime: 28s808ms - TotalThreadsVoluntaryContextSwitches: 48.56K (48561) Fragment Instance Lifecycle Timings: - ExecTime: 53s535ms - ExecTreeExecTime: 351.084ms - OpenTime: 34.326ms - ExecTreeOpenTime: 53.835us - PrepareTime: 25.285ms - ExecTreePrepareTime: 368.323us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s640ms, non-child: 8s640ms, % non-child: 100.00%) - BytesSent: 516.24 MB (541321888) - NetworkThroughput(*): 299.92 MB/sec - OverallThroughput: 59.74 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 25.26M (25256825) - SerializeBatchTime: 6s910ms - TransmitDataRPCTime: 1s721ms - UncompressedRowBatchSize: 891.31 MB (934601201) CodeGen:(Total: 56.921ms, non-child: 56.921ms, % non-child: 100.00%) - CodegenTime: 1.516ms - CompileTime: 10.576ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.991ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 22.776ms HDFS_SCAN_NODE (id=1):(Total: 286.441ms, non-child: 286.441ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:1/253.39 MB 1:1/253.41 MB 10:1/253.38 MB 9:1/253.40 MB 5:2/506.78 MB 0:2/506.77 MB 4:2/506.76 MB 8:2/506.78 MB 7:3/760.18 MB 6:2/481.03 MB 2:2/506.80 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 993.82 MB, 1.05 GB, 1.05 GB, 1.13 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.26 GB, 1.30 GB, 1.34 GB, 1.46 GB, 1.46 GB, 1.46 GB, 1.46 GB, 1.50 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB - FooterProcessingTime: (Avg: 4.528ms ; Min: 2.897ms ; Max: 8.093ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.75 - BytesRead: 1.54 GB (1655172689) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.54 GB (1655172689) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.54 GB (1655172689) - DecompressionTime: 9s413ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 11 (11) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1179853170) - PerReadThreadRawHdfsThroughput: 1.33 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 33.675ms - RowBatchQueuePutWaitTime: 9m13s - RowsRead: 130.80M (130797981) - RowsReturned: 25.26M (25256825) - RowsReturnedRate: 88.17 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 9.75K (9750) - ScannerThreadsTotalWallClockTime: 9m36s - MaterializeTupleTime(*): 11s291ms - ScannerThreadsSysTime: 1s276ms - ScannerThreadsUserTime: 20s767ms - ScannerThreadsVoluntaryContextSwitches: 24.67K (24666) - TotalRawHdfsReadTime(*): 1s161ms - TotalReadThroughput: 29.50 MB/sec Instance 5c46a32203f83096:fcd5b82100000001 (host=vb0230.halxg.cloudera.com:22000):(Total: 53s347ms, non-child: 44s438ms, % non-child: 83.30%) Hdfs split stats (:<# splits>/): 9:1/253.41 MB 2:1/253.39 MB 5:2/506.78 MB 3:2/506.81 MB 6:1/253.38 MB 7:2/506.78 MB 4:2/506.81 MB 1:3/760.19 MB 0:5/1.24 GB MemoryUsage(1s000ms): 1.08 GB, 1.08 GB, 1.08 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.07 GB, 1.05 GB, 1.01 GB, 991.68 MB, 934.66 MB, 910.79 MB, 900.75 MB, 877.11 MB, 863.73 MB, 830.37 MB, 801.19 MB, 757.06 MB, 706.72 MB, 672.00 MB, 635.39 MB, 700.66 MB, 665.88 MB, 673.76 MB, 693.83 MB, 661.96 MB, 660.12 MB, 674.43 MB, 633.67 MB, 621.07 MB, 637.34 MB, 677.26 MB, 661.25 MB, 643.62 MB, 615.71 MB, 627.74 MB, 692.32 MB, 704.15 MB, 647.36 MB, 602.22 MB, 548.80 MB, 488.69 MB, 473.11 MB, 426.83 MB, 388.11 MB, 343.37 MB, 327.59 MB, 279.53 MB, 233.14 MB, 181.56 MB, 126.15 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 11, 11, 9, 8, 8, 7, 7, 7, 7, 5, 5, 5, 4 - AverageThreadTokens: 11.61 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1188326417) - PerHostPeakMemUsage: 1.11 GB (1188326417) - RowsProduced: 25.08M (25076265) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 45s204ms - TotalStorageWaitTime: 618.667ms - TotalThreadsInvoluntaryContextSwitches: 15.69K (15689) - TotalThreadsTotalWallClockTime: 10m17s - TotalThreadsSysTime: 2s177ms - TotalThreadsUserTime: 28s723ms - TotalThreadsVoluntaryContextSwitches: 48.30K (48304) Fragment Instance Lifecycle Timings: - ExecTime: 53s287ms - ExecTreeExecTime: 356.138ms - OpenTime: 34.739ms - ExecTreeOpenTime: 54.709us - PrepareTime: 25.583ms - ExecTreePrepareTime: 332.672us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s557ms, non-child: 8s557ms, % non-child: 100.00%) - BytesSent: 512.53 MB (537431466) - NetworkThroughput(*): 297.86 MB/sec - OverallThroughput: 59.89 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 25.08M (25076265) - SerializeBatchTime: 6s828ms - TransmitDataRPCTime: 1s720ms - UncompressedRowBatchSize: 884.93 MB (927919769) CodeGen:(Total: 57.731ms, non-child: 57.731ms, % non-child: 100.00%) - CodegenTime: 1.374ms - CompileTime: 10.502ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 23.349ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 23.312ms HDFS_SCAN_NODE (id=1):(Total: 293.458ms, non-child: 293.458ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 9:1/253.41 MB 2:1/253.39 MB 5:2/506.78 MB 3:2/506.81 MB 6:1/253.38 MB 7:2/506.78 MB 4:2/506.81 MB 1:3/760.19 MB 0:5/1.24 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1.06 GB, 1.06 GB, 1.10 GB, 1.14 GB, 1.14 GB, 1.18 GB, 1.22 GB, 1.22 GB, 1.26 GB, 1.31 GB, 1.39 GB, 1.39 GB, 1.39 GB, 1.39 GB, 1.43 GB, 1.51 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB - FooterProcessingTime: (Avg: 5.246ms ; Min: 2.726ms ; Max: 9.274ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.61 - BytesRead: 1.55 GB (1664143238) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.55 GB (1664143238) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.55 GB (1664143238) - DecompressionTime: 9s419ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1188081158) - PerReadThreadRawHdfsThroughput: 1.42 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 31.502ms - RowBatchQueuePutWaitTime: 9m2s - RowsRead: 131.50M (131502184) - RowsReturned: 25.08M (25076265) - RowsReturnedRate: 85.45 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 11.71K (11713) - ScannerThreadsTotalWallClockTime: 9m24s - MaterializeTupleTime(*): 11s351ms - ScannerThreadsSysTime: 1s234ms - ScannerThreadsUserTime: 20s835ms - ScannerThreadsVoluntaryContextSwitches: 24.59K (24595) - TotalRawHdfsReadTime(*): 1s088ms - TotalReadThroughput: 29.94 MB/sec Instance 5c46a32203f83096:fcd5b82100000008 (host=vb0232.halxg.cloudera.com:22000):(Total: 53s201ms, non-child: 43s913ms, % non-child: 82.54%) Hdfs split stats (:<# splits>/): 6:2/506.80 MB 9:3/760.17 MB 7:1/253.39 MB 8:5/1.24 GB 1:2/260.77 MB 5:3/760.18 MB 4:1/253.40 MB 2:1/253.40 MB 0:1/253.38 MB MemoryUsage(1s000ms): 1.01 GB, 1.01 GB, 1.01 GB, 1.01 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.06 GB, 1.01 GB, 984.23 MB, 955.95 MB, 946.74 MB, 927.68 MB, 902.54 MB, 870.51 MB, 847.26 MB, 801.99 MB, 778.88 MB, 751.11 MB, 712.75 MB, 674.03 MB, 646.23 MB, 626.64 MB, 578.42 MB, 577.23 MB, 586.06 MB, 559.30 MB, 636.89 MB, 655.26 MB, 616.14 MB, 668.14 MB, 724.13 MB, 771.75 MB, 742.63 MB, 706.94 MB, 687.51 MB, 671.68 MB, 647.14 MB, 573.83 MB, 521.13 MB, 470.56 MB, 460.15 MB, 430.64 MB, 399.04 MB, 366.39 MB, 314.22 MB, 271.62 MB, 264.73 MB, 205.31 MB, 147.16 MB, 105.62 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 12, 9, 8, 8, 8, 7, 7, 7, 7, 6, 6, 5, 4, 3 - AverageThreadTokens: 11.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1187388491) - PerHostPeakMemUsage: 1.11 GB (1187388491) - RowsProduced: 24.90M (24904227) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 44s824ms - TotalStorageWaitTime: 565.104ms - TotalThreadsInvoluntaryContextSwitches: 15.03K (15028) - TotalThreadsTotalWallClockTime: 10m4s - TotalThreadsSysTime: 2s140ms - TotalThreadsUserTime: 28s474ms - TotalThreadsVoluntaryContextSwitches: 48.04K (48037) Fragment Instance Lifecycle Timings: - ExecTime: 53s142ms - ExecTreeExecTime: 406.954ms - OpenTime: 32.661ms - ExecTreeOpenTime: 41.475us - PrepareTime: 25.684ms - ExecTreePrepareTime: 298.674us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s888ms, non-child: 8s888ms, % non-child: 100.00%) - BytesSent: 509.00 MB (533725492) - NetworkThroughput(*): 276.47 MB/sec - OverallThroughput: 57.26 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.90M (24904227) - SerializeBatchTime: 7s038ms - TransmitDataRPCTime: 1s841ms - UncompressedRowBatchSize: 878.86 MB (921553699) CodeGen:(Total: 55.888ms, non-child: 55.888ms, % non-child: 100.00%) - CodegenTime: 1.372ms - CompileTime: 9.653ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.266ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 23.439ms HDFS_SCAN_NODE (id=1):(Total: 342.662ms, non-child: 342.662ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:2/506.80 MB 9:3/760.17 MB 7:1/253.39 MB 8:5/1.24 GB 1:2/260.77 MB 5:3/760.18 MB 4:1/253.40 MB 2:1/253.40 MB 0:1/253.38 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 921.45 MB, 921.45 MB, 921.45 MB, 921.45 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1004.98 MB, 1.02 GB, 1.06 GB, 1.06 GB, 1.19 GB, 1.23 GB, 1.23 GB, 1.31 GB, 1.39 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 5.057ms ; Min: 2.538ms ; Max: 7.669ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.40 - BytesRead: 1.47 GB (1579317980) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1579317980) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1579317980) - DecompressionTime: 9s119ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1186892587) - PerReadThreadRawHdfsThroughput: 1.42 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 33.847ms - RowBatchQueuePutWaitTime: 8m49s - RowsRead: 124.77M (124770123) - RowsReturned: 24.90M (24904227) - RowsReturnedRate: 72.68 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.93K (10927) - ScannerThreadsTotalWallClockTime: 9m11s - MaterializeTupleTime(*): 11s036ms - ScannerThreadsSysTime: 1s210ms - ScannerThreadsUserTime: 20s349ms - ScannerThreadsVoluntaryContextSwitches: 24.52K (24517) - TotalRawHdfsReadTime(*): 1s035ms - TotalReadThroughput: 28.42 MB/sec Instance 5c46a32203f83096:fcd5b8210000000b (host=vb0222.halxg.cloudera.com:22000):(Total: 52s753ms, non-child: 43s630ms, % non-child: 82.71%) Hdfs split stats (:<# splits>/): 6:1/253.38 MB 3:3/514.23 MB 10:1/253.41 MB 4:3/760.21 MB 9:1/253.37 MB 8:2/506.80 MB 7:2/506.80 MB 5:2/506.79 MB 1:1/253.40 MB 2:3/760.16 MB MemoryUsage(1s000ms): 595.87 MB, 1.01 GB, 1.01 GB, 1.01 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.06 GB, 1016.92 MB, 959.76 MB, 933.65 MB, 926.80 MB, 925.44 MB, 907.58 MB, 887.15 MB, 858.73 MB, 811.20 MB, 779.57 MB, 755.22 MB, 719.58 MB, 701.45 MB, 694.64 MB, 672.07 MB, 622.09 MB, 596.06 MB, 567.25 MB, 606.56 MB, 562.60 MB, 516.77 MB, 500.53 MB, 470.56 MB, 504.49 MB, 512.35 MB, 536.50 MB, 567.33 MB, 589.31 MB, 586.66 MB, 609.79 MB, 574.07 MB, 546.31 MB, 496.28 MB, 456.79 MB, 413.20 MB, 369.85 MB, 344.50 MB, 294.79 MB, 204.93 MB, 181.05 MB, 119.47 MB, 60.82 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 9, 8, 7, 7, 7, 7, 7, 6, 4, 4, 3, 2 - AverageThreadTokens: 11.38 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1182555971) - PerHostPeakMemUsage: 1.10 GB (1182555971) - RowsProduced: 24.52M (24523519) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 44s645ms - TotalStorageWaitTime: 521.380ms - TotalThreadsInvoluntaryContextSwitches: 14.81K (14815) - TotalThreadsTotalWallClockTime: 10m - TotalThreadsSysTime: 2s070ms - TotalThreadsUserTime: 27s791ms - TotalThreadsVoluntaryContextSwitches: 47.30K (47299) Fragment Instance Lifecycle Timings: - ExecTime: 52s694ms - ExecTreeExecTime: 350.722ms - OpenTime: 32.743ms - ExecTreeOpenTime: 42.930us - PrepareTime: 26.124ms - ExecTreePrepareTime: 308.612us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s778ms, non-child: 8s778ms, % non-child: 100.00%) - BytesSent: 501.25 MB (525595582) - NetworkThroughput(*): 258.73 MB/sec - OverallThroughput: 57.10 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.52M (24523519) - SerializeBatchTime: 6s831ms - TransmitDataRPCTime: 1s937ms - UncompressedRowBatchSize: 865.43 MB (907466011) CodeGen:(Total: 56.342ms, non-child: 56.342ms, % non-child: 100.00%) - CodegenTime: 1.455ms - CompileTime: 9.709ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.310ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 23.810ms HDFS_SCAN_NODE (id=1):(Total: 287.250ms, non-child: 287.250ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:1/253.38 MB 3:3/514.23 MB 10:1/253.41 MB 4:3/760.21 MB 9:1/253.37 MB 8:2/506.80 MB 7:2/506.80 MB 5:2/506.79 MB 1:1/253.40 MB 2:3/760.16 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:99.06% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0.9434% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 502.63 MB, 921.49 MB, 921.49 MB, 921.49 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1005.02 MB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.14 GB, 1.19 GB, 1.23 GB, 1.27 GB, 1.31 GB, 1.35 GB, 1.43 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 4.834ms ; Min: 2.662ms ; Max: 7.782ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.08 - AverageScannerThreadConcurrency: 10.38 - BytesRead: 1.47 GB (1579318713) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1579318713) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1579318713) - DecompressionTime: 8s976ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 10 (10) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1182308672) - PerReadThreadRawHdfsThroughput: 1.39 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 23.242ms - RowBatchQueuePutWaitTime: 8m46s - RowsRead: 124.77M (124773003) - RowsReturned: 24.52M (24523519) - RowsReturnedRate: 85.37 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.93K (10931) - ScannerThreadsTotalWallClockTime: 9m7s - MaterializeTupleTime(*): 10s817ms - ScannerThreadsSysTime: 1s254ms - ScannerThreadsUserTime: 19s877ms - ScannerThreadsVoluntaryContextSwitches: 24.09K (24085) - TotalRawHdfsReadTime(*): 1s060ms - TotalReadThroughput: 28.42 MB/sec Instance 5c46a32203f83096:fcd5b8210000000a (host=vb0228.halxg.cloudera.com:22000):(Total: 52s366ms, non-child: 43s469ms, % non-child: 83.01%) Hdfs split stats (:<# splits>/): 6:1/253.41 MB 4:2/506.76 MB 2:1/253.42 MB 10:2/445.01 MB 8:3/760.18 MB 3:1/187.22 MB 7:2/506.79 MB 5:4/1013.55 MB 0:3/760.18 MB MemoryUsage(1s000ms): 549.00 MB, 1.07 GB, 1.07 GB, 1.08 GB, 1.08 GB, 1.09 GB, 1.09 GB, 1.09 GB, 1.06 GB, 1.02 GB, 1021.11 MB, 968.15 MB, 914.26 MB, 892.19 MB, 880.88 MB, 869.51 MB, 849.92 MB, 819.54 MB, 776.18 MB, 768.60 MB, 781.00 MB, 735.78 MB, 704.44 MB, 696.95 MB, 663.03 MB, 614.69 MB, 625.97 MB, 636.70 MB, 636.71 MB, 615.37 MB, 585.68 MB, 604.45 MB, 659.36 MB, 717.47 MB, 692.47 MB, 680.62 MB, 652.80 MB, 686.66 MB, 674.16 MB, 617.95 MB, 563.55 MB, 512.83 MB, 497.90 MB, 475.51 MB, 427.13 MB, 399.73 MB, 378.27 MB, 330.31 MB, 294.30 MB, 216.90 MB, 168.62 MB, 119.09 MB ThreadUsage(1s000ms): 7, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 11, 9, 8, 8, 8, 8, 8, 8, 7, 7, 5, 4, 3 - AverageThreadTokens: 11.37 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.09 GB (1169881239) - PerHostPeakMemUsage: 1.09 GB (1169881239) - RowsProduced: 24.20M (24199028) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 44s400ms - TotalStorageWaitTime: 585.237ms - TotalThreadsInvoluntaryContextSwitches: 14.63K (14633) - TotalThreadsTotalWallClockTime: 9m59s - TotalThreadsSysTime: 2s253ms - TotalThreadsUserTime: 28s133ms - TotalThreadsVoluntaryContextSwitches: 46.74K (46745) Fragment Instance Lifecycle Timings: - ExecTime: 52s306ms - ExecTreeExecTime: 355.594ms - OpenTime: 33.587ms - ExecTreeOpenTime: 47.636us - PrepareTime: 26.692ms - ExecTreePrepareTime: 319.628us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s547ms, non-child: 8s547ms, % non-child: 100.00%) - BytesSent: 494.62 MB (518648130) - NetworkThroughput(*): 266.57 MB/sec - OverallThroughput: 57.87 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.20M (24199028) - SerializeBatchTime: 6s683ms - TransmitDataRPCTime: 1s855ms - UncompressedRowBatchSize: 853.98 MB (895458584) CodeGen:(Total: 57.705ms, non-child: 57.705ms, % non-child: 100.00%) - CodegenTime: 1.466ms - CompileTime: 10.143ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.665ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 24.329ms HDFS_SCAN_NODE (id=1):(Total: 291.603ms, non-child: 291.603ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:1/253.41 MB 4:2/506.76 MB 2:1/253.42 MB 10:2/445.01 MB 8:3/760.18 MB 3:1/187.22 MB 7:2/506.79 MB 5:4/1013.55 MB 0:3/760.18 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 490.26 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 980.53 MB, 1022.29 MB, 1.04 GB, 1.04 GB, 1.04 GB, 1.04 GB, 1.04 GB, 1.04 GB, 1.10 GB, 1.14 GB, 1.18 GB, 1.18 GB, 1.18 GB, 1.26 GB, 1.35 GB, 1.43 GB, 1.43 GB, 1.43 GB, 1.43 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB - FooterProcessingTime: (Avg: 5.406ms ; Min: 2.789ms ; Max: 8.176ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.47 - BytesRead: 1.51 GB (1619983151) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.51 GB (1619983151) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.51 GB (1619983151) - DecompressionTime: 9s256ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.09 GB (1169667732) - PerReadThreadRawHdfsThroughput: 1.41 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 37.290ms - RowBatchQueuePutWaitTime: 8m44s - RowsRead: 128.00M (128001189) - RowsReturned: 24.20M (24199028) - RowsReturnedRate: 82.99 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.74K (10736) - ScannerThreadsTotalWallClockTime: 9m6s - MaterializeTupleTime(*): 11s193ms - ScannerThreadsSysTime: 1s328ms - ScannerThreadsUserTime: 20s398ms - ScannerThreadsVoluntaryContextSwitches: 23.81K (23809) - TotalRawHdfsReadTime(*): 1s069ms - TotalReadThroughput: 29.43 MB/sec Instance 5c46a32203f83096:fcd5b8210000000c (host=vb0220.halxg.cloudera.com:22000):(Total: 52s231ms, non-child: 43s224ms, % non-child: 82.75%) Hdfs split stats (:<# splits>/): 8:2/506.79 MB 5:2/506.80 MB 10:1/253.39 MB 9:2/506.77 MB 7:1/253.41 MB 2:3/732.64 MB 3:4/1013.58 MB 0:3/760.18 MB 1:1/253.41 MB MemoryUsage(1s000ms): 1.07 GB, 1.07 GB, 1.07 GB, 1.08 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.06 GB, 1.02 GB, 1009.48 MB, 973.96 MB, 932.52 MB, 898.72 MB, 875.07 MB, 845.34 MB, 825.92 MB, 799.21 MB, 762.74 MB, 712.58 MB, 671.80 MB, 629.61 MB, 572.03 MB, 634.22 MB, 625.13 MB, 677.51 MB, 697.80 MB, 655.40 MB, 705.06 MB, 768.67 MB, 715.44 MB, 759.04 MB, 747.83 MB, 788.91 MB, 768.31 MB, 721.17 MB, 682.98 MB, 677.61 MB, 658.59 MB, 626.25 MB, 565.36 MB, 497.36 MB, 451.66 MB, 427.97 MB, 387.03 MB, 342.01 MB, 320.71 MB, 285.66 MB, 226.33 MB, 181.87 MB, 134.78 MB, 81.95 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 12, 11, 10, 8, 8, 8, 7, 7, 7, 7, 5, 5, 4, 2 - AverageThreadTokens: 11.46 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1183204545) - PerHostPeakMemUsage: 1.10 GB (1183204545) - RowsProduced: 24.28M (24277670) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 44s060ms - TotalStorageWaitTime: 631.800ms - TotalThreadsInvoluntaryContextSwitches: 14.16K (14157) - TotalThreadsTotalWallClockTime: 9m56s - TotalThreadsSysTime: 2s314ms - TotalThreadsUserTime: 28s074ms - TotalThreadsVoluntaryContextSwitches: 46.64K (46644) Fragment Instance Lifecycle Timings: - ExecTime: 52s171ms - ExecTreeExecTime: 355.063ms - OpenTime: 33.548ms - ExecTreeOpenTime: 50.386us - PrepareTime: 26.344ms - ExecTreePrepareTime: 371.024us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s656ms, non-child: 8s656ms, % non-child: 100.00%) - BytesSent: 496.22 MB (520319489) - NetworkThroughput(*): 281.42 MB/sec - OverallThroughput: 57.33 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.28M (24277670) - SerializeBatchTime: 6s883ms - TransmitDataRPCTime: 1s763ms - UncompressedRowBatchSize: 856.75 MB (898368646) CodeGen:(Total: 57.290ms, non-child: 57.290ms, % non-child: 100.00%) - CodegenTime: 1.516ms - CompileTime: 10.188ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.589ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 23.948ms HDFS_SCAN_NODE (id=1):(Total: 294.076ms, non-child: 294.076ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 8:2/506.79 MB 5:2/506.80 MB 10:1/253.39 MB 9:2/506.77 MB 7:1/253.41 MB 2:3/732.64 MB 3:4/1013.58 MB 0:3/760.18 MB 1:1/253.41 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:98.08% 1:1.923% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 993.29 MB, 1.05 GB, 1.09 GB, 1.17 GB, 1.21 GB, 1.21 GB, 1.30 GB, 1.38 GB, 1.38 GB, 1.46 GB, 1.48 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB - FooterProcessingTime: (Avg: 4.999ms ; Min: 2.629ms ; Max: 7.981ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 10.46 - BytesRead: 1.54 GB (1654662021) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.54 GB (1654662021) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.54 GB (1654662021) - DecompressionTime: 9s296ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1182723012) - PerReadThreadRawHdfsThroughput: 1.42 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 35.494ms - RowBatchQueuePutWaitTime: 8m42s - RowsRead: 130.75M (130748945) - RowsReturned: 24.28M (24277670) - RowsReturnedRate: 82.56 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 9.82K (9818) - ScannerThreadsTotalWallClockTime: 9m4s - MaterializeTupleTime(*): 11s060ms - ScannerThreadsSysTime: 1s282ms - ScannerThreadsUserTime: 20s415ms - ScannerThreadsVoluntaryContextSwitches: 23.82K (23824) - TotalRawHdfsReadTime(*): 1s087ms - TotalReadThroughput: 30.35 MB/sec Instance 5c46a32203f83096:fcd5b8210000000e (host=vb0206.halxg.cloudera.com:22000):(Total: 52s119ms, non-child: 43s205ms, % non-child: 82.90%) Hdfs split stats (:<# splits>/): 2:2/506.78 MB 9:3/760.18 MB 6:2/506.81 MB 7:1/253.39 MB 5:2/506.80 MB 4:3/760.17 MB 1:2/506.77 MB 0:3/760.20 MB MemoryUsage(1s000ms): 1.09 GB, 1.09 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.11 GB, 1.10 GB, 1.07 GB, 1.05 GB, 1018.03 MB, 954.51 MB, 929.18 MB, 912.30 MB, 896.93 MB, 890.94 MB, 857.80 MB, 817.10 MB, 754.69 MB, 724.56 MB, 680.97 MB, 662.64 MB, 639.69 MB, 594.71 MB, 644.64 MB, 637.26 MB, 664.55 MB, 675.76 MB, 688.53 MB, 675.67 MB, 647.69 MB, 632.05 MB, 604.22 MB, 571.44 MB, 613.98 MB, 651.37 MB, 619.80 MB, 567.46 MB, 529.79 MB, 487.22 MB, 454.50 MB, 410.71 MB, 399.68 MB, 383.39 MB, 343.55 MB, 297.89 MB, 269.18 MB, 245.25 MB, 221.30 MB, 184.76 MB, 119.75 MB, 54.79 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 11, 11, 9, 8, 7, 7, 7, 7, 7, 6, 6, 6, 6, 6, 5, 3, 2 - AverageThreadTokens: 11.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1192059941) - PerHostPeakMemUsage: 1.11 GB (1192059941) - RowsProduced: 24.21M (24210513) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 44s030ms - TotalStorageWaitTime: 571.045ms - TotalThreadsInvoluntaryContextSwitches: 15.62K (15617) - TotalThreadsTotalWallClockTime: 9m34s - TotalThreadsSysTime: 2s187ms - TotalThreadsUserTime: 27s829ms - TotalThreadsVoluntaryContextSwitches: 46.48K (46484) Fragment Instance Lifecycle Timings: - ExecTime: 52s059ms - ExecTreeExecTime: 359.923ms - OpenTime: 34.010ms - ExecTreeOpenTime: 49.762us - PrepareTime: 25.894ms - ExecTreePrepareTime: 310.949us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s556ms, non-child: 8s556ms, % non-child: 100.00%) - BytesSent: 494.84 MB (518880607) - NetworkThroughput(*): 274.51 MB/sec - OverallThroughput: 57.83 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.21M (24210513) - SerializeBatchTime: 6s745ms - TransmitDataRPCTime: 1s802ms - UncompressedRowBatchSize: 854.38 MB (895883569) CodeGen:(Total: 57.297ms, non-child: 57.297ms, % non-child: 100.00%) - CodegenTime: 1.504ms - CompileTime: 10.344ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.938ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 23.452ms HDFS_SCAN_NODE (id=1):(Total: 300.398ms, non-child: 300.398ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:2/506.78 MB 9:3/760.18 MB 6:2/506.81 MB 7:1/253.39 MB 5:2/506.80 MB 4:3/760.17 MB 1:2/506.77 MB 0:3/760.20 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 18 out of 18 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:72 BytesRead(1s000ms): 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1.06 GB, 1.10 GB, 1.18 GB, 1.22 GB, 1.26 GB, 1.31 GB, 1.31 GB, 1.31 GB, 1.31 GB, 1.31 GB, 1.39 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 5.497ms ; Min: 2.850ms ; Max: 8.165ms ; Number of samples: 18) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.00 - BytesRead: 1.47 GB (1576537765) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1576537765) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1576537765) - DecompressionTime: 9s050ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 8 (8) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1191747098) - PerReadThreadRawHdfsThroughput: 1.52 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 33.105ms - RowBatchQueuePutWaitTime: 8m20s - RowsRead: 124.58M (124580613) - RowsReturned: 24.21M (24210513) - RowsReturnedRate: 80.59 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 11.69K (11693) - ScannerThreadsTotalWallClockTime: 8m42s - MaterializeTupleTime(*): 10s935ms - ScannerThreadsSysTime: 1s319ms - ScannerThreadsUserTime: 19s874ms - ScannerThreadsVoluntaryContextSwitches: 23.82K (23819) - TotalRawHdfsReadTime(*): 963.267ms - TotalReadThroughput: 28.91 MB/sec Instance 5c46a32203f83096:fcd5b8210000000d (host=vb0208.halxg.cloudera.com:22000):(Total: 51s907ms, non-child: 43s133ms, % non-child: 83.10%) Hdfs split stats (:<# splits>/): 9:1/253.39 MB 7:1/234.34 MB 5:2/506.78 MB 4:3/760.21 MB 8:3/760.15 MB 1:2/506.82 MB 3:1/253.41 MB 2:4/1013.57 MB 0:2/506.80 MB MemoryUsage(1s000ms): 934.73 MB, 1.09 GB, 1.09 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.07 GB, 1.03 GB, 1006.50 MB, 958.39 MB, 926.33 MB, 903.06 MB, 891.44 MB, 863.84 MB, 819.73 MB, 800.91 MB, 740.78 MB, 724.39 MB, 671.27 MB, 636.64 MB, 596.68 MB, 601.57 MB, 609.32 MB, 554.93 MB, 647.06 MB, 640.70 MB, 611.38 MB, 657.72 MB, 714.09 MB, 689.98 MB, 728.55 MB, 761.38 MB, 707.41 MB, 690.36 MB, 682.41 MB, 645.65 MB, 604.94 MB, 548.79 MB, 511.27 MB, 486.12 MB, 469.08 MB, 436.26 MB, 398.68 MB, 363.65 MB, 327.27 MB, 288.14 MB, 217.68 MB, 171.48 MB, 123.29 MB, 64.56 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 11, 11, 11, 11, 10, 10, 8, 8, 8, 8, 8, 8, 8, 8, 7, 5, 4, 4, 2 - AverageThreadTokens: 11.18 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1185807681) - PerHostPeakMemUsage: 1.10 GB (1185807681) - RowsProduced: 24.00M (24004509) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 43s857ms - TotalStorageWaitTime: 607.549ms - TotalThreadsInvoluntaryContextSwitches: 14.76K (14763) - TotalThreadsTotalWallClockTime: 9m39s - TotalThreadsSysTime: 2s243ms - TotalThreadsUserTime: 28s802ms - TotalThreadsVoluntaryContextSwitches: 46.22K (46218) Fragment Instance Lifecycle Timings: - ExecTime: 51s846ms - ExecTreeExecTime: 363.700ms - OpenTime: 33.694ms - ExecTreeOpenTime: 60.039us - PrepareTime: 27.290ms - ExecTreePrepareTime: 314.001us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s415ms, non-child: 8s415ms, % non-child: 100.00%) - BytesSent: 490.64 MB (514476323) - NetworkThroughput(*): 289.74 MB/sec - OverallThroughput: 58.30 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.00M (24004509) - SerializeBatchTime: 6s712ms - TransmitDataRPCTime: 1s693ms - UncompressedRowBatchSize: 847.11 MB (888260613) CodeGen:(Total: 58.397ms, non-child: 58.397ms, % non-child: 100.00%) - CodegenTime: 1.466ms - CompileTime: 10.143ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.750ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 24.933ms HDFS_SCAN_NODE (id=1):(Total: 300.426ms, non-child: 300.426ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 9:1/253.39 MB 7:1/234.34 MB 5:2/506.78 MB 4:3/760.21 MB 8:3/760.15 MB 1:2/506.82 MB 3:1/253.41 MB 2:4/1013.57 MB 0:2/506.80 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:99.04% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0.9615% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 811.37 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 996.05 MB, 1.01 GB, 1.05 GB, 1.05 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.30 GB, 1.38 GB, 1.38 GB, 1.46 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB - FooterProcessingTime: (Avg: 5.432ms ; Min: 2.632ms ; Max: 8.945ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.06 - AverageScannerThreadConcurrency: 10.18 - BytesRead: 1.54 GB (1657536250) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.54 GB (1657536250) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.54 GB (1657536250) - DecompressionTime: 9s511ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1185327918) - PerReadThreadRawHdfsThroughput: 1.40 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 34.533ms - RowBatchQueuePutWaitTime: 8m25s - RowsRead: 130.98M (130980854) - RowsReturned: 24.00M (24004509) - RowsReturnedRate: 79.90 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.74K (10744) - ScannerThreadsTotalWallClockTime: 8m47s - MaterializeTupleTime(*): 11s451ms - ScannerThreadsSysTime: 1s304ms - ScannerThreadsUserTime: 20s973ms - ScannerThreadsVoluntaryContextSwitches: 23.62K (23618) - TotalRawHdfsReadTime(*): 1s100ms - TotalReadThroughput: 30.40 MB/sec Instance 5c46a32203f83096:fcd5b82100000003 (host=vb0212.halxg.cloudera.com:22000):(Total: 51s327ms, non-child: 42s622ms, % non-child: 83.04%) Hdfs split stats (:<# splits>/): 10:1/253.38 MB 1:3/760.18 MB 5:1/253.39 MB 4:2/506.79 MB 7:3/760.20 MB 0:1/253.40 MB 6:2/506.82 MB 3:2/506.82 MB 2:3/760.21 MB MemoryUsage(1s000ms): 1020.01 MB, 1.09 GB, 1.09 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.11 GB, 1.10 GB, 1.08 GB, 1.07 GB, 1.03 GB, 980.03 MB, 936.83 MB, 917.77 MB, 904.60 MB, 900.75 MB, 873.12 MB, 847.52 MB, 788.69 MB, 752.57 MB, 712.57 MB, 686.22 MB, 656.71 MB, 659.89 MB, 691.96 MB, 675.35 MB, 677.41 MB, 650.49 MB, 609.60 MB, 609.21 MB, 596.63 MB, 564.02 MB, 539.12 MB, 524.80 MB, 621.73 MB, 683.25 MB, 650.54 MB, 618.40 MB, 578.17 MB, 528.80 MB, 447.56 MB, 432.60 MB, 397.41 MB, 340.74 MB, 320.95 MB, 308.28 MB, 276.85 MB, 223.63 MB, 186.30 MB, 158.60 MB, 76.49 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 11, 10, 9, 7, 7, 6, 6, 6, 6, 6, 5, 5, 4, 2 - AverageThreadTokens: 11.13 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1189089353) - PerHostPeakMemUsage: 1.11 GB (1189089353) - RowsProduced: 23.47M (23465800) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 43s486ms - TotalStorageWaitTime: 544.631ms - TotalThreadsInvoluntaryContextSwitches: 14.36K (14363) - TotalThreadsTotalWallClockTime: 9m31s - TotalThreadsSysTime: 1s985ms - TotalThreadsUserTime: 27s576ms - TotalThreadsVoluntaryContextSwitches: 45.25K (45251) Fragment Instance Lifecycle Timings: - ExecTime: 51s267ms - ExecTreeExecTime: 375.250ms - OpenTime: 32.615ms - ExecTreeOpenTime: 41.141us - PrepareTime: 26.756ms - ExecTreePrepareTime: 300.896us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s335ms, non-child: 8s335ms, % non-child: 100.00%) - BytesSent: 479.63 MB (502926019) - NetworkThroughput(*): 271.34 MB/sec - OverallThroughput: 57.54 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 23.47M (23465800) - SerializeBatchTime: 6s559ms - TransmitDataRPCTime: 1s767ms - UncompressedRowBatchSize: 828.10 MB (868326276) CodeGen:(Total: 56.881ms, non-child: 56.881ms, % non-child: 100.00%) - CodegenTime: 1.439ms - CompileTime: 9.593ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.254ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 24.503ms HDFS_SCAN_NODE (id=1):(Total: 312.368ms, non-child: 312.368ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 10:1/253.38 MB 1:3/760.18 MB 5:1/253.39 MB 4:2/506.79 MB 7:3/760.20 MB 0:1/253.40 MB 6:2/506.82 MB 3:2/506.82 MB 2:3/760.21 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 18 out of 18 Hdfs Read Thread Concurrency Bucket: 0:98.06% 1:0.9709% 2:0% 3:0.9709% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:72 BytesRead(1s000ms): 900.44 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1.02 GB, 1.06 GB, 1.09 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.18 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.39 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 4.708ms ; Min: 2.608ms ; Max: 6.936ms ; Number of samples: 18) - AverageHdfsReadThreadConcurrency: 0.04 - AverageScannerThreadConcurrency: 10.13 - BytesRead: 1.47 GB (1576589528) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1576589528) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1576589528) - DecompressionTime: 9s006ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1188875846) - PerReadThreadRawHdfsThroughput: 1.50 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 30.949ms - RowBatchQueuePutWaitTime: 8m19s - RowsRead: 124.58M (124581157) - RowsReturned: 23.47M (23465800) - RowsReturnedRate: 75.12 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 10.50K (10502) - ScannerThreadsTotalWallClockTime: 8m40s - MaterializeTupleTime(*): 10s790ms - ScannerThreadsSysTime: 1s129ms - ScannerThreadsUserTime: 19s966ms - ScannerThreadsVoluntaryContextSwitches: 23.03K (23031) - TotalRawHdfsReadTime(*): 979.137ms - TotalReadThroughput: 29.20 MB/sec Instance 5c46a32203f83096:fcd5b82100000004 (host=vb0214.halxg.cloudera.com:22000):(Total: 50s552ms, non-child: 43s077ms, % non-child: 85.21%) Hdfs split stats (:<# splits>/): 7:1/253.39 MB 4:2/506.80 MB 0:1/253.39 MB 8:2/506.80 MB 9:2/506.77 MB 3:1/253.37 MB 5:3/760.16 MB 2:4/959.93 MB 1:3/760.19 MB MemoryUsage(1s000ms): 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.07 GB, 1.05 GB, 1.00 GB, 974.21 MB, 932.86 MB, 914.16 MB, 907.90 MB, 897.45 MB, 874.61 MB, 857.49 MB, 829.33 MB, 779.47 MB, 754.11 MB, 721.41 MB, 693.10 MB, 676.58 MB, 692.29 MB, 698.88 MB, 687.93 MB, 696.22 MB, 694.34 MB, 666.17 MB, 649.09 MB, 623.26 MB, 575.19 MB, 573.67 MB, 593.93 MB, 574.24 MB, 545.62 MB, 668.78 MB, 639.01 MB, 702.41 MB, 691.59 MB, 669.70 MB, 698.26 MB, 617.17 MB, 552.67 MB, 535.37 MB, 508.23 MB, 472.51 MB, 448.82 MB, 425.16 MB, 396.50 MB, 363.07 MB, 275.90 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 10, 8, 8, 8, 8, 8, 8, 8, 7, 6 - AverageThreadTokens: 11.93 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1187690489) - PerHostPeakMemUsage: 10.87 GB (11666282693) - RowsProduced: 25.73M (25731516) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 43s816ms - TotalStorageWaitTime: 500.958ms - TotalThreadsInvoluntaryContextSwitches: 12.31K (12314) - TotalThreadsTotalWallClockTime: 10m1s - TotalThreadsSysTime: 1s637ms - TotalThreadsUserTime: 22s449ms - TotalThreadsVoluntaryContextSwitches: 49.55K (49548) Fragment Instance Lifecycle Timings: - ExecTime: 50s492ms - ExecTreeExecTime: 339.646ms - OpenTime: 33.246ms - ExecTreeOpenTime: 52.544us - PrepareTime: 26.934ms - ExecTreePrepareTime: 407.485us DataStreamSender (dst_id=4):(Total: 7s121ms, non-child: 7s121ms, % non-child: 100.00%) - BytesSent: 525.93 MB (551479392) - NetworkThroughput(*): 381.49 MB/sec - OverallThroughput: 73.85 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 25.73M (25731516) - SerializeBatchTime: 5s736ms - TransmitDataRPCTime: 1s378ms - UncompressedRowBatchSize: 908.06 MB (952166628) CodeGen:(Total: 57.717ms, non-child: 57.717ms, % non-child: 100.00%) - CodegenTime: 1.298ms - CompileTime: 9.763ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.663ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 24.767ms HDFS_SCAN_NODE (id=1):(Total: 296.012ms, non-child: 296.012ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:1/253.39 MB 4:2/506.80 MB 0:1/253.39 MB 8:2/506.80 MB 9:2/506.77 MB 3:1/253.37 MB 5:3/760.16 MB 2:4/959.93 MB 1:3/760.19 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:99.01% 1:0.9901% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1002.30 MB, 1.04 GB, 1.06 GB, 1.06 GB, 1.10 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.18 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.39 GB, 1.39 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB - FooterProcessingTime: (Avg: 3.370ms ; Min: 1.903ms ; Max: 6.424ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.01 - AverageScannerThreadConcurrency: 10.93 - BytesRead: 1.53 GB (1645560524) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.53 GB (1645560524) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.53 GB (1645560524) - DecompressionTime: 7s164ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1187443190) - PerReadThreadRawHdfsThroughput: 1.59 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 26.042ms - RowBatchQueuePutWaitTime: 8m54s - RowsRead: 130.03M (130033379) - RowsReturned: 25.73M (25731516) - RowsReturnedRate: 86.93 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 8.54K (8541) - ScannerThreadsTotalWallClockTime: 9m11s - MaterializeTupleTime(*): 8s746ms - ScannerThreadsSysTime: 1s025ms - ScannerThreadsUserTime: 15s953ms - ScannerThreadsVoluntaryContextSwitches: 25.28K (25283) - TotalRawHdfsReadTime(*): 964.559ms - TotalReadThroughput: 31.08 MB/sec Instance 5c46a32203f83096:fcd5b82100000007 (host=vb0234.halxg.cloudera.com:22000):(Total: 50s544ms, non-child: 42s265ms, % non-child: 83.62%) Hdfs split stats (:<# splits>/): 4:1/253.41 MB 5:2/506.79 MB 9:2/506.77 MB 1:1/253.39 MB 7:3/760.20 MB 6:3/760.17 MB 3:3/760.18 MB 2:2/258.65 MB 0:2/506.79 MB MemoryUsage(1s000ms): 1.09 GB, 1.09 GB, 1.09 GB, 1.09 GB, 1.11 GB, 1.10 GB, 1.11 GB, 1.10 GB, 1.07 GB, 1.04 GB, 1016.45 MB, 964.23 MB, 928.02 MB, 908.67 MB, 891.86 MB, 891.18 MB, 849.84 MB, 827.67 MB, 768.78 MB, 728.53 MB, 690.15 MB, 664.97 MB, 625.05 MB, 639.53 MB, 722.66 MB, 718.37 MB, 715.99 MB, 700.59 MB, 660.24 MB, 613.22 MB, 597.46 MB, 574.44 MB, 562.66 MB, 630.62 MB, 680.96 MB, 655.04 MB, 622.13 MB, 602.89 MB, 574.82 MB, 501.49 MB, 448.53 MB, 410.65 MB, 391.46 MB, 330.50 MB, 310.10 MB, 278.27 MB, 210.77 MB, 188.31 MB, 137.89 MB, 66.49 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 11, 10, 10, 8, 7, 7, 7, 6, 6, 6, 5, 5, 4, 2 - AverageThreadTokens: 11.13 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1191156077) - PerHostPeakMemUsage: 1.11 GB (1191156077) - RowsProduced: 23.36M (23361849) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 42s880ms - TotalStorageWaitTime: 604.906ms - TotalThreadsInvoluntaryContextSwitches: 14.37K (14369) - TotalThreadsTotalWallClockTime: 9m23s - TotalThreadsSysTime: 2s084ms - TotalThreadsUserTime: 27s196ms - TotalThreadsVoluntaryContextSwitches: 44.86K (44864) Fragment Instance Lifecycle Timings: - ExecTime: 50s485ms - ExecTreeExecTime: 318.874ms - OpenTime: 32.534ms - ExecTreeOpenTime: 43.100us - PrepareTime: 26.921ms - ExecTreePrepareTime: 300.714us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 7s964ms, non-child: 7s964ms, % non-child: 100.00%) - BytesSent: 477.51 MB (500701167) - NetworkThroughput(*): 309.42 MB/sec - OverallThroughput: 59.95 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 23.36M (23361849) - SerializeBatchTime: 6s412ms - TransmitDataRPCTime: 1s543ms - UncompressedRowBatchSize: 824.43 MB (864479689) CodeGen:(Total: 56.944ms, non-child: 56.944ms, % non-child: 100.00%) - CodegenTime: 1.417ms - CompileTime: 9.582ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.203ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 24.649ms HDFS_SCAN_NODE (id=1):(Total: 258.376ms, non-child: 258.376ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:1/253.41 MB 5:2/506.79 MB 9:2/506.77 MB 1:1/253.39 MB 7:3/760.20 MB 6:3/760.17 MB 3:3/760.18 MB 2:2/258.65 MB 0:2/506.79 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:99.01% 1:0.9901% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1004.30 MB, 1.02 GB, 1.14 GB, 1.18 GB, 1.23 GB, 1.23 GB, 1.23 GB, 1.23 GB, 1.23 GB, 1.23 GB, 1.25 GB, 1.39 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 5.322ms ; Min: 3.033ms ; Max: 8.064ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.01 - AverageScannerThreadConcurrency: 10.13 - BytesRead: 1.47 GB (1578597766) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1578597766) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1578597766) - DecompressionTime: 8s957ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1190676314) - PerReadThreadRawHdfsThroughput: 1.47 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 20.020ms - RowBatchQueuePutWaitTime: 8m11s - RowsRead: 124.71M (124712354) - RowsReturned: 23.36M (23361849) - RowsReturnedRate: 90.42 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.70K (10702) - ScannerThreadsTotalWallClockTime: 8m33s - MaterializeTupleTime(*): 10s713ms - ScannerThreadsSysTime: 1s236ms - ScannerThreadsUserTime: 19s691ms - ScannerThreadsVoluntaryContextSwitches: 22.91K (22910) - TotalRawHdfsReadTime(*): 1s002ms - TotalReadThroughput: 29.81 MB/sec ================================================================================ ================================================================================ SUMMARY: +-----------------+--------+----------+---------+----------+---------+-----------+ | Operator | #Hosts | Avg Time | Std Dev | Max Time | #Rows | Est #Rows | +-----------------+--------+----------+---------+----------+---------+-----------+ | 06:AGGREGATE | 1 | 245.25us | 0ns | 245.25us | 1 | 1 | | 05:EXCHANGE | 1 | 19.79us | 0ns | 19.79us | 1 | 1 | | 03:AGGREGATE | 1 | 2.49ms | 0ns | 2.49ms | 1 | 1 | | 02:HASH JOIN | 1 | 71.45s | 0ns | 71.45s | 0 | 3 | | |--04:EXCHANGE | 1 | 7.72s | 0ns | 7.72s | 350.01M | 180.00M | | | 01:SCAN HDFS | 14 | 534.29ms | 0ns | 3.55s | 350.01M | 180.00M | | 00:SCAN HDFS | 1 | 6.92ms | 0ns | 6.92ms | 0 | 3 | +-----------------+--------+----------+---------+----------+---------+-----------+ PROFILE: Query (id=f5478537469ada54:a960c8d800000000): Summary: Session ID: 87433499cc381244:12db1bd5509dd7ac Session Type: BEESWAX Start Time: 2017-02-17 04:38:41.929035000 End Time: Query Type: QUERY Query State: CREATED Query Status: OK Impala Version: impalad version 2.9.0-SNAPSHOT RELEASE (build 9842bfcdbff9dfbb2fb1a8da8eda98ed2d56e76a) User: jenkins Connected User: jenkins Delegated User: Network Address: 10.17.193.6:38992 Default Db: tpch_300_parquet Sql Statement: -- Description : Large broadcast join with no rows qualifying from the probe side. -- Target test case : Broadcast large amounts of data over the network while stressing the -- exchange operator along with HashTable build. SELECT /* +straight_join */ count(*) FROM nation JOIN /* +broadcast */ lineitem ON l_orderkey = n_nationkey WHERE l_partkey > 0 AND l_suppkey > 0 AND l_linenumber > 0 AND n_regionkey < 0 AND l_orderkey < 350000000 Coordinator: vb0204.halxg.cloudera.com:22000 Query Options (non default): NUM_SCANNER_THREADS=12,SYNC_DDL=1,MAX_NUM_RUNTIME_FILTERS=50 Plan: ---------------- Estimated Per-Host Requirements: Memory=6.24GB VCores=2 PLAN-ROOT SINK | 06:AGGREGATE [FINALIZE] | output: count:merge(*) | hosts=1 per-host-mem=unavailable | tuple-ids=2 row-size=8B cardinality=1 | 05:EXCHANGE [UNPARTITIONED] | hosts=1 per-host-mem=unavailable | tuple-ids=2 row-size=8B cardinality=1 | 03:AGGREGATE | output: count(*) | hosts=1 per-host-mem=10.00MB | tuple-ids=2 row-size=8B cardinality=1 | 02:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: n_nationkey = l_orderkey | runtime filters: RF000 <- l_orderkey | hosts=1 per-host-mem=5.90GB | tuple-ids=0,1 row-size=48B cardinality=3 | |--04:EXCHANGE [BROADCAST] | | hosts=14 per-host-mem=0B | | tuple-ids=1 row-size=32B cardinality=179998909 | | | 01:SCAN HDFS [tpch_300_parquet.lineitem, RANDOM] | partitions=1/1 files=264 size=64.36GB | predicates: l_linenumber > 0, l_orderkey < 350000000, l_partkey > 0, l_suppkey > 0 | table stats: 1799989091 rows total | column stats: all | hosts=14 per-host-mem=352.00MB | tuple-ids=1 row-size=32B cardinality=179998909 | 00:SCAN HDFS [tpch_300_parquet.nation, RANDOM] partitions=1/1 files=1 size=2.19KB predicates: n_regionkey < 0, tpch_300_parquet.nation.n_nationkey < 350000000 runtime filters: RF000 -> n_nationkey table stats: 25 rows total column stats: all hosts=1 per-host-mem=32.00MB tuple-ids=0 row-size=16B cardinality=3 ---------------- Estimated Per-Host Mem: 6705060455 Estimated Per-Host VCores: 2 Request Pool: root.jenkins Admission result: Admitted immediately Planner Timeline: 6.228ms - Analysis finished: 1.879ms (1.879ms) - Equivalence classes computed: 1.995ms (115.927us) - Single node plan created: 3.363ms (1.368ms) - Runtime filters computed: 3.476ms (112.759us) - Distributed plan created: 3.709ms (233.273us) - Lineage info computed: 3.834ms (125.018us) - Planning finished: 6.228ms (2.393ms) Query Timeline: 1m23s - Query submitted: 46.532us (46.532us) - Planning finished: 8.738ms (8.692ms) - Submit for admission: 10.013ms (1.275ms) - Completed admission: 10.121ms (108.096us) - Ready to start 16 fragment instances: 10.446ms (324.984us) - All 16 fragment instances started: 1s921ms (1s910ms) - Rows available: 1m23s (1m21s) - First row fetched: 1m23s (29.589ms) - ComputeScanRangeAssignmentTimer: 587.325us ImpalaServer: - ClientFetchWaitTimer: 29.598ms - RowMaterializationTimer: 4.020ms Execution Profile f5478537469ada54:a960c8d800000000:(Total: 1m23s, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 1 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed Enabled ---------------------------------------------------------------------------------------------------------------------------- 0 2 0 1 LOCAL false 0 (1) N/A N/A true Fragment instance start latencies: Count: 16, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 2ms, 90th %-ile: 2ms, 95th %-ile: 291ms, 99.9th %-ile: 1s909ms Per Node Peak Memory Usage: vb0206.halxg.cloudera.com:22000(1.11 GB) vb0220.halxg.cloudera.com:22000(1.10 GB) vb0228.halxg.cloudera.com:22000(1.07 GB) vb0222.halxg.cloudera.com:22000(1.11 GB) vb0224.halxg.cloudera.com:22000(1.11 GB) vb0232.halxg.cloudera.com:22000(1.11 GB) vb0234.halxg.cloudera.com:22000(1.11 GB) vb0218.halxg.cloudera.com:22000(1.10 GB) vb0214.halxg.cloudera.com:22000(26.43 GB) vb0212.halxg.cloudera.com:22000(1.11 GB) vb0216.halxg.cloudera.com:22000(1.10 GB) vb0208.halxg.cloudera.com:22000(1.10 GB) vb0230.halxg.cloudera.com:22000(1.11 GB) vb0204.halxg.cloudera.com:22000(1.10 GB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Coordinator Fragment F02: Instance f5478537469ada54:a960c8d800000000 (host=vb0204.halxg.cloudera.com:22000):(Total: 1m23s, non-child: 30.347ms, % non-child: 0.04%) MemoryUsage(2s000ms): 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 28.01 KB (28680) - PerHostPeakMemUsage: 1.10 GB (1182540513) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1m23s - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 1m23s - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 5 (5) Fragment Instance Lifecycle Timings: - ExecTime: 29.709ms - ExecTreeExecTime: 6.896us - OpenTime: 1m23s - ExecTreeOpenTime: 1m23s - PrepareTime: 21.578ms - ExecTreePrepareTime: 120.364us PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 21.163ms, non-child: 21.163ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 20.623ms AGGREGATION_NODE (id=6):(Total: 1m23s, non-child: 245.253us, % non-child: 0.00%) - BuildTime: 8.601us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 20.00 KB (20480) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=5):(Total: 1m23s, non-child: 19.793us, % non-child: 0.00%) BytesReceived(2s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 - BytesReceived: 16.00 B (16) - ConvertRowBatchTime: 1.734us - DeserializeRowBatchTimer: 76.197us - FirstBatchArrivalWaitTime: 1m23s - PeakMemoryUsage: 0 - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 1m20s, non-child: 120.457ms, % non-child: 0.15%) split sizes: min: 2.19 KB, max: 2.19 KB, avg: 2.19 KB, stddev: 0 completion times: min:1m21s max:1m21s mean: 1m21s stddev:0.000ns execution rates: min:27.00 B/sec max:27.00 B/sec mean:27.00 B/sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 1.98 - BloomFilterBytes: 16.00 MB (16777216) - PeakMemoryUsage: 26.43 GB (28380916821) - PerHostPeakMemUsage: 26.43 GB (28380916821) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1s401ms - TotalNetworkSendTime: 742.771us - TotalStorageWaitTime: 5.651ms - TotalThreadsInvoluntaryContextSwitches: 8.51K (8505) - TotalThreadsTotalWallClockTime: 2m38s - TotalThreadsSysTime: 18s314ms - TotalThreadsUserTime: 1m - TotalThreadsVoluntaryContextSwitches: 6.53K (6534) Fragment Instance Lifecycle Timings: - ExecTime: 1.129ms - ExecTreeExecTime: 4.950us - OpenTime: 1m19s - ExecTreeOpenTime: 1m19s - PrepareTime: 590.328ms - ExecTreePrepareTime: 260.716us DataStreamSender (dst_id=5):(Total: 443.068ms, non-child: 443.068ms, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 482.28 KB/sec - OverallThroughput: 36.00 B/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.888us - TransmitDataRPCTime: 32.398us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 326.705ms, non-child: 326.705ms, % non-child: 100.00%) - CodegenTime: 1.921ms - CompileTime: 86.020ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 90 (90) - NumInstructions: 2.93K (2928) - OptimizationTime: 211.979ms - PeakMemoryUsage: 1.43 MB (1499136) - PrepareTime: 28.138ms AGGREGATION_NODE (id=3):(Total: 1m19s, non-child: 2.494ms, % non-child: 0.00%) - BuildTime: 2.320ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 24.00 KB (24576) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=2):(Total: 1m19s, non-child: 1m11s, % non-child: 90.24%) - BuildRows: 350.01M (350008706) - BuildTime: 1m10s - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 26.42 GB (28364049664) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 1.344us - RowsReturned: 0 (0) - RowsReturnedRate: 0 Hash Join Builder (join_node_id=2): - BuildRowsPartitionTime: 41s858ms - BuildRowsPartitioned: 350.01M (350008706) - GetNewBlockTime: 9.089ms - HashBuckets: 536.87M (536870912) - HashCollisions: 0 (0) - HashTablesBuildTime: 28s267ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 26.42 GB (28364002432) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=4):(Total: 7s716ms, non-child: 7s716ms, % non-child: 100.00%) - BytesReceived: 6.99 GB (7501460242) - ConvertRowBatchTime: 5s971ms - DeserializeRowBatchTimer: 16s886ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 350.01M (350008706) - RowsReturnedRate: 45.36 M/sec - SendersBlockedTimer: 48s371ms - SendersBlockedTotalTimer(*): 9m22s HDFS_SCAN_NODE (id=0):(Total: 6.917ms, non-child: 6.917ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 2.44 KB (2497) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.44 KB (2497) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.44 KB (2497) - DecompressionTime: 7.041us - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDisksAccessed: 1 (1) - NumRowGroups: 1 (1) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 73.00 KB (74752) - PerReadThreadRawHdfsThroughput: 69.20 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 6.068ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 25 (25) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 6.134ms - MaterializeTupleTime(*): 31.896us - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 4 (4) - TotalRawHdfsReadTime(*): 34.411us - TotalReadThroughput: 0.00 /sec Filter 0 (16.00 MB): - Rows processed: 25 (25) - Rows rejected: 0 (0) - Rows total: 25 (25) Fragment F00: Instance f5478537469ada54:a960c8d80000000f (host=vb0214.halxg.cloudera.com:22000):(Total: 1m20s, non-child: 120.457ms, % non-child: 0.15%) Hdfs split stats (:<# splits>/): 6:1/2.19 KB Filter 0 arrival: 51s226ms MemoryUsage(2s000ms): 86.50 MB, 492.26 MB, 940.26 MB, 1.36 GB, 1.73 GB, 2.20 GB, 2.61 GB, 3.07 GB, 3.48 GB, 3.92 GB, 4.36 GB, 4.79 GB, 5.20 GB, 5.61 GB, 6.01 GB, 6.45 GB, 6.86 GB, 7.32 GB, 7.73 GB, 8.17 GB, 8.61 GB, 9.04 GB, 9.48 GB, 9.89 GB, 10.36 GB, 10.85 GB, 11.98 GB, 13.24 GB, 14.59 GB, 15.58 GB, 16.58 GB, 17.59 GB, 18.58 GB, 19.56 GB, 20.54 GB, 21.70 GB, 22.76 GB, 23.94 GB, 25.12 GB, 25.81 GB ThreadUsage(2s000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1 - AverageThreadTokens: 1.98 - BloomFilterBytes: 16.00 MB (16777216) - PeakMemoryUsage: 26.43 GB (28380916821) - PerHostPeakMemUsage: 26.43 GB (28380916821) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1s401ms - TotalNetworkSendTime: 742.771us - TotalStorageWaitTime: 5.651ms - TotalThreadsInvoluntaryContextSwitches: 8.51K (8505) - TotalThreadsTotalWallClockTime: 2m38s - TotalThreadsSysTime: 18s314ms - TotalThreadsUserTime: 1m - TotalThreadsVoluntaryContextSwitches: 6.53K (6534) Fragment Instance Lifecycle Timings: - ExecTime: 1.129ms - ExecTreeExecTime: 4.950us - OpenTime: 1m19s - ExecTreeOpenTime: 1m19s - PrepareTime: 590.328ms - ExecTreePrepareTime: 260.716us DataStreamSender (dst_id=5):(Total: 443.068ms, non-child: 443.068ms, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 482.28 KB/sec - OverallThroughput: 36.00 B/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.888us - TransmitDataRPCTime: 32.398us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 326.705ms, non-child: 326.705ms, % non-child: 100.00%) - CodegenTime: 1.921ms - CompileTime: 86.020ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 90 (90) - NumInstructions: 2.93K (2928) - OptimizationTime: 211.979ms - PeakMemoryUsage: 1.43 MB (1499136) - PrepareTime: 28.138ms AGGREGATION_NODE (id=3):(Total: 1m19s, non-child: 2.494ms, % non-child: 0.00%) ExecOption: Codegen Enabled - BuildTime: 2.320ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 24.00 KB (24576) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=2):(Total: 1m19s, non-child: 1m11s, % non-child: 90.24%) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 350.01M (350008706) - BuildTime: 1m10s - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 26.42 GB (28364049664) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 1.344us - RowsReturned: 0 (0) - RowsReturnedRate: 0 Hash Join Builder (join_node_id=2): ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 0 of 1 Runtime Filter Published, 1 Disabled - BuildRowsPartitionTime: 41s858ms - BuildRowsPartitioned: 350.01M (350008706) - GetNewBlockTime: 9.089ms - HashBuckets: 536.87M (536870912) - HashCollisions: 0 (0) - HashTablesBuildTime: 28s267ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 26.42 GB (28364002432) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=4):(Total: 7s716ms, non-child: 7s716ms, % non-child: 100.00%) BytesReceived(2s000ms): 32.35 MB, 273.42 MB, 552.08 MB, 833.08 MB, 1.09 GB, 1.37 GB, 1.65 GB, 1.93 GB, 2.21 GB, 2.49 GB, 2.78 GB, 3.06 GB, 3.33 GB, 3.59 GB, 3.86 GB, 4.13 GB, 4.41 GB, 4.69 GB, 4.97 GB, 5.26 GB, 5.54 GB, 5.82 GB, 6.10 GB, 6.38 GB, 6.66 GB, 6.93 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB, 6.99 GB - BytesReceived: 6.99 GB (7501460242) - ConvertRowBatchTime: 5s971ms - DeserializeRowBatchTimer: 16s886ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 350.01M (350008706) - RowsReturnedRate: 45.36 M/sec - SendersBlockedTimer: 48s371ms - SendersBlockedTotalTimer(*): 9m22s HDFS_SCAN_NODE (id=0):(Total: 6.917ms, non-child: 6.917ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:1/2.19 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:2 BytesRead(2s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 - FooterProcessingTime: (Avg: 3.605ms ; Min: 3.605ms ; Max: 3.605ms ; Number of samples: 1) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 2.44 KB (2497) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.44 KB (2497) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.44 KB (2497) - DecompressionTime: 7.041us - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDisksAccessed: 1 (1) - NumRowGroups: 1 (1) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 73.00 KB (74752) - PerReadThreadRawHdfsThroughput: 69.20 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 6.068ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 25 (25) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 6.134ms - MaterializeTupleTime(*): 31.896us - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 4 (4) - TotalRawHdfsReadTime(*): 34.411us - TotalReadThroughput: 0.00 /sec Filter 0 (16.00 MB): - Rows processed: 25 (25) - Rows rejected: 0 (0) - Rows total: 25 (25) Averaged Fragment F01:(Total: 52s641ms, non-child: 43s440ms, % non-child: 82.52%) split sizes: min: 4.45 GB, max: 4.70 GB, avg: 4.60 GB, stddev: 110.56 MB completion times: min:49s821ms max:54s136ms mean: 52s642ms stddev:1s174ms execution rates: min:86.19 MB/sec max:92.37 MB/sec mean:89.44 MB/sec stddev:1.78 MB/sec num instances: 14 - AverageThreadTokens: 11.60 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1184549546) - PerHostPeakMemUsage: 1.82 GB (1949465136) - RowsProduced: 25.00M (25000621) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 44s340ms - TotalStorageWaitTime: 580.060ms - TotalThreadsInvoluntaryContextSwitches: 14.37K (14370) - TotalThreadsTotalWallClockTime: 10m10s - TotalThreadsSysTime: 2s096ms - TotalThreadsUserTime: 27s830ms - TotalThreadsVoluntaryContextSwitches: 47.97K (47967) Fragment Instance Lifecycle Timings: - ExecTime: 52s340ms - ExecTreeExecTime: 366.179ms - OpenTime: 33.921ms - ExecTreeOpenTime: 49.833us - PrepareTime: 266.623ms - ExecTreePrepareTime: 115.914ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 100 (100) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 786.29 MB (824480329) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s577ms, non-child: 8s577ms, % non-child: 100.00%) - BytesSent: 511.00 MB (535818588) - NetworkThroughput(*): 287.72 MB/sec - OverallThroughput: 59.82 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 25.00M (25000621) - SerializeBatchTime: 6s777ms - TransmitDataRPCTime: 1s791ms - UncompressedRowBatchSize: 882.26 MB (925120683) CodeGen:(Total: 89.697ms, non-child: 89.697ms, % non-child: 100.00%) - CodegenTime: 1.512ms - CompileTime: 10.121ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.888ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 56.094ms HDFS_SCAN_NODE (id=1):(Total: 534.286ms, non-child: 534.286ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 10.68 - BytesRead: 1.52 GB (1627133960) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.52 GB (1627133960) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.52 GB (1627133960) - DecompressionTime: 9s033ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1184216811) - PerReadThreadRawHdfsThroughput: 1.42 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 31.543ms - RowBatchQueuePutWaitTime: 8m57s - RowsRead: 128.57M (128570649) - RowsReturned: 25.00M (25000621) - RowsReturnedRate: 77.49 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 10.51K (10508) - ScannerThreadsTotalWallClockTime: 9m18s - MaterializeTupleTime(*): 10s872ms - ScannerThreadsSysTime: 1s231ms - ScannerThreadsUserTime: 19s978ms - ScannerThreadsVoluntaryContextSwitches: 24.48K (24480) - TotalRawHdfsReadTime(*): 1s067ms - TotalReadThroughput: 29.50 MB/sec Fragment F01: Instance f5478537469ada54:a960c8d800000006 (host=vb0218.halxg.cloudera.com:22000):(Total: 54s135ms, non-child: 43s830ms, % non-child: 80.96%) Hdfs split stats (:<# splits>/): 8:1/253.38 MB 1:1/253.39 MB 9:1/253.38 MB 10:1/253.40 MB 5:2/506.79 MB 2:5/1.24 GB 7:2/506.80 MB 6:2/506.81 MB 3:3/760.16 MB 0:1/253.39 MB MemoryUsage(1s000ms): 1.07 GB, 1.09 GB, 1.09 GB, 1.09 GB, 1.09 GB, 1.10 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.06 GB, 999.36 MB, 946.71 MB, 931.22 MB, 931.34 MB, 931.32 MB, 928.17 MB, 907.30 MB, 875.44 MB, 824.22 MB, 795.68 MB, 764.06 MB, 745.01 MB, 736.95 MB, 720.11 MB, 707.80 MB, 681.04 MB, 654.71 MB, 616.51 MB, 589.80 MB, 544.36 MB, 510.11 MB, 469.50 MB, 428.29 MB, 401.18 MB, 372.97 MB, 417.32 MB, 436.43 MB, 613.19 MB, 583.23 MB, 719.84 MB, 756.32 MB, 686.71 MB, 642.19 MB, 596.71 MB, 564.11 MB, 540.71 MB, 511.84 MB, 480.36 MB, 449.06 MB, 409.25 MB, 353.56 MB, 240.79 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 9, 8, 8, 8, 8, 8, 8, 8, 8, 8, 7 - AverageThreadTokens: 11.96 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1180392221) - PerHostPeakMemUsage: 1.10 GB (1180392221) - RowsProduced: 27.36M (27356338) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 45s103ms - TotalStorageWaitTime: 615.751ms - TotalThreadsInvoluntaryContextSwitches: 15.29K (15294) - TotalThreadsTotalWallClockTime: 10m44s - TotalThreadsSysTime: 2s177ms - TotalThreadsUserTime: 29s858ms - TotalThreadsVoluntaryContextSwitches: 52.00K (52003) Fragment Instance Lifecycle Timings: - ExecTime: 54s076ms - ExecTreeExecTime: 403.211ms - OpenTime: 34.396ms - ExecTreeOpenTime: 42.789us - PrepareTime: 24.569ms - ExecTreePrepareTime: 310.673us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 9s912ms, non-child: 9s912ms, % non-child: 100.00%) - BytesSent: 559.15 MB (586311676) - NetworkThroughput(*): 251.23 MB/sec - OverallThroughput: 56.41 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 27.36M (27356338) - SerializeBatchTime: 7s677ms - TransmitDataRPCTime: 2s225ms - UncompressedRowBatchSize: 965.40 MB (1012291378) CodeGen:(Total: 56.560ms, non-child: 56.560ms, % non-child: 100.00%) - CodegenTime: 1.236ms - CompileTime: 9.713ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 23.943ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 22.363ms HDFS_SCAN_NODE (id=1):(Total: 335.835ms, non-child: 335.835ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 8:1/253.38 MB 1:1/253.39 MB 9:1/253.38 MB 10:1/253.40 MB 5:2/506.79 MB 2:5/1.24 GB 7:2/506.80 MB 6:2/506.81 MB 3:3/760.16 MB 0:1/253.39 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:98.15% 1:1.852% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 987.05 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1.06 GB, 1.10 GB, 1.31 GB, 1.31 GB, 1.49 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB - FooterProcessingTime: (Avg: 5.677ms ; Min: 2.444ms ; Max: 17.429ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 10.96 - BytesRead: 1.55 GB (1664143743) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.55 GB (1664143743) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.55 GB (1664143743) - DecompressionTime: 9s455ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 10 (10) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1180345634) - PerReadThreadRawHdfsThroughput: 1.40 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 30.957ms - RowBatchQueuePutWaitTime: 9m28s - RowsRead: 131.50M (131502589) - RowsReturned: 27.36M (27356338) - RowsReturnedRate: 81.46 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 11.19K (11192) - ScannerThreadsTotalWallClockTime: 9m50s - MaterializeTupleTime(*): 11s423ms - ScannerThreadsSysTime: 1s242ms - ScannerThreadsUserTime: 21s070ms - ScannerThreadsVoluntaryContextSwitches: 26.74K (26736) - TotalRawHdfsReadTime(*): 1s106ms - TotalReadThroughput: 29.39 MB/sec Instance f5478537469ada54:a960c8d800000005 (host=vb0204.halxg.cloudera.com:22000):(Total: 54s053ms, non-child: 44s242ms, % non-child: 81.85%) Hdfs split stats (:<# splits>/): 5:1/253.39 MB 2:1/253.41 MB 7:2/506.80 MB 6:1/253.41 MB 1:2/506.79 MB 8:2/506.78 MB 3:5/1.24 GB 4:4/1013.63 MB 0:1/253.40 MB MemoryUsage(1s000ms): 549.12 MB, 1.07 GB, 1.07 GB, 1.07 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.08 GB, 1.05 GB, 1019.85 MB, 950.14 MB, 926.97 MB, 918.38 MB, 914.06 MB, 901.89 MB, 889.99 MB, 854.23 MB, 818.08 MB, 788.59 MB, 741.57 MB, 717.67 MB, 694.77 MB, 693.96 MB, 677.84 MB, 714.31 MB, 676.02 MB, 663.35 MB, 642.89 MB, 591.94 MB, 548.61 MB, 503.18 MB, 488.44 MB, 518.70 MB, 543.00 MB, 536.50 MB, 528.05 MB, 514.60 MB, 516.23 MB, 623.55 MB, 657.49 MB, 619.45 MB, 591.49 MB, 570.68 MB, 498.44 MB, 457.14 MB, 456.70 MB, 429.45 MB, 394.83 MB, 354.23 MB, 314.29 MB, 213.61 MB ThreadUsage(1s000ms): 7, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 10, 9, 8, 8, 7, 7, 7, 7, 7, 7, 7, 6 - AverageThreadTokens: 11.59 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1182524129) - PerHostPeakMemUsage: 1.10 GB (1182540513) - RowsProduced: 27.05M (27045946) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 45s287ms - TotalStorageWaitTime: 586.732ms - TotalThreadsInvoluntaryContextSwitches: 15.34K (15335) - TotalThreadsTotalWallClockTime: 10m33s - TotalThreadsSysTime: 2s105ms - TotalThreadsUserTime: 29s679ms - TotalThreadsVoluntaryContextSwitches: 51.62K (51620) Fragment Instance Lifecycle Timings: - ExecTime: 53s996ms - ExecTreeExecTime: 427.265ms - OpenTime: 33.525ms - ExecTreeOpenTime: 51.036us - PrepareTime: 22.962ms - ExecTreePrepareTime: 299.146us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 9s396ms, non-child: 9s396ms, % non-child: 100.00%) - BytesSent: 552.79 MB (579639503) - NetworkThroughput(*): 278.01 MB/sec - OverallThroughput: 58.83 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 27.05M (27045946) - SerializeBatchTime: 7s398ms - TransmitDataRPCTime: 1s988ms - UncompressedRowBatchSize: 954.44 MB (1000805662) CodeGen:(Total: 54.194ms, non-child: 54.194ms, % non-child: 100.00%) - CodegenTime: 1.280ms - CompileTime: 9.788ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.794ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 21.065ms HDFS_SCAN_NODE (id=1):(Total: 360.198ms, non-child: 360.198ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:1/253.39 MB 2:1/253.41 MB 7:2/506.80 MB 6:1/253.41 MB 1:2/506.79 MB 8:2/506.78 MB 3:5/1.24 GB 4:4/1013.63 MB 0:1/253.40 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:99.07% 1:0.9346% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 501.18 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.14 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.26 GB, 1.42 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB - FooterProcessingTime: (Avg: 5.112ms ; Min: 2.700ms ; Max: 7.703ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.01 - AverageScannerThreadConcurrency: 10.79 - BytesRead: 1.55 GB (1664181733) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.55 GB (1664181733) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.55 GB (1664181733) - DecompressionTime: 9s467ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1182010574) - PerReadThreadRawHdfsThroughput: 1.47 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 33.387ms - RowBatchQueuePutWaitTime: 9m17s - RowsRead: 131.50M (131502331) - RowsReturned: 27.05M (27045946) - RowsReturnedRate: 75.09 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 11.29K (11285) - ScannerThreadsTotalWallClockTime: 9m39s - MaterializeTupleTime(*): 11s386ms - ScannerThreadsSysTime: 1s193ms - ScannerThreadsUserTime: 21s145ms - ScannerThreadsVoluntaryContextSwitches: 26.39K (26391) - TotalRawHdfsReadTime(*): 1s052ms - TotalReadThroughput: 29.39 MB/sec Instance f5478537469ada54:a960c8d800000004 (host=vb0214.halxg.cloudera.com:22000):(Total: 53s968ms, non-child: 41s332ms, % non-child: 76.59%) Hdfs split stats (:<# splits>/): 7:1/253.39 MB 4:2/506.80 MB 0:1/253.39 MB 8:2/506.80 MB 9:2/506.77 MB 3:1/253.37 MB 5:3/760.16 MB 2:4/959.93 MB 1:3/760.19 MB MemoryUsage(1s000ms): 0, 0, 0, 557.96 MB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.08 GB, 1.08 GB, 1.06 GB, 1.00 GB, 972.99 MB, 925.74 MB, 918.56 MB, 919.56 MB, 907.20 MB, 888.27 MB, 866.52 MB, 841.43 MB, 803.49 MB, 765.38 MB, 739.02 MB, 719.76 MB, 695.28 MB, 740.72 MB, 729.89 MB, 726.74 MB, 705.55 MB, 669.79 MB, 639.67 MB, 618.97 MB, 591.11 MB, 562.63 MB, 507.70 MB, 566.17 MB, 534.26 MB, 498.95 MB, 597.69 MB, 599.58 MB, 629.70 MB, 647.27 MB, 627.27 MB, 637.60 MB, 698.30 MB, 646.25 MB, 596.00 MB, 571.76 MB, 507.99 MB, 468.28 MB, 451.84 MB, 389.45 MB, 193.51 MB ThreadUsage(1s000ms): 1, 1, 1, 7, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 11, 9, 8, 8, 8, 8, 7, 5 - AverageThreadTokens: 11.50 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1183589243) - PerHostPeakMemUsage: 11.08 GB (11892391120) - RowsProduced: 25.73M (25731516) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 42s659ms - TotalStorageWaitTime: 517.826ms - TotalThreadsInvoluntaryContextSwitches: 12.77K (12765) - TotalThreadsTotalWallClockTime: 10m19s - TotalThreadsSysTime: 1s587ms - TotalThreadsUserTime: 25s791ms - TotalThreadsVoluntaryContextSwitches: 48.64K (48639) Fragment Instance Lifecycle Timings: - ExecTime: 50s530ms - ExecTreeExecTime: 369.564ms - OpenTime: 37.622ms - ExecTreeOpenTime: 61.972us - PrepareTime: 3s400ms - ExecTreePrepareTime: 1s618ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 1.41K (1408) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 10.75 GB (11542724608) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s566ms, non-child: 8s566ms, % non-child: 100.00%) - BytesSent: 525.93 MB (551479392) - NetworkThroughput(*): 305.86 MB/sec - OverallThroughput: 61.40 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 25.73M (25731516) - SerializeBatchTime: 6s838ms - TransmitDataRPCTime: 1s719ms - UncompressedRowBatchSize: 908.06 MB (952166628) CodeGen:(Total: 519.172ms, non-child: 519.172ms, % non-child: 100.00%) - CodegenTime: 3.117ms - CompileTime: 10.240ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 24.726ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 483.036ms HDFS_SCAN_NODE (id=1):(Total: 3s551ms, non-child: 3s551ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:1/253.39 MB 4:2/506.80 MB 0:1/253.39 MB 8:2/506.80 MB 9:2/506.77 MB 3:1/253.37 MB 5:3/760.16 MB 2:4/959.93 MB 1:3/760.19 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 0, 0, 0, 501.16 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1002.32 MB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.26 GB, 1.31 GB, 1.35 GB, 1.39 GB, 1.39 GB, 1.43 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB, 1.53 GB - FooterProcessingTime: (Avg: 3.501ms ; Min: 2.693ms ; Max: 4.942ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.23 - BytesRead: 1.53 GB (1645560524) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.53 GB (1645560524) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.53 GB (1645560524) - DecompressionTime: 8s137ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1182974312) - PerReadThreadRawHdfsThroughput: 1.49 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 30.004ms - RowBatchQueuePutWaitTime: 9m9s - RowsRead: 130.03M (130033379) - RowsReturned: 25.73M (25731516) - RowsReturnedRate: 7.25 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 8.79K (8791) - ScannerThreadsTotalWallClockTime: 9m29s - MaterializeTupleTime(*): 9s828ms - ScannerThreadsSysTime: 1s015ms - ScannerThreadsUserTime: 18s028ms - ScannerThreadsVoluntaryContextSwitches: 24.60K (24602) - TotalRawHdfsReadTime(*): 1s030ms - TotalReadThroughput: 29.06 MB/sec Instance f5478537469ada54:a960c8d800000002 (host=vb0216.halxg.cloudera.com:22000):(Total: 53s745ms, non-child: 46s277ms, % non-child: 86.10%) Hdfs split stats (:<# splits>/): 1:1/253.38 MB 3:2/506.78 MB 10:1/253.39 MB 7:2/506.78 MB 9:5/1.24 GB 6:1/253.41 MB 5:2/506.78 MB 2:1/253.41 MB 0:4/1013.59 MB MemoryUsage(1s000ms): 1.09 GB, 1.09 GB, 1.09 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.06 GB, 1.03 GB, 1005.55 MB, 951.75 MB, 925.20 MB, 913.73 MB, 902.55 MB, 876.91 MB, 858.42 MB, 827.51 MB, 785.81 MB, 750.27 MB, 716.20 MB, 676.07 MB, 663.91 MB, 709.27 MB, 659.34 MB, 640.43 MB, 617.66 MB, 568.18 MB, 593.90 MB, 594.26 MB, 550.49 MB, 615.08 MB, 699.83 MB, 681.58 MB, 672.82 MB, 655.49 MB, 660.13 MB, 757.56 MB, 703.59 MB, 660.27 MB, 612.19 MB, 564.09 MB, 538.86 MB, 505.86 MB, 478.91 MB, 454.78 MB, 427.35 MB, 389.43 MB, 353.50 MB, 333.47 MB, 277.02 MB, 189.94 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 10, 9, 8, 8, 8, 8, 8, 8, 8, 8, 8, 7, 5 - AverageThreadTokens: 11.69 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1186257743) - PerHostPeakMemUsage: 1.10 GB (1186257743) - RowsProduced: 26.59M (26594701) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 46s972ms - TotalStorageWaitTime: 512.064ms - TotalThreadsInvoluntaryContextSwitches: 12.00K (12004) - TotalThreadsTotalWallClockTime: 10m27s - TotalThreadsSysTime: 1s799ms - TotalThreadsUserTime: 23s418ms - TotalThreadsVoluntaryContextSwitches: 51.47K (51471) Fragment Instance Lifecycle Timings: - ExecTime: 53s683ms - ExecTreeExecTime: 321.462ms - OpenTime: 34.104ms - ExecTreeOpenTime: 53.357us - PrepareTime: 27.347ms - ExecTreePrepareTime: 352.336us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 7s138ms, non-child: 7s138ms, % non-child: 100.00%) - BytesSent: 543.60 MB (570003508) - NetworkThroughput(*): 365.31 MB/sec - OverallThroughput: 76.15 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 26.59M (26594701) - SerializeBatchTime: 5s643ms - TransmitDataRPCTime: 1s488ms - UncompressedRowBatchSize: 938.52 MB (984107841) CodeGen:(Total: 58.884ms, non-child: 58.884ms, % non-child: 100.00%) - CodegenTime: 1.468ms - CompileTime: 10.605ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.730ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 24.981ms HDFS_SCAN_NODE (id=1):(Total: 271.040ms, non-child: 271.040ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:1/253.38 MB 3:2/506.78 MB 10:1/253.39 MB 7:2/506.78 MB 9:5/1.24 GB 6:1/253.41 MB 5:2/506.78 MB 2:1/253.41 MB 0:4/1013.59 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:99.07% 1:0.9346% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1002.34 MB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.06 GB, 1.11 GB, 1.14 GB, 1.14 GB, 1.26 GB, 1.39 GB, 1.39 GB, 1.39 GB, 1.39 GB, 1.43 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB - FooterProcessingTime: (Avg: 4.027ms ; Min: 1.585ms ; Max: 7.126ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.01 - AverageScannerThreadConcurrency: 10.69 - BytesRead: 1.55 GB (1664130351) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.55 GB (1664130351) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.55 GB (1664130351) - DecompressionTime: 7s471ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1185906938) - PerReadThreadRawHdfsThroughput: 1.54 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 27.367ms - RowBatchQueuePutWaitTime: 9m16s - RowsRead: 131.50M (131502389) - RowsReturned: 26.59M (26594701) - RowsReturnedRate: 98.12 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 8.37K (8365) - ScannerThreadsTotalWallClockTime: 9m34s - MaterializeTupleTime(*): 9s115ms - ScannerThreadsSysTime: 1s052ms - ScannerThreadsUserTime: 16s802ms - ScannerThreadsVoluntaryContextSwitches: 26.03K (26033) - TotalRawHdfsReadTime(*): 1s004ms - TotalReadThroughput: 29.66 MB/sec Instance f5478537469ada54:a960c8d800000009 (host=vb0224.halxg.cloudera.com:22000):(Total: 53s409ms, non-child: 44s357ms, % non-child: 83.05%) Hdfs split stats (:<# splits>/): 3:1/253.39 MB 1:1/253.41 MB 10:1/253.38 MB 9:1/253.40 MB 5:2/506.78 MB 0:2/506.77 MB 4:2/506.76 MB 8:2/506.78 MB 7:3/760.18 MB 6:2/481.03 MB 2:2/506.80 MB MemoryUsage(1s000ms): 559.06 MB, 1.09 GB, 1.09 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.07 GB, 1.04 GB, 1020.81 MB, 985.35 MB, 937.81 MB, 908.64 MB, 899.91 MB, 880.04 MB, 855.57 MB, 832.40 MB, 791.58 MB, 753.42 MB, 723.63 MB, 687.43 MB, 649.06 MB, 618.17 MB, 664.72 MB, 713.07 MB, 684.49 MB, 649.44 MB, 693.31 MB, 665.58 MB, 647.26 MB, 622.82 MB, 627.49 MB, 638.27 MB, 696.55 MB, 748.26 MB, 773.73 MB, 748.71 MB, 738.96 MB, 696.42 MB, 660.23 MB, 613.58 MB, 581.34 MB, 522.11 MB, 472.01 MB, 451.44 MB, 420.32 MB, 398.65 MB, 330.03 MB, 294.14 MB, 239.21 MB, 180.27 MB ThreadUsage(1s000ms): 7, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 11, 10, 9, 8, 8, 8, 8, 7, 7, 6, 5 - AverageThreadTokens: 11.72 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1192723541) - PerHostPeakMemUsage: 1.11 GB (1192723541) - RowsProduced: 25.26M (25256825) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 45s206ms - TotalStorageWaitTime: 610.444ms - TotalThreadsInvoluntaryContextSwitches: 14.39K (14394) - TotalThreadsTotalWallClockTime: 10m29s - TotalThreadsSysTime: 2s214ms - TotalThreadsUserTime: 28s756ms - TotalThreadsVoluntaryContextSwitches: 48.54K (48541) Fragment Instance Lifecycle Timings: - ExecTime: 53s349ms - ExecTreeExecTime: 369.308ms - OpenTime: 33.772ms - ExecTreeOpenTime: 53.003us - PrepareTime: 26.503ms - ExecTreePrepareTime: 330.889us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s695ms, non-child: 8s695ms, % non-child: 100.00%) - BytesSent: 516.24 MB (541321888) - NetworkThroughput(*): 286.55 MB/sec - OverallThroughput: 59.37 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 25.26M (25256825) - SerializeBatchTime: 6s884ms - TransmitDataRPCTime: 1s801ms - UncompressedRowBatchSize: 891.31 MB (934601201) CodeGen:(Total: 57.637ms, non-child: 57.637ms, % non-child: 100.00%) - CodegenTime: 1.508ms - CompileTime: 10.293ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.715ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 24.061ms HDFS_SCAN_NODE (id=1):(Total: 298.899ms, non-child: 298.899ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:1/253.39 MB 1:1/253.41 MB 10:1/253.38 MB 9:1/253.40 MB 5:2/506.78 MB 0:2/506.77 MB 4:2/506.76 MB 8:2/506.78 MB 7:3/760.18 MB 6:2/481.03 MB 2:2/506.80 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:99.06% 1:0.9434% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 501.14 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1002.27 MB, 1.06 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.26 GB, 1.32 GB, 1.42 GB, 1.50 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB - FooterProcessingTime: (Avg: 5.962ms ; Min: 2.686ms ; Max: 22.113ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.01 - AverageScannerThreadConcurrency: 10.82 - BytesRead: 1.54 GB (1655172689) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.54 GB (1655172689) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.54 GB (1655172689) - DecompressionTime: 9s405ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 11 (11) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1192144442) - PerReadThreadRawHdfsThroughput: 1.30 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 35.213ms - RowBatchQueuePutWaitTime: 9m13s - RowsRead: 130.80M (130797981) - RowsReturned: 25.26M (25256825) - RowsReturnedRate: 84.50 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.62K (10615) - ScannerThreadsTotalWallClockTime: 9m35s - MaterializeTupleTime(*): 11s276ms - ScannerThreadsSysTime: 1s344ms - ScannerThreadsUserTime: 20s678ms - ScannerThreadsVoluntaryContextSwitches: 24.67K (24666) - TotalRawHdfsReadTime(*): 1s188ms - TotalReadThroughput: 29.50 MB/sec Instance f5478537469ada54:a960c8d800000001 (host=vb0230.halxg.cloudera.com:22000):(Total: 53s048ms, non-child: 44s161ms, % non-child: 83.25%) Hdfs split stats (:<# splits>/): 9:1/253.41 MB 2:1/253.39 MB 5:2/506.78 MB 3:2/506.81 MB 6:1/253.38 MB 7:2/506.78 MB 4:2/506.81 MB 1:3/760.19 MB 0:5/1.24 GB MemoryUsage(1s000ms): 762.33 MB, 1.08 GB, 1.08 GB, 1.08 GB, 1.10 GB, 1.10 GB, 1.11 GB, 1.11 GB, 1.10 GB, 1.07 GB, 1.02 GB, 1017.76 MB, 975.65 MB, 941.77 MB, 918.22 MB, 893.35 MB, 878.05 MB, 843.87 MB, 825.16 MB, 788.53 MB, 749.93 MB, 710.69 MB, 684.26 MB, 621.67 MB, 639.26 MB, 650.77 MB, 612.48 MB, 656.23 MB, 650.33 MB, 593.83 MB, 716.14 MB, 707.73 MB, 690.75 MB, 715.29 MB, 744.72 MB, 784.44 MB, 784.09 MB, 758.24 MB, 731.40 MB, 700.03 MB, 664.32 MB, 623.68 MB, 574.43 MB, 510.27 MB, 473.82 MB, 439.36 MB, 421.97 MB, 364.21 MB, 326.14 MB, 289.14 MB, 260.71 MB, 187.98 MB, 106.79 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 12, 11, 11, 8, 8, 8, 8, 7, 7, 7, 7, 5, 3 - AverageThreadTokens: 11.73 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1190921123) - PerHostPeakMemUsage: 1.11 GB (1190921123) - RowsProduced: 25.08M (25076265) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 44s896ms - TotalStorageWaitTime: 593.866ms - TotalThreadsInvoluntaryContextSwitches: 15.35K (15346) - TotalThreadsTotalWallClockTime: 10m19s - TotalThreadsSysTime: 2s149ms - TotalThreadsUserTime: 28s866ms - TotalThreadsVoluntaryContextSwitches: 48.06K (48065) Fragment Instance Lifecycle Timings: - ExecTime: 52s988ms - ExecTreeExecTime: 344.417ms - OpenTime: 33.617ms - ExecTreeOpenTime: 64.997us - PrepareTime: 26.711ms - ExecTreePrepareTime: 194.977us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s549ms, non-child: 8s549ms, % non-child: 100.00%) - BytesSent: 512.53 MB (537431466) - NetworkThroughput(*): 303.73 MB/sec - OverallThroughput: 59.95 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 25.08M (25076265) - SerializeBatchTime: 6s853ms - TransmitDataRPCTime: 1s687ms - UncompressedRowBatchSize: 884.93 MB (927919769) CodeGen:(Total: 57.813ms, non-child: 57.813ms, % non-child: 100.00%) - CodegenTime: 1.368ms - CompileTime: 10.253ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.586ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 24.409ms HDFS_SCAN_NODE (id=1):(Total: 279.951ms, non-child: 279.951ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 9:1/253.41 MB 2:1/253.39 MB 5:2/506.78 MB 3:2/506.81 MB 6:1/253.38 MB 7:2/506.78 MB 4:2/506.81 MB 1:3/760.19 MB 0:5/1.24 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:98.11% 1:0.9434% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0.9434% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 648.47 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1002.36 MB, 1.02 GB, 1.06 GB, 1.06 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.31 GB, 1.31 GB, 1.31 GB, 1.39 GB, 1.47 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB, 1.55 GB - FooterProcessingTime: (Avg: 4.789ms ; Min: 2.685ms ; Max: 7.297ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.09 - AverageScannerThreadConcurrency: 10.73 - BytesRead: 1.55 GB (1664143238) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.55 GB (1664143238) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.55 GB (1664143238) - DecompressionTime: 9s463ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1190642072) - PerReadThreadRawHdfsThroughput: 1.38 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 29.994ms - RowBatchQueuePutWaitTime: 9m3s - RowsRead: 131.50M (131502184) - RowsReturned: 25.08M (25076265) - RowsReturnedRate: 89.57 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 11.59K (11591) - ScannerThreadsTotalWallClockTime: 9m26s - MaterializeTupleTime(*): 11s342ms - ScannerThreadsSysTime: 1s207ms - ScannerThreadsUserTime: 20s866ms - ScannerThreadsVoluntaryContextSwitches: 24.55K (24549) - TotalRawHdfsReadTime(*): 1s125ms - TotalReadThroughput: 29.94 MB/sec Instance f5478537469ada54:a960c8d800000008 (host=vb0232.halxg.cloudera.com:22000):(Total: 53s003ms, non-child: 43s884ms, % non-child: 82.80%) Hdfs split stats (:<# splits>/): 6:2/506.80 MB 9:3/760.17 MB 7:1/253.39 MB 8:5/1.24 GB 1:2/260.77 MB 5:3/760.18 MB 4:1/253.40 MB 2:1/253.40 MB 0:1/253.38 MB MemoryUsage(1s000ms): 1021.99 MB, 1021.99 MB, 1021.99 MB, 1022.35 MB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.07 GB, 1.04 GB, 1.00 GB, 992.76 MB, 947.06 MB, 920.28 MB, 895.79 MB, 881.10 MB, 868.68 MB, 846.15 MB, 821.97 MB, 790.48 MB, 758.76 MB, 699.13 MB, 666.27 MB, 638.12 MB, 617.83 MB, 589.71 MB, 672.09 MB, 684.11 MB, 658.00 MB, 708.94 MB, 680.53 MB, 653.11 MB, 686.89 MB, 699.43 MB, 714.90 MB, 678.22 MB, 661.44 MB, 683.33 MB, 687.56 MB, 640.16 MB, 600.97 MB, 560.41 MB, 517.45 MB, 473.66 MB, 403.03 MB, 382.07 MB, 366.50 MB, 321.94 MB, 267.41 MB, 226.80 MB, 189.77 MB, 71.71 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 11, 10, 9, 8, 7, 7, 7, 6, 6, 6, 5, 2 - AverageThreadTokens: 11.69 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1187760113) - PerHostPeakMemUsage: 1.11 GB (1187760113) - RowsProduced: 24.90M (24904227) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 44s750ms - TotalStorageWaitTime: 549.077ms - TotalThreadsInvoluntaryContextSwitches: 14.40K (14399) - TotalThreadsTotalWallClockTime: 10m20s - TotalThreadsSysTime: 2s145ms - TotalThreadsUserTime: 28s148ms - TotalThreadsVoluntaryContextSwitches: 47.82K (47817) Fragment Instance Lifecycle Timings: - ExecTime: 52s942ms - ExecTreeExecTime: 374.635ms - OpenTime: 33.161ms - ExecTreeOpenTime: 46.944us - PrepareTime: 27.299ms - ExecTreePrepareTime: 310.021us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s752ms, non-child: 8s752ms, % non-child: 100.00%) - BytesSent: 509.00 MB (533725492) - NetworkThroughput(*): 283.04 MB/sec - OverallThroughput: 58.15 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.90M (24904227) - SerializeBatchTime: 6s945ms - TransmitDataRPCTime: 1s798ms - UncompressedRowBatchSize: 878.86 MB (921553699) CodeGen:(Total: 57.942ms, non-child: 57.942ms, % non-child: 100.00%) - CodegenTime: 1.405ms - CompileTime: 10.037ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.365ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 25.014ms HDFS_SCAN_NODE (id=1):(Total: 308.009ms, non-child: 308.009ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:2/506.80 MB 9:3/760.17 MB 7:1/253.39 MB 8:5/1.24 GB 1:2/260.77 MB 5:3/760.18 MB 4:1/253.40 MB 2:1/253.40 MB 0:1/253.38 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 921.47 MB, 921.47 MB, 921.47 MB, 921.47 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1004.99 MB, 1.10 GB, 1.14 GB, 1.14 GB, 1.23 GB, 1.23 GB, 1.23 GB, 1.31 GB, 1.35 GB, 1.39 GB, 1.39 GB, 1.39 GB, 1.43 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 4.632ms ; Min: 1.718ms ; Max: 7.682ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.69 - BytesRead: 1.47 GB (1579317980) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1579317980) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1579317980) - DecompressionTime: 9s068ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1187542616) - PerReadThreadRawHdfsThroughput: 1.49 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 25.450ms - RowBatchQueuePutWaitTime: 9m6s - RowsRead: 124.77M (124770123) - RowsReturned: 24.90M (24904227) - RowsReturnedRate: 80.86 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.50K (10502) - ScannerThreadsTotalWallClockTime: 9m27s - MaterializeTupleTime(*): 10s940ms - ScannerThreadsSysTime: 1s295ms - ScannerThreadsUserTime: 20s053ms - ScannerThreadsVoluntaryContextSwitches: 24.37K (24369) - TotalRawHdfsReadTime(*): 984.332ms - TotalReadThroughput: 28.42 MB/sec Instance f5478537469ada54:a960c8d80000000e (host=vb0206.halxg.cloudera.com:22000):(Total: 52s249ms, non-child: 42s978ms, % non-child: 82.26%) Hdfs split stats (:<# splits>/): 2:2/506.78 MB 9:3/760.18 MB 6:2/506.81 MB 7:1/253.39 MB 5:2/506.80 MB 4:3/760.17 MB 1:2/506.77 MB 0:3/760.20 MB MemoryUsage(1s000ms): 751.11 MB, 1.09 GB, 1.09 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.05 GB, 1.04 GB, 1023.02 MB, 967.80 MB, 924.51 MB, 904.65 MB, 886.66 MB, 874.31 MB, 865.11 MB, 830.40 MB, 784.15 MB, 742.85 MB, 699.63 MB, 643.22 MB, 626.78 MB, 592.27 MB, 609.08 MB, 630.65 MB, 680.05 MB, 724.04 MB, 743.50 MB, 809.74 MB, 760.36 MB, 729.28 MB, 708.46 MB, 691.83 MB, 644.83 MB, 643.68 MB, 650.55 MB, 641.18 MB, 622.48 MB, 581.92 MB, 478.86 MB, 441.84 MB, 417.45 MB, 394.95 MB, 379.20 MB, 333.77 MB, 307.02 MB, 278.78 MB, 238.10 MB, 175.02 MB, 127.58 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 12, 12, 7, 7, 7, 7, 7, 7, 7, 7, 7, 5, 4 - AverageThreadTokens: 11.48 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1191724061) - PerHostPeakMemUsage: 1.11 GB (1191724061) - RowsProduced: 24.21M (24210513) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 44s028ms - TotalStorageWaitTime: 576.191ms - TotalThreadsInvoluntaryContextSwitches: 14.79K (14786) - TotalThreadsTotalWallClockTime: 10m - TotalThreadsSysTime: 2s276ms - TotalThreadsUserTime: 27s858ms - TotalThreadsVoluntaryContextSwitches: 46.66K (46660) Fragment Instance Lifecycle Timings: - ExecTime: 52s187ms - ExecTreeExecTime: 372.864ms - OpenTime: 34.069ms - ExecTreeOpenTime: 51.169us - PrepareTime: 27.168ms - ExecTreePrepareTime: 367.671us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s898ms, non-child: 8s898ms, % non-child: 100.00%) - BytesSent: 494.84 MB (518880607) - NetworkThroughput(*): 244.51 MB/sec - OverallThroughput: 55.61 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.21M (24210513) - SerializeBatchTime: 6s866ms - TransmitDataRPCTime: 2s023ms - UncompressedRowBatchSize: 854.38 MB (895883569) CodeGen:(Total: 58.640ms, non-child: 58.640ms, % non-child: 100.00%) - CodegenTime: 1.499ms - CompileTime: 10.276ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 23.046ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 24.752ms HDFS_SCAN_NODE (id=1):(Total: 313.123ms, non-child: 313.123ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:2/506.78 MB 9:3/760.18 MB 6:2/506.81 MB 7:1/253.39 MB 5:2/506.80 MB 4:3/760.17 MB 1:2/506.77 MB 0:3/760.20 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 18 out of 18 Hdfs Read Thread Concurrency Bucket: 0:98.1% 1:0.9524% 2:0% 3:0% 4:0.9524% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:72 BytesRead(1s000ms): 637.27 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1002.35 MB, 1.02 GB, 1.06 GB, 1.19 GB, 1.26 GB, 1.31 GB, 1.39 GB, 1.39 GB, 1.39 GB, 1.39 GB, 1.39 GB, 1.39 GB, 1.43 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 5.419ms ; Min: 2.033ms ; Max: 7.941ms ; Number of samples: 18) - AverageHdfsReadThreadConcurrency: 0.05 - AverageScannerThreadConcurrency: 10.48 - BytesRead: 1.47 GB (1576537765) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1576537765) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1576537765) - DecompressionTime: 9s083ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 8 (8) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1191111170) - PerReadThreadRawHdfsThroughput: 1.50 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 36.053ms - RowBatchQueuePutWaitTime: 8m46s - RowsRead: 124.58M (124580613) - RowsReturned: 24.21M (24210513) - RowsReturnedRate: 77.32 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 10.87K (10870) - ScannerThreadsTotalWallClockTime: 9m7s - MaterializeTupleTime(*): 11s006ms - ScannerThreadsSysTime: 1s392ms - ScannerThreadsUserTime: 19s929ms - ScannerThreadsVoluntaryContextSwitches: 23.81K (23812) - TotalRawHdfsReadTime(*): 981.649ms - TotalReadThroughput: 28.64 MB/sec Instance f5478537469ada54:a960c8d80000000b (host=vb0222.halxg.cloudera.com:22000):(Total: 52s217ms, non-child: 43s319ms, % non-child: 82.96%) Hdfs split stats (:<# splits>/): 6:1/253.38 MB 3:3/514.23 MB 10:1/253.41 MB 4:3/760.21 MB 9:1/253.37 MB 8:2/506.80 MB 7:2/506.80 MB 5:2/506.79 MB 1:1/253.40 MB 2:3/760.16 MB MemoryUsage(1s000ms): 1.09 GB, 1.09 GB, 1.09 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.11 GB, 1.10 GB, 1.07 GB, 1.05 GB, 1021.85 MB, 961.45 MB, 933.04 MB, 914.46 MB, 900.92 MB, 889.60 MB, 877.03 MB, 846.31 MB, 800.30 MB, 763.36 MB, 719.52 MB, 678.15 MB, 659.57 MB, 646.14 MB, 698.40 MB, 653.43 MB, 662.16 MB, 684.69 MB, 644.21 MB, 675.51 MB, 645.15 MB, 608.32 MB, 592.26 MB, 626.08 MB, 586.03 MB, 569.03 MB, 549.42 MB, 599.91 MB, 559.94 MB, 550.44 MB, 493.31 MB, 454.61 MB, 396.00 MB, 366.78 MB, 332.24 MB, 295.71 MB, 272.09 MB, 217.31 MB, 172.33 MB, 118.85 MB, 85.98 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 10, 8, 7, 6, 6, 6, 6, 6, 5, 4, 3, 3 - AverageThreadTokens: 11.37 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1188721721) - PerHostPeakMemUsage: 1.11 GB (1188721721) - RowsProduced: 24.52M (24523519) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 44s208ms - TotalStorageWaitTime: 530.357ms - TotalThreadsInvoluntaryContextSwitches: 14.24K (14241) - TotalThreadsTotalWallClockTime: 9m49s - TotalThreadsSysTime: 2s040ms - TotalThreadsUserTime: 27s485ms - TotalThreadsVoluntaryContextSwitches: 47.16K (47162) Fragment Instance Lifecycle Timings: - ExecTime: 52s156ms - ExecTreeExecTime: 364.277ms - OpenTime: 32.784ms - ExecTreeOpenTime: 39.831us - PrepareTime: 27.610ms - ExecTreePrepareTime: 366.141us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s542ms, non-child: 8s542ms, % non-child: 100.00%) - BytesSent: 501.25 MB (525595582) - NetworkThroughput(*): 281.14 MB/sec - OverallThroughput: 58.68 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.52M (24523519) - SerializeBatchTime: 6s750ms - TransmitDataRPCTime: 1s782ms - UncompressedRowBatchSize: 865.43 MB (907466011) CodeGen:(Total: 57.911ms, non-child: 57.911ms, % non-child: 100.00%) - CodegenTime: 1.311ms - CompileTime: 9.637ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.447ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 25.310ms HDFS_SCAN_NODE (id=1):(Total: 297.216ms, non-child: 297.216ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:1/253.38 MB 3:3/514.23 MB 10:1/253.41 MB 4:3/760.21 MB 9:1/253.37 MB 8:2/506.80 MB 7:2/506.80 MB 5:2/506.79 MB 1:1/253.40 MB 2:3/760.16 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1005.01 MB, 1.06 GB, 1.06 GB, 1.10 GB, 1.14 GB, 1.14 GB, 1.23 GB, 1.23 GB, 1.23 GB, 1.23 GB, 1.31 GB, 1.31 GB, 1.31 GB, 1.31 GB, 1.39 GB, 1.39 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 4.518ms ; Min: 2.108ms ; Max: 8.202ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.37 - BytesRead: 1.47 GB (1579318713) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1579318713) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1579318713) - DecompressionTime: 8s881ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 10 (10) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1188343334) - PerReadThreadRawHdfsThroughput: 1.40 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 22.283ms - RowBatchQueuePutWaitTime: 8m36s - RowsRead: 124.77M (124773003) - RowsReturned: 24.52M (24523519) - RowsReturnedRate: 82.51 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.47K (10469) - ScannerThreadsTotalWallClockTime: 8m57s - MaterializeTupleTime(*): 10s662ms - ScannerThreadsSysTime: 1s175ms - ScannerThreadsUserTime: 19s682ms - ScannerThreadsVoluntaryContextSwitches: 24.15K (24153) - TotalRawHdfsReadTime(*): 1s054ms - TotalReadThroughput: 28.96 MB/sec Instance f5478537469ada54:a960c8d80000000a (host=vb0228.halxg.cloudera.com:22000):(Total: 52s102ms, non-child: 43s128ms, % non-child: 82.78%) Hdfs split stats (:<# splits>/): 6:1/253.41 MB 4:2/506.76 MB 2:1/253.42 MB 10:2/445.01 MB 8:3/760.18 MB 3:1/187.22 MB 7:2/506.79 MB 5:4/1013.55 MB 0:3/760.18 MB MemoryUsage(1s000ms): 1.03 GB, 1.05 GB, 1.05 GB, 1.05 GB, 1.06 GB, 1.06 GB, 1.07 GB, 1.07 GB, 1.05 GB, 1.02 GB, 1014.75 MB, 960.53 MB, 906.91 MB, 882.82 MB, 860.75 MB, 842.17 MB, 827.44 MB, 812.45 MB, 764.56 MB, 725.68 MB, 763.96 MB, 719.57 MB, 689.46 MB, 653.54 MB, 646.29 MB, 625.12 MB, 608.68 MB, 667.40 MB, 687.24 MB, 731.23 MB, 749.88 MB, 726.71 MB, 761.62 MB, 812.89 MB, 797.28 MB, 745.37 MB, 727.26 MB, 705.23 MB, 680.99 MB, 632.28 MB, 595.41 MB, 523.36 MB, 490.42 MB, 469.96 MB, 434.39 MB, 397.77 MB, 366.79 MB, 329.98 MB, 289.96 MB, 223.89 MB, 175.13 MB, 106.84 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 11, 10, 8, 8, 8, 8, 8, 8, 8, 7, 6, 5, 3 - AverageThreadTokens: 11.68 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.07 GB (1147810903) - PerHostPeakMemUsage: 1.07 GB (1147810903) - RowsProduced: 24.20M (24199028) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 44s074ms - TotalStorageWaitTime: 563.013ms - TotalThreadsInvoluntaryContextSwitches: 15.01K (15012) - TotalThreadsTotalWallClockTime: 10m6s - TotalThreadsSysTime: 2s176ms - TotalThreadsUserTime: 28s176ms - TotalThreadsVoluntaryContextSwitches: 46.63K (46627) Fragment Instance Lifecycle Timings: - ExecTime: 52s045ms - ExecTreeExecTime: 354.930ms - OpenTime: 33.969ms - ExecTreeOpenTime: 45.808us - PrepareTime: 22.950ms - ExecTreePrepareTime: 325.402us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s629ms, non-child: 8s629ms, % non-child: 100.00%) - BytesSent: 494.62 MB (518648130) - NetworkThroughput(*): 264.53 MB/sec - OverallThroughput: 57.31 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.20M (24199028) - SerializeBatchTime: 6s751ms - TransmitDataRPCTime: 1s869ms - UncompressedRowBatchSize: 853.98 MB (895458584) CodeGen:(Total: 54.334ms, non-child: 54.334ms, % non-child: 100.00%) - CodegenTime: 1.537ms - CompileTime: 10.549ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.673ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 20.556ms HDFS_SCAN_NODE (id=1):(Total: 289.590ms, non-child: 289.590ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:1/253.41 MB 4:2/506.76 MB 2:1/253.42 MB 10:2/445.01 MB 8:3/760.18 MB 3:1/187.22 MB 7:2/506.79 MB 5:4/1013.55 MB 0:3/760.18 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:99.04% 1:0% 2:0.9615% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 934.02 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 960.28 MB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.02 GB, 1.06 GB, 1.18 GB, 1.22 GB, 1.30 GB, 1.35 GB, 1.35 GB, 1.43 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB, 1.51 GB - FooterProcessingTime: (Avg: 5.115ms ; Min: 2.687ms ; Max: 7.621ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 10.68 - BytesRead: 1.51 GB (1619983151) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.51 GB (1619983151) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.51 GB (1619983151) - DecompressionTime: 9s221ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.07 GB (1147730524) - PerReadThreadRawHdfsThroughput: 1.43 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 31.376ms - RowBatchQueuePutWaitTime: 8m52s - RowsRead: 128.00M (128001189) - RowsReturned: 24.20M (24199028) - RowsReturnedRate: 83.56 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 11.10K (11097) - ScannerThreadsTotalWallClockTime: 9m14s - MaterializeTupleTime(*): 11s164ms - ScannerThreadsSysTime: 1s271ms - ScannerThreadsUserTime: 20s366ms - ScannerThreadsVoluntaryContextSwitches: 23.81K (23812) - TotalRawHdfsReadTime(*): 1s053ms - TotalReadThroughput: 29.71 MB/sec Instance f5478537469ada54:a960c8d80000000d (host=vb0208.halxg.cloudera.com:22000):(Total: 52s030ms, non-child: 43s397ms, % non-child: 83.41%) Hdfs split stats (:<# splits>/): 9:1/253.39 MB 7:1/234.34 MB 5:2/506.78 MB 4:3/760.21 MB 8:3/760.15 MB 1:2/506.82 MB 3:1/253.41 MB 2:4/1013.57 MB 0:2/506.80 MB MemoryUsage(1s000ms): 1.09 GB, 1.09 GB, 1.09 GB, 1.09 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.08 GB, 1.05 GB, 1.01 GB, 989.48 MB, 961.37 MB, 913.51 MB, 904.91 MB, 883.55 MB, 862.18 MB, 841.62 MB, 790.52 MB, 766.66 MB, 735.54 MB, 694.46 MB, 643.78 MB, 611.83 MB, 595.47 MB, 562.76 MB, 541.36 MB, 681.52 MB, 666.73 MB, 641.83 MB, 661.36 MB, 657.49 MB, 637.21 MB, 666.73 MB, 759.14 MB, 756.37 MB, 725.79 MB, 715.32 MB, 686.37 MB, 632.90 MB, 594.22 MB, 537.60 MB, 487.26 MB, 460.22 MB, 440.76 MB, 390.10 MB, 341.38 MB, 302.45 MB, 248.87 MB, 222.94 MB, 131.86 MB, 89.68 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 12, 12, 12, 11, 10, 8, 8, 8, 8, 7, 7, 7, 6, 6, 4, 3 - AverageThreadTokens: 11.47 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1185374505) - PerHostPeakMemUsage: 1.10 GB (1185374505) - RowsProduced: 24.00M (24004509) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 44s157ms - TotalStorageWaitTime: 608.060ms - TotalThreadsInvoluntaryContextSwitches: 14.15K (14146) - TotalThreadsTotalWallClockTime: 9m56s - TotalThreadsSysTime: 2s318ms - TotalThreadsUserTime: 28s162ms - TotalThreadsVoluntaryContextSwitches: 46.30K (46300) Fragment Instance Lifecycle Timings: - ExecTime: 51s972ms - ExecTreeExecTime: 352.641ms - OpenTime: 33.806ms - ExecTreeOpenTime: 47.971us - PrepareTime: 23.932ms - ExecTreePrepareTime: 320.992us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s290ms, non-child: 8s290ms, % non-child: 100.00%) - BytesSent: 490.64 MB (514476323) - NetworkThroughput(*): 282.78 MB/sec - OverallThroughput: 59.18 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.00M (24004509) - SerializeBatchTime: 6s546ms - TransmitDataRPCTime: 1s735ms - UncompressedRowBatchSize: 847.11 MB (888260613) CodeGen:(Total: 55.196ms, non-child: 55.196ms, % non-child: 100.00%) - CodegenTime: 1.464ms - CompileTime: 10.205ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.839ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 21.591ms HDFS_SCAN_NODE (id=1):(Total: 286.888ms, non-child: 286.888ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 9:1/253.39 MB 7:1/234.34 MB 5:2/506.78 MB 4:3/760.21 MB 8:3/760.15 MB 1:2/506.82 MB 3:1/253.41 MB 2:4/1013.57 MB 0:2/506.80 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 996.04 MB, 1.01 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.26 GB, 1.30 GB, 1.30 GB, 1.38 GB, 1.50 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB - FooterProcessingTime: (Avg: 5.787ms ; Min: 2.853ms ; Max: 9.080ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.47 - BytesRead: 1.54 GB (1657536250) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.54 GB (1657536250) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.54 GB (1657536250) - DecompressionTime: 9s350ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1184892612) - PerReadThreadRawHdfsThroughput: 1.39 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 37.677ms - RowBatchQueuePutWaitTime: 8m42s - RowsRead: 130.98M (130980854) - RowsReturned: 24.00M (24004509) - RowsReturnedRate: 83.67 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.30K (10296) - ScannerThreadsTotalWallClockTime: 9m4s - MaterializeTupleTime(*): 11s217ms - ScannerThreadsSysTime: 1s371ms - ScannerThreadsUserTime: 20s517ms - ScannerThreadsVoluntaryContextSwitches: 23.69K (23694) - TotalRawHdfsReadTime(*): 1s111ms - TotalReadThroughput: 30.40 MB/sec Instance f5478537469ada54:a960c8d80000000c (host=vb0220.halxg.cloudera.com:22000):(Total: 51s820ms, non-child: 42s959ms, % non-child: 82.90%) Hdfs split stats (:<# splits>/): 8:2/506.79 MB 5:2/506.80 MB 10:1/253.39 MB 9:2/506.77 MB 7:1/253.41 MB 2:3/732.64 MB 3:4/1013.58 MB 0:3/760.18 MB 1:1/253.41 MB MemoryUsage(1s000ms): 552.93 MB, 1.08 GB, 1.08 GB, 1.08 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.04 GB, 1.02 GB, 1017.34 MB, 977.31 MB, 929.83 MB, 888.87 MB, 869.21 MB, 857.34 MB, 830.77 MB, 803.95 MB, 756.13 MB, 716.44 MB, 666.28 MB, 627.02 MB, 654.99 MB, 670.82 MB, 729.50 MB, 741.54 MB, 756.75 MB, 770.78 MB, 736.21 MB, 749.80 MB, 767.23 MB, 719.89 MB, 674.92 MB, 683.28 MB, 703.28 MB, 690.39 MB, 688.63 MB, 712.02 MB, 688.84 MB, 660.30 MB, 592.64 MB, 535.85 MB, 474.34 MB, 454.71 MB, 407.63 MB, 372.39 MB, 347.51 MB, 289.39 MB, 222.37 MB, 188.83 MB, 111.99 MB ThreadUsage(1s000ms): 7, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 11, 9, 8, 8, 8, 8, 8, 7, 6, 6, 4 - AverageThreadTokens: 11.75 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.10 GB (1184075157) - PerHostPeakMemUsage: 1.10 GB (1184075157) - RowsProduced: 24.28M (24277670) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 43s715ms - TotalStorageWaitTime: 660.842ms - TotalThreadsInvoluntaryContextSwitches: 14.88K (14882) - TotalThreadsTotalWallClockTime: 10m13s - TotalThreadsSysTime: 2s376ms - TotalThreadsUserTime: 28s434ms - TotalThreadsVoluntaryContextSwitches: 46.62K (46624) Fragment Instance Lifecycle Timings: - ExecTime: 51s758ms - ExecTreeExecTime: 367.598ms - OpenTime: 34.105ms - ExecTreeOpenTime: 52.763us - PrepareTime: 28.011ms - ExecTreePrepareTime: 368.859us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s494ms, non-child: 8s494ms, % non-child: 100.00%) - BytesSent: 496.22 MB (520319489) - NetworkThroughput(*): 285.84 MB/sec - OverallThroughput: 58.41 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 24.28M (24277670) - SerializeBatchTime: 6s749ms - TransmitDataRPCTime: 1s736ms - UncompressedRowBatchSize: 856.75 MB (898368646) CodeGen:(Total: 59.638ms, non-child: 59.638ms, % non-child: 100.00%) - CodegenTime: 1.402ms - CompileTime: 10.578ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.766ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 25.716ms HDFS_SCAN_NODE (id=1):(Total: 306.511ms, non-child: 306.511ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 8:2/506.79 MB 5:2/506.80 MB 10:1/253.39 MB 9:2/506.77 MB 7:1/253.41 MB 2:3/732.64 MB 3:4/1013.58 MB 0:3/760.18 MB 1:1/253.41 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:98.06% 1:1.942% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 496.66 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 993.31 MB, 1.05 GB, 1.09 GB, 1.17 GB, 1.21 GB, 1.26 GB, 1.30 GB, 1.30 GB, 1.34 GB, 1.38 GB, 1.38 GB, 1.38 GB, 1.42 GB, 1.46 GB, 1.46 GB, 1.50 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB, 1.54 GB - FooterProcessingTime: (Avg: 4.839ms ; Min: 2.056ms ; Max: 7.884ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 10.85 - BytesRead: 1.54 GB (1654662021) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.54 GB (1654662021) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.54 GB (1654662021) - DecompressionTime: 9s406ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.10 GB (1183994778) - PerReadThreadRawHdfsThroughput: 1.33 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 35.976ms - RowBatchQueuePutWaitTime: 8m59s - RowsRead: 130.75M (130748945) - RowsReturned: 24.28M (24277670) - RowsReturnedRate: 79.21 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.80K (10804) - ScannerThreadsTotalWallClockTime: 9m21s - MaterializeTupleTime(*): 11s244ms - ScannerThreadsSysTime: 1s307ms - ScannerThreadsUserTime: 20s768ms - ScannerThreadsVoluntaryContextSwitches: 23.89K (23894) - TotalRawHdfsReadTime(*): 1s154ms - TotalReadThroughput: 30.35 MB/sec Instance f5478537469ada54:a960c8d800000003 (host=vb0212.halxg.cloudera.com:22000):(Total: 51s374ms, non-child: 42s766ms, % non-child: 83.24%) Hdfs split stats (:<# splits>/): 10:1/253.38 MB 1:3/760.18 MB 5:1/253.39 MB 4:2/506.79 MB 7:3/760.20 MB 0:1/253.40 MB 6:2/506.82 MB 3:2/506.82 MB 2:3/760.21 MB MemoryUsage(1s000ms): 1.09 GB, 1.09 GB, 1.09 GB, 1.09 GB, 1.10 GB, 1.11 GB, 1.11 GB, 1.11 GB, 1.08 GB, 1.04 GB, 1.01 GB, 983.39 MB, 946.38 MB, 910.43 MB, 882.12 MB, 876.19 MB, 866.06 MB, 837.57 MB, 796.77 MB, 754.22 MB, 701.78 MB, 674.12 MB, 660.50 MB, 616.94 MB, 626.51 MB, 661.43 MB, 713.60 MB, 707.36 MB, 683.73 MB, 647.92 MB, 636.25 MB, 599.40 MB, 566.73 MB, 536.39 MB, 646.57 MB, 699.52 MB, 660.23 MB, 614.14 MB, 583.88 MB, 544.86 MB, 489.50 MB, 450.53 MB, 423.14 MB, 390.93 MB, 366.10 MB, 329.88 MB, 280.88 MB, 249.28 MB, 209.76 MB, 189.17 MB, 103.66 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 11, 9, 8, 8, 7, 7, 7, 7, 6, 6, 6, 6, 6, 3 - AverageThreadTokens: 11.26 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1193095253) - PerHostPeakMemUsage: 1.11 GB (1193095253) - RowsProduced: 23.47M (23465800) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 43s553ms - TotalStorageWaitTime: 627.276ms - TotalThreadsInvoluntaryContextSwitches: 14.17K (14171) - TotalThreadsTotalWallClockTime: 9m39s - TotalThreadsSysTime: 1s934ms - TotalThreadsUserTime: 27s741ms - TotalThreadsVoluntaryContextSwitches: 45.26K (45257) Fragment Instance Lifecycle Timings: - ExecTime: 51s317ms - ExecTreeExecTime: 373.424ms - OpenTime: 32.776ms - ExecTreeOpenTime: 43.115us - PrepareTime: 23.443ms - ExecTreePrepareTime: 301.284us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 8s242ms, non-child: 8s242ms, % non-child: 100.00%) - BytesSent: 479.63 MB (502926019) - NetworkThroughput(*): 284.95 MB/sec - OverallThroughput: 58.19 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 23.47M (23465800) - SerializeBatchTime: 6s550ms - TransmitDataRPCTime: 1s683ms - UncompressedRowBatchSize: 828.10 MB (868326276) CodeGen:(Total: 53.846ms, non-child: 53.846ms, % non-child: 100.00%) - CodegenTime: 1.288ms - CompileTime: 9.619ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.333ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 21.362ms HDFS_SCAN_NODE (id=1):(Total: 311.704ms, non-child: 311.704ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 10:1/253.38 MB 1:3/760.18 MB 5:1/253.39 MB 4:2/506.79 MB 7:3/760.20 MB 0:1/253.40 MB 6:2/506.82 MB 3:2/506.82 MB 2:3/760.21 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 18 out of 18 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:72 BytesRead(1s000ms): 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1002.37 MB, 1.02 GB, 1.10 GB, 1.18 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.22 GB, 1.39 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 5.106ms ; Min: 2.807ms ; Max: 8.456ms ; Number of samples: 18) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.26 - BytesRead: 1.47 GB (1576589528) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1576589528) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1576589528) - DecompressionTime: 9s052ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1192883786) - PerReadThreadRawHdfsThroughput: 1.32 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 35.248ms - RowBatchQueuePutWaitTime: 8m26s - RowsRead: 124.58M (124581157) - RowsReturned: 23.47M (23465800) - RowsReturnedRate: 75.28 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 10.45K (10450) - ScannerThreadsTotalWallClockTime: 8m48s - MaterializeTupleTime(*): 10s864ms - ScannerThreadsSysTime: 1s130ms - ScannerThreadsUserTime: 20s016ms - ScannerThreadsVoluntaryContextSwitches: 23.04K (23039) - TotalRawHdfsReadTime(*): 1s111ms - TotalReadThroughput: 29.20 MB/sec Instance f5478537469ada54:a960c8d800000007 (host=vb0234.halxg.cloudera.com:22000):(Total: 49s820ms, non-child: 41s524ms, % non-child: 83.35%) Hdfs split stats (:<# splits>/): 4:1/253.41 MB 5:2/506.79 MB 9:2/506.77 MB 1:1/253.39 MB 7:3/760.20 MB 6:3/760.17 MB 3:3/760.18 MB 2:2/258.65 MB 0:2/506.79 MB MemoryUsage(1s000ms): 1.09 GB, 1.09 GB, 1.09 GB, 1.09 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.09 GB, 1.06 GB, 1.03 GB, 997.72 MB, 957.92 MB, 917.07 MB, 902.48 MB, 894.70 MB, 879.68 MB, 855.57 MB, 812.51 MB, 774.48 MB, 732.44 MB, 693.25 MB, 669.68 MB, 630.59 MB, 644.27 MB, 645.12 MB, 685.24 MB, 666.77 MB, 636.18 MB, 627.32 MB, 660.91 MB, 686.61 MB, 651.01 MB, 664.44 MB, 641.98 MB, 697.32 MB, 677.24 MB, 634.49 MB, 592.93 MB, 561.05 MB, 486.55 MB, 435.11 MB, 399.77 MB, 347.62 MB, 290.99 MB, 255.25 MB, 216.22 MB, 153.70 MB, 119.82 MB ThreadUsage(1s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 11, 11, 10, 9, 8, 7, 6, 6, 5, 5, 4, 4 - AverageThreadTokens: 11.52 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.11 GB (1188723941) - PerHostPeakMemUsage: 1.11 GB (1188723941) - RowsProduced: 23.36M (23361849) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 42s158ms - TotalStorageWaitTime: 569.343ms - TotalThreadsInvoluntaryContextSwitches: 14.41K (14412) - TotalThreadsTotalWallClockTime: 9m32s - TotalThreadsSysTime: 2s051ms - TotalThreadsUserTime: 27s251ms - TotalThreadsVoluntaryContextSwitches: 44.76K (44759) Fragment Instance Lifecycle Timings: - ExecTime: 49s763ms - ExecTreeExecTime: 330.903ms - OpenTime: 33.193ms - ExecTreeOpenTime: 42.913us - PrepareTime: 23.289ms - ExecTreePrepareTime: 368.464us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 7s971ms, non-child: 7s971ms, % non-child: 100.00%) - BytesSent: 477.51 MB (500701167) - NetworkThroughput(*): 310.68 MB/sec - OverallThroughput: 59.90 MB/sec - PeakMemoryUsage: 3.88 KB (3968) - RowsReturned: 23.36M (23361849) - SerializeBatchTime: 6s426ms - TransmitDataRPCTime: 1s536ms - UncompressedRowBatchSize: 824.43 MB (864479689) CodeGen:(Total: 53.992ms, non-child: 53.992ms, % non-child: 100.00%) - CodegenTime: 1.285ms - CompileTime: 9.903ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (2003016) - NumFunctions: 22 (22) - NumInstructions: 390 (390) - OptimizationTime: 22.463ms - PeakMemoryUsage: 195.00 KB (199680) - PrepareTime: 21.105ms HDFS_SCAN_NODE (id=1):(Total: 269.976ms, non-child: 269.976ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:1/253.41 MB 5:2/506.79 MB 9:2/506.77 MB 1:1/253.39 MB 7:3/760.20 MB 6:3/760.17 MB 3:3/760.18 MB 2:2/258.65 MB 0:2/506.79 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 19 out of 19 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:76 BytesRead(1s000ms): 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1002.33 MB, 1.02 GB, 1.06 GB, 1.14 GB, 1.14 GB, 1.14 GB, 1.18 GB, 1.27 GB, 1.31 GB, 1.31 GB, 1.35 GB, 1.39 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB, 1.47 GB - FooterProcessingTime: (Avg: 4.895ms ; Min: 2.924ms ; Max: 6.570ms ; Number of samples: 19) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.52 - BytesRead: 1.47 GB (1578597766) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.47 GB (1578597766) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.47 GB (1578597766) - DecompressionTime: 8s997ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 12 (12) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 1.11 GB (1188512564) - PerReadThreadRawHdfsThroughput: 1.49 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 30.617ms - RowBatchQueuePutWaitTime: 8m21s - RowsRead: 124.71M (124712354) - RowsReturned: 23.36M (23361849) - RowsReturnedRate: 86.53 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 10.79K (10793) - ScannerThreadsTotalWallClockTime: 8m42s - MaterializeTupleTime(*): 10s743ms - ScannerThreadsSysTime: 1s235ms - ScannerThreadsUserTime: 19s764ms - ScannerThreadsVoluntaryContextSwitches: 22.98K (22982) - TotalRawHdfsReadTime(*): 986.443ms - TotalReadThroughput: 30.41 MB/sec ================================================================================