13000000 +--------------+--------+----------+----------+--------+------------+-----------+---------------+-----------------------+ | Operator | #Hosts | Avg Time | Max Time | #Rows | Est. #Rows | Peak Mem | Est. Peak Mem | Detail | +--------------+--------+----------+----------+--------+------------+-----------+---------------+-----------------------+ | 09:AGGREGATE | 1 | 134.96ms | 134.96ms | 1 | 1 | 24.00 KB | -1 B | FINALIZE | | 08:EXCHANGE | 1 | 42.05us | 42.05us | 3 | 1 | 0 B | -1 B | UNPARTITIONED | | 03:AGGREGATE | 3 | 159.19ms | 162.38ms | 3 | 1 | 12.00 KB | 10.00 MB | | | 07:AGGREGATE | 3 | 1.51s | 1.52s | 13.00M | -1 | 307.03 MB | 128.00 MB | | | 06:EXCHANGE | 3 | 556.05ms | 556.20ms | 13.00M | -1 | 0 B | 0 B | HASH((i)) | | 02:AGGREGATE | 3 | 1.44s | 1.46s | 13.00M | -1 | 307.01 MB | 128.00 MB | | | 05:AGGREGATE | 3 | 1.46s | 1.50s | 13.00M | -1 | 307.03 MB | 128.00 MB | FINALIZE | | 04:EXCHANGE | 3 | 515.49ms | 533.99ms | 13.00M | -1 | 0 B | 0 B | HASH(i) | | 01:AGGREGATE | 3 | 2.09s | 2.35s | 13.00M | -1 | 307.23 MB | 128.00 MB | | | 00:SCAN HDFS | 3 | 449.49ms | 496.77ms | 13.00M | -1 | 18.27 MB | 32.00 MB | jbappletest.counter13 | +--------------+--------+----------+----------+--------+------------+-----------+---------------+-----------------------+ Query Runtime Profile: Query (id=344463ad4603c165:7bf25b60e6357ea0): Summary: Session ID: 3e452ad241ada0fb:1d886d612d6038b6 Session Type: BEESWAX Start Time: 2015-10-06 14:46:03.831953000 End Time: 2015-10-06 14:46:19.741912000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.3.0-cdh5-INTERNAL DEBUG (build b68c8f93dd21fbcc0687effbff15d6663f20350a) User: jbapple Connected User: jbapple Delegated User: Network Address: ::ffff:127.0.0.1:55461 Default Db: default Sql Statement: select count(distinct(i)) as ndv from (select i from jbappletest.counter13 where i < 13000000 group by i) a Coordinator: jbapple-optiplex:22000 Plan: ---------------- Estimated Per-Host Requirements: Memory=256.00MB VCores=2 WARNING: The following tables are missing relevant table and/or column statistics. jbappletest.counter13 F03:PLAN FRAGMENT [UNPARTITIONED] 09:AGGREGATE [FINALIZE] | output: count:merge((i)) | hosts=3 per-host-mem=unavailable | tuple-ids=4 row-size=8B cardinality=1 | 08:EXCHANGE [UNPARTITIONED] hosts=3 per-host-mem=unavailable tuple-ids=4 row-size=8B cardinality=1 F02:PLAN FRAGMENT [HASH((i))] DATASTREAM SINK [FRAGMENT=F03, EXCHANGE=08, UNPARTITIONED] 03:AGGREGATE | output: count((i)) | hosts=3 per-host-mem=10.00MB | tuple-ids=4 row-size=8B cardinality=1 | 07:AGGREGATE | group by: (i) | hosts=3 per-host-mem=128.00MB | tuple-ids=3 row-size=4B cardinality=unavailable | 06:EXCHANGE [HASH((i))] hosts=3 per-host-mem=0B tuple-ids=3 row-size=4B cardinality=unavailable F01:PLAN FRAGMENT [HASH(i)] DATASTREAM SINK [FRAGMENT=F02, EXCHANGE=06, HASH((i))] 02:AGGREGATE | group by: i | hosts=3 per-host-mem=128.00MB | tuple-ids=3 row-size=4B cardinality=unavailable | 05:AGGREGATE [FINALIZE] | group by: i | hosts=3 per-host-mem=128.00MB | tuple-ids=1 row-size=4B cardinality=unavailable | 04:EXCHANGE [HASH(i)] hosts=3 per-host-mem=0B tuple-ids=1 row-size=4B cardinality=unavailable F00:PLAN FRAGMENT [RANDOM] DATASTREAM SINK [FRAGMENT=F01, EXCHANGE=04, HASH(i)] 01:AGGREGATE | group by: i | hosts=3 per-host-mem=128.00MB | tuple-ids=1 row-size=4B cardinality=unavailable | 00:SCAN HDFS [jbappletest.counter13, RANDOM] partitions=1/1 files=3 size=49.85MB predicates: i < 13000000 table stats: unavailable column stats: unavailable hosts=3 per-host-mem=32.00MB tuple-ids=0 row-size=4B cardinality=unavailable ---------------- Estimated Per-Host Mem: 268435456 Estimated Per-Host VCores: 2 Tables Missing Stats: jbappletest.counter13 Request Pool: default-pool ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ----------------------------------------------------------------------------------------------------------------- 09:AGGREGATE 1 134.957ms 134.957ms 1 1 24.00 KB -1.00 B FINALIZE 08:EXCHANGE 1 42.53us 42.53us 3 1 0 -1.00 B UNPARTITIONED 03:AGGREGATE 3 159.192ms 162.378ms 3 1 12.00 KB 10.00 MB 07:AGGREGATE 3 1s508ms 1s521ms 13.00M -1 307.03 MB 128.00 MB 06:EXCHANGE 3 556.52ms 556.202ms 13.00M -1 0 0 HASH((i)) 02:AGGREGATE 3 1s439ms 1s458ms 13.00M -1 307.01 MB 128.00 MB 05:AGGREGATE 3 1s463ms 1s502ms 13.00M -1 307.03 MB 128.00 MB FINALIZE 04:EXCHANGE 3 515.491ms 533.985ms 13.00M -1 0 0 HASH(i) 01:AGGREGATE 3 2s093ms 2s348ms 13.00M -1 307.23 MB 128.00 MB 00:SCAN HDFS 3 449.486ms 496.767ms 13.00M -1 18.27 MB 32.00 MB jbappletest.counter13 Planner Timeline: 125.743ms - Analysis finished: 22.57ms (22.57ms) - Equivalence classes computed: 42.793ms (20.736ms) - Single node plan created: 53.44ms (10.251ms) - Distributed plan created: 57.380ms (4.336ms) - Planning finished: 125.743ms (68.362ms) Query Timeline: 15s914ms - Start execution: 46.575us (46.575us) - Planning finished: 4s527ms (4s527ms) - Ready to start remote fragments: 4s662ms (135.515ms) - Remote fragments started: 5s371ms (708.487ms) - Rows available: 15s589ms (10s218ms) - First row fetched: 15s908ms (318.903ms) - Unregister query: 15s909ms (1.551ms) ImpalaServer: - ClientFetchWaitTimer: 319.304ms - RowMaterializationTimer: 6.75us Execution Profile 344463ad4603c165:7bf25b60e6357ea0:(Total: 11s063ms, non-child: 0ns, % non-child: 0.00%) Fragment start latencies: count: 9, last: 0.317409ns, min: 0.141255ns, max: 0.317409ns, mean: 0.205549ns, stddev: 0.057394ns Per Node Peak Memory Usage: jbapple-optiplex:22001(343.42 MB) jbapple-optiplex:22002(343.29 MB) jbapple-optiplex:22000(361.41 MB) - FinalizationTimer: 0ns Coordinator Fragment F03:(Total: 10s100ms, non-child: 0ns, % non-child: 0.00%) MemoryUsage(500.0ms): 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB ThreadUsage(500.0ms): 1 - AverageThreadTokens: 1.00 - PeakMemoryUsage: 32.02 KB (32784) - PerHostPeakMemUsage: 0 - PrepareTime: 134.883ms - RowsProduced: 1 (1) - TotalCpuTime: 1s416ms - TotalNetworkReceiveTime: 9s964ms - TotalNetworkSendTime: 0ns - TotalStorageWaitTime: 0ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 264 (264) - BlocksRecycled: 184 (184) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 20.05 GB (21531490304) - PeakMemoryUsage: 331.35 MB (347447296) - TotalBufferWaitTime: 0ns - TotalEncryptionTime: 0ns - TotalIntegrityCheckTime: 0ns - TotalReadBlockTime: 0ns CodeGen:(Total: 387.395ms, non-child: 387.395ms, % non-child: 100.00%) - CodegenTime: 595.856us - CompileTime: 433.196us - LoadTime: 22.313us - ModuleFileSize: 2.62 MB (2744720) - OptimizationTime: 252.660ms - PrepareTime: 133.325ms AGGREGATION_NODE (id=9):(Total: 10s099ms, non-child: 134.957ms, % non-child: 1.34%) ExecOption: Codegen Enabled - BuildTime: 1.511us - GetResultsTime: 0ns - HTResizeTime: 0ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 24.00 KB (24576) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=8):(Total: 9s965ms, non-child: 42.53us, % non-child: 0.00%) BytesReceived(500.0ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 - BytesReceived: 48.00 B (48) - ConvertRowBatchTime: 5.54us - DeserializeRowBatchTimer: 20.212us - FirstBatchArrivalWaitTime: 9s960ms - PeakMemoryUsage: 0 - RowsReturned: 3 (3) - RowsReturnedRate: 0 - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns Averaged Fragment F02:(Total: 10s588ms, non-child: 0ns, % non-child: 0.00%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:10s776ms max:10s786ms mean: 10s781ms stddev:4.039807.39.807394ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 3 - AverageThreadTokens: 1.00 - PeakMemoryUsage: 307.18 MB (322102613) - PerHostPeakMemUsage: 349.37 MB (366344874) - PrepareTime: 142.841ms - RowsProduced: 1 (1) - TotalCpuTime: 2s546ms - TotalNetworkReceiveTime: 8s377ms - TotalNetworkSendTime: 175.884us - TotalStorageWaitTime: 0ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 258 (258) - BlocksRecycled: 189 (189) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 20.05 GB (21531490304) - PeakMemoryUsage: 314.36 MB (329625600) - TotalBufferWaitTime: 0ns - TotalEncryptionTime: 0ns - TotalIntegrityCheckTime: 0ns - TotalReadBlockTime: 0ns CodeGen:(Total: 474.318ms, non-child: 474.318ms, % non-child: 100.00%) - CodegenTime: 2.887ms - CompileTime: 8.562ms - LoadTime: 15.82us - ModuleFileSize: 2.62 MB (2744720) - OptimizationTime: 326.108ms - PrepareTime: 138.794ms DataStreamSender (dst_id=8):(Total: 35.642us, non-child: 35.642us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 141.47 KB/sec - OverallThroughput: 446.93 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - SerializeBatchTime: 9.662us - ThriftTransmitTime(*): 112.405us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=3):(Total: 10s588ms, non-child: 159.192ms, % non-child: 1.50%) - BuildTime: 17.170ms - GetResultsTime: 0ns - HTResizeTime: 0ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 12.00 KB (12288) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) AGGREGATION_NODE (id=7):(Total: 10s429ms, non-child: 1s508ms, % non-child: 14.46%) - BuildTime: 1s058ms - GetNewBlockTime: 480.305us - GetResultsTime: 438.589ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.02 MB (321938816) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4333333) - RowsReturnedRate: 415.49 K/sec - SpilledPartitions: 0 (0) - UnpinTime: 2.998us EXCHANGE_NODE (id=6):(Total: 8s921ms, non-child: 8s921ms, % non-child: 100.00%) - BytesReceived: 37.85 MB (39692895) - ConvertRowBatchTime: 535.153ms - DeserializeRowBatchTimer: 119.709ms - FirstBatchArrivalWaitTime: 6s718ms - PeakMemoryUsage: 0 - RowsReturned: 4.33M (4333333) - RowsReturnedRate: 485.74 K/sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns Averaged Fragment F01:(Total: 9s812ms, non-child: 0ns, % non-child: 0.00%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:9s881ms max:10s151ms mean: 9s989ms stddev:116.490416.490.416311ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 3 - AverageThreadTokens: 1.00 - PeakMemoryUsage: 329.18 MB (345169920) - PerHostPeakMemUsage: 349.37 MB (366344874) - PrepareTime: 205.342ms - RowsProduced: 4.33M (4333333) - TotalCpuTime: 6s255ms - TotalNetworkReceiveTime: 3s907ms - TotalNetworkSendTime: 32.178ms - TotalStorageWaitTime: 0ns CodeGen:(Total: 572.123ms, non-child: 572.123ms, % non-child: 100.00%) - CodegenTime: 8.293ms - CompileTime: 9.543ms - LoadTime: 19.174us - ModuleFileSize: 2.62 MB (2744720) - OptimizationTime: 366.585ms - PrepareTime: 195.118ms DataStreamSender (dst_id=6):(Total: 2s493ms, non-child: 2s493ms, % non-child: 100.00%) - BytesSent: 37.85 MB (39692895) - NetworkThroughput(*): 57.43 MB/sec - OverallThroughput: 15.19 MB/sec - PeakMemoryUsage: 76.00 KB (77824) - SerializeBatchTime: 1s060ms - ThriftTransmitTime(*): 659.499ms - UncompressedRowBatchSize: 49.60 MB (52008465) AGGREGATION_NODE (id=2):(Total: 7s314ms, non-child: 1s439ms, % non-child: 19.68%) - BuildTime: 590.233ms - GetNewBlockTime: 437.980us - GetResultsTime: 641.907ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.01 MB (321921024) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4333333) - RowsReturnedRate: 592.42 K/sec - SpilledPartitions: 0 (0) - UnpinTime: 4.506us AGGREGATION_NODE (id=5):(Total: 5s875ms, non-child: 1s463ms, % non-child: 24.92%) - BuildTime: 970.268ms - GetNewBlockTime: 678.524us - GetResultsTime: 479.746ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.02 MB (321938816) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4333333) - RowsReturnedRate: 737.57 K/sec - SpilledPartitions: 0 (0) - UnpinTime: 6.251us EXCHANGE_NODE (id=4):(Total: 4s411ms, non-child: 4s411ms, % non-child: 100.00%) - BytesReceived: 37.85 MB (39693017) - ConvertRowBatchTime: 495.719ms - DeserializeRowBatchTimer: 113.34ms - FirstBatchArrivalWaitTime: 1s740ms - PeakMemoryUsage: 0 - RowsReturned: 4.33M (4333333) - RowsReturnedRate: 982.50 K/sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns Averaged Fragment F00:(Total: 5s237ms, non-child: 0ns, % non-child: 0.00%) split sizes: min: 16.61 MB, max: 16.62 MB, avg: 16.62 MB, stddev: 2.25 KB completion times: min:4s860ms max:5s467ms mean: 5s247ms stddev:274.273812.273.811847ms execution rates: min:3.04 MB/sec max:3.42 MB/sec mean:3.18 MB/sec stddev:176.29 KB/sec num instances: 3 - AverageThreadTokens: 1.28 - PeakMemoryUsage: 313.60 MB (328831744) - PerHostPeakMemUsage: 349.37 MB (366344874) - PrepareTime: 263.842ms - RowsProduced: 4.33M (4333333) - TotalCpuTime: 6s919ms - TotalNetworkReceiveTime: 0ns - TotalNetworkSendTime: 24.230ms - TotalStorageWaitTime: 114.449ms CodeGen:(Total: 511.732ms, non-child: 511.732ms, % non-child: 100.00%) - CodegenTime: 4.278ms - CompileTime: 5.170ms - LoadTime: 25.606us - ModuleFileSize: 2.62 MB (2744720) - OptimizationTime: 262.314ms - PrepareTime: 243.338ms DataStreamSender (dst_id=4):(Total: 2s690ms, non-child: 2s690ms, % non-child: 100.00%) - BytesSent: 37.85 MB (39693017) - NetworkThroughput(*): 60.81 MB/sec - OverallThroughput: 14.13 MB/sec - PeakMemoryUsage: 76.00 KB (77824) - SerializeBatchTime: 1s139ms - ThriftTransmitTime(*): 622.818ms - UncompressedRowBatchSize: 49.60 MB (52008465) AGGREGATION_NODE (id=1):(Total: 2s542ms, non-child: 2s093ms, % non-child: 82.32%) - BuildTime: 1s114ms - GetNewBlockTime: 659.860us - GetResultsTime: 721.12ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.21 MB (322137629) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4333333) - RowsReturnedRate: 1.71 M/sec - SpilledPartitions: 0 (0) - UnpinTime: 4.247us HDFS_SCAN_NODE (id=0):(Total: 449.486ms, non-child: 449.486ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 16.71 MB (17526707) - BytesReadDataNodeCache: 0 - BytesReadLocal: 16.71 MB (17526707) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 16.71 MB (17526707) - DecompressionTime: 4.257ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 1 (1) - NumRowGroups: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 18.09 MB (18966136) - PerReadThreadRawHdfsThroughput: 2.33 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 4.33M (4333333) - RowsReturned: 4.33M (4333333) - RowsReturnedRate: 9.80 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 121 (121) - ScannerThreadsTotalWallClockTime: 1s547ms - MaterializeTupleTime(*): 1s422ms - ScannerThreadsSysTime: 1.333ms - ScannerThreadsUserTime: 1s382ms - ScannerThreadsVoluntaryContextSwitches: 235 (235) - TotalRawHdfsReadTime(*): 7.297ms - TotalReadThroughput: 8.36 MB/sec Fragment F02: Instance 344463ad4603c165:7bf25b60e6357ea2 (host=jbapple-optiplex:22002):(Total: 10s631ms, non-child: 0ns, % non-child: 0.00%) MemoryUsage(500.0ms): 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 12.68 MB, 60.17 MB, 107.64 MB, 155.14 MB, 202.61 MB, 249.09 MB, 287.07 MB, 134.35 MB ThreadUsage(500.0ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - PeakMemoryUsage: 307.18 MB (322097152) - PerHostPeakMemUsage: 343.29 MB (359964672) - PrepareTime: 140.83ms - RowsProduced: 1 (1) - TotalCpuTime: 2s510ms - TotalNetworkReceiveTime: 8s415ms - TotalNetworkSendTime: 181.617us - TotalStorageWaitTime: 0ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 257 (257) - BlocksRecycled: 191 (191) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 20.05 GB (21531490304) - PeakMemoryUsage: 314.35 MB (329621504) - TotalBufferWaitTime: 0ns - TotalEncryptionTime: 0ns - TotalIntegrityCheckTime: 0ns - TotalReadBlockTime: 0ns CodeGen:(Total: 430.532ms, non-child: 430.532ms, % non-child: 100.00%) - CodegenTime: 2.906ms - CompileTime: 8.696ms - LoadTime: 14.917us - ModuleFileSize: 2.62 MB (2744720) - OptimizationTime: 284.996ms - PrepareTime: 135.895ms DataStreamSender (dst_id=8):(Total: 31.506us, non-child: 31.506us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 152.18 KB/sec - OverallThroughput: 495.94 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - SerializeBatchTime: 7.941us - ThriftTransmitTime(*): 102.671us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=3):(Total: 10s631ms, non-child: 156.678ms, % non-child: 1.47%) ExecOption: Codegen Enabled - BuildTime: 17.376ms - GetResultsTime: 0ns - HTResizeTime: 0ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 12.00 KB (12288) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) AGGREGATION_NODE (id=7):(Total: 10s474ms, non-child: 1s514ms, % non-child: 14.45%) ExecOption: Codegen Enabled - BuildTime: 1s062ms - GetNewBlockTime: 489.451us - GetResultsTime: 440.219ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.03 MB (321941632) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4334096) - RowsReturnedRate: 413.76 K/sec - SpilledPartitions: 0 (0) - UnpinTime: 3.136us EXCHANGE_NODE (id=6):(Total: 8s960ms, non-child: 556.202ms, % non-child: 6.21%) BytesReceived(500.0ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 439.71 KB, 6.44 MB, 12.43 MB, 18.34 MB, 24.28 MB, 29.13 MB, 34.47 MB - BytesReceived: 37.86 MB (39699770) - ConvertRowBatchTime: 536.66ms - DeserializeRowBatchTimer: 119.260ms - FirstBatchArrivalWaitTime: 6s760ms - PeakMemoryUsage: 0 - RowsReturned: 4.33M (4334096) - RowsReturnedRate: 483.68 K/sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns Instance 344463ad4603c165:7bf25b60e6357ea4 (host=jbapple-optiplex:22000):(Total: 10s577ms, non-child: 0ns, % non-child: 0.00%) MemoryUsage(500.0ms): 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 4.46 MB, 60.26 MB, 116.14 MB, 155.13 MB, 211.12 MB, 268.11 MB, 307.07 MB, 153.54 MB ThreadUsage(500.0ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - PeakMemoryUsage: 307.18 MB (322097152) - PerHostPeakMemUsage: 361.41 MB (378966016) - PrepareTime: 146.101ms - RowsProduced: 1 (1) - TotalCpuTime: 2s540ms - TotalNetworkReceiveTime: 8s381ms - TotalNetworkSendTime: 192.704us - TotalStorageWaitTime: 0ns CodeGen:(Total: 487.423ms, non-child: 487.423ms, % non-child: 100.00%) - CodegenTime: 2.849ms - CompileTime: 8.171ms - LoadTime: 15.86us - ModuleFileSize: 2.62 MB (2744720) - OptimizationTime: 336.316ms - PrepareTime: 142.243ms DataStreamSender (dst_id=8):(Total: 42.928us, non-child: 42.928us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 116.32 KB/sec - OverallThroughput: 363.98 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - SerializeBatchTime: 13.484us - ThriftTransmitTime(*): 134.328us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=3):(Total: 10s576ms, non-child: 162.378ms, % non-child: 1.54%) ExecOption: Codegen Enabled - BuildTime: 17.89ms - GetResultsTime: 0ns - HTResizeTime: 0ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 12.00 KB (12288) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) AGGREGATION_NODE (id=7):(Total: 10s414ms, non-child: 1s489ms, % non-child: 14.30%) ExecOption: Codegen Enabled - BuildTime: 1s040ms - GetNewBlockTime: 479.851us - GetResultsTime: 436.549ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.02 MB (321937408) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4332707) - RowsReturnedRate: 416.03 K/sec - SpilledPartitions: 0 (0) - UnpinTime: 3.152us EXCHANGE_NODE (id=6):(Total: 8s925ms, non-child: 555.964ms, % non-child: 6.23%) BytesReceived(500.0ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 219.89 KB, 6.23 MB, 12.54 MB, 18.70 MB, 24.65 MB, 31.54 MB, 37.30 MB - BytesReceived: 37.85 MB (39687164) - ConvertRowBatchTime: 535.199ms - DeserializeRowBatchTimer: 115.602ms - FirstBatchArrivalWaitTime: 6s710ms - PeakMemoryUsage: 0 - RowsReturned: 4.33M (4332707) - RowsReturnedRate: 485.44 K/sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns Instance 344463ad4603c165:7bf25b60e6357ea3 (host=jbapple-optiplex:22001):(Total: 10s557ms, non-child: 0ns, % non-child: 0.00%) MemoryUsage(500.0ms): 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 3.15 MB, 12.43 MB, 60.17 MB, 116.16 MB, 155.14 MB, 211.11 MB, 250.12 MB, 306.07 MB, 153.54 MB ThreadUsage(500.0ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - PeakMemoryUsage: 307.19 MB (322113536) - PerHostPeakMemUsage: 343.42 MB (360103936) - PrepareTime: 142.339ms - RowsProduced: 1 (1) - TotalCpuTime: 2s589ms - TotalNetworkReceiveTime: 8s335ms - TotalNetworkSendTime: 153.332us - TotalStorageWaitTime: 0ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 260 (260) - BlocksRecycled: 188 (188) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 20.05 GB (21531490304) - PeakMemoryUsage: 314.36 MB (329629696) - TotalBufferWaitTime: 0ns - TotalEncryptionTime: 0ns - TotalIntegrityCheckTime: 0ns - TotalReadBlockTime: 0ns CodeGen:(Total: 504.999ms, non-child: 504.999ms, % non-child: 100.00%) - CodegenTime: 2.906ms - CompileTime: 8.820ms - LoadTime: 15.245us - ModuleFileSize: 2.62 MB (2744720) - OptimizationTime: 357.13ms - PrepareTime: 138.245ms DataStreamSender (dst_id=8):(Total: 32.493us, non-child: 32.493us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 155.91 KB/sec - OverallThroughput: 480.87 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - SerializeBatchTime: 7.563us - ThriftTransmitTime(*): 100.218us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=3):(Total: 10s557ms, non-child: 158.519ms, % non-child: 1.50%) ExecOption: Codegen Enabled - BuildTime: 17.45ms - GetResultsTime: 0ns - HTResizeTime: 0ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 12.00 KB (12288) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) AGGREGATION_NODE (id=7):(Total: 10s399ms, non-child: 1s521ms, % non-child: 14.63%) ExecOption: Codegen Enabled - BuildTime: 1s071ms - GetNewBlockTime: 471.615us - GetResultsTime: 438.999ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.02 MB (321937408) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4333197) - RowsReturnedRate: 416.69 K/sec - SpilledPartitions: 0 (0) - UnpinTime: 2.708us EXCHANGE_NODE (id=6):(Total: 8s877ms, non-child: 555.989ms, % non-child: 6.26%) BytesReceived(500.0ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 329.80 KB, 6.69 MB, 12.76 MB, 18.87 MB, 24.71 MB, 29.95 MB, 36.64 MB - BytesReceived: 37.85 MB (39691752) - ConvertRowBatchTime: 534.194ms - DeserializeRowBatchTimer: 124.264ms - FirstBatchArrivalWaitTime: 6s683ms - PeakMemoryUsage: 0 - RowsReturned: 4.33M (4333197) - RowsReturnedRate: 488.10 K/sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns Fragment F01: Instance 344463ad4603c165:7bf25b60e6357ea5 (host=jbapple-optiplex:22002):(Total: 9s932ms, non-child: 0ns, % non-child: 0.00%) MemoryUsage(500.0ms): 0, 6.30 MB, 6.30 MB, 6.30 MB, 6.30 MB, 24.34 MB, 63.34 MB, 102.44 MB, 139.29 MB, 186.78 MB, 233.27 MB, 271.23 MB, 327.98 MB, 308.43 MB, 307.07 MB, 268.71 MB, 211.12 MB, 172.75 MB, 115.21 MB, 76.83 MB, 38.46 MB ThreadUsage(500.0ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - PeakMemoryUsage: 329.18 MB (345169920) - PerHostPeakMemUsage: 343.29 MB (359964672) - PrepareTime: 164.254ms - RowsProduced: 4.33M (4334096) - TotalCpuTime: 6s295ms - TotalNetworkReceiveTime: 3s992ms - TotalNetworkSendTime: 28.742ms - TotalStorageWaitTime: 0ns CodeGen:(Total: 534.228ms, non-child: 534.228ms, % non-child: 100.00%) - CodegenTime: 8.604ms - CompileTime: 7.928ms - LoadTime: 18.293us - ModuleFileSize: 2.62 MB (2744720) - OptimizationTime: 371.524ms - PrepareTime: 153.881ms DataStreamSender (dst_id=6):(Total: 2s593ms, non-child: 2s593ms, % non-child: 100.00%) - BytesSent: 37.86 MB (39699770) - NetworkThroughput(*): 56.86 MB/sec - OverallThroughput: 14.60 MB/sec - PeakMemoryUsage: 76.00 KB (77824) - SerializeBatchTime: 1s108ms - ThriftTransmitTime(*): 665.912ms - UncompressedRowBatchSize: 49.61 MB (52017620) AGGREGATION_NODE (id=2):(Total: 7s333ms, non-child: 1s425ms, % non-child: 19.44%) ExecOption: Codegen Enabled - BuildTime: 594.219ms - GetNewBlockTime: 439.883us - GetResultsTime: 664.685ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.01 MB (321921024) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4334096) - RowsReturnedRate: 591.02 K/sec - SpilledPartitions: 0 (0) - UnpinTime: 5.459us AGGREGATION_NODE (id=5):(Total: 5s907ms, non-child: 1s423ms, % non-child: 24.10%) ExecOption: Codegen Enabled - BuildTime: 934.972ms - GetNewBlockTime: 541.858us - GetResultsTime: 474.900ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.03 MB (321941632) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4334096) - RowsReturnedRate: 733.61 K/sec - SpilledPartitions: 0 (0) - UnpinTime: 4.923us EXCHANGE_NODE (id=4):(Total: 4s484ms, non-child: 504.193ms, % non-child: 11.24%) BytesReceived(500.0ms): 0, 0, 0, 0, 1.02 MB, 6.78 MB, 12.00 MB, 16.51 MB, 21.67 MB, 27.14 MB, 31.77 MB - BytesReceived: 37.86 MB (39699745) - ConvertRowBatchTime: 483.338ms - DeserializeRowBatchTimer: 112.373ms - FirstBatchArrivalWaitTime: 1s941ms - PeakMemoryUsage: 0 - RowsReturned: 4.33M (4334096) - RowsReturnedRate: 966.48 K/sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns Instance 344463ad4603c165:7bf25b60e6357ea6 (host=jbapple-optiplex:22001):(Total: 9s821ms, non-child: 0ns, % non-child: 0.00%) MemoryUsage(500.0ms): 0, 6.30 MB, 6.30 MB, 6.30 MB, 6.30 MB, 43.33 MB, 81.31 MB, 129.79 MB, 177.29 MB, 233.25 MB, 290.23 MB, 310.21 MB, 327.98 MB, 327.62 MB, 307.08 MB, 268.71 MB, 211.13 MB, 172.75 MB, 115.21 MB, 76.82 MB, 19.25 MB ThreadUsage(500.0ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - PeakMemoryUsage: 329.18 MB (345169920) - PerHostPeakMemUsage: 343.42 MB (360103936) - PrepareTime: 228.693ms - RowsProduced: 4.33M (4333197) - TotalCpuTime: 6s206ms - TotalNetworkReceiveTime: 3s926ms - TotalNetworkSendTime: 31.123ms - TotalStorageWaitTime: 0ns CodeGen:(Total: 558.313ms, non-child: 558.313ms, % non-child: 100.00%) - CodegenTime: 6.821ms - CompileTime: 12.967ms - LoadTime: 17.596us - ModuleFileSize: 2.62 MB (2744720) - OptimizationTime: 324.878ms - PrepareTime: 219.692ms DataStreamSender (dst_id=6):(Total: 2s469ms, non-child: 2s469ms, % non-child: 100.00%) - BytesSent: 37.85 MB (39691752) - NetworkThroughput(*): 59.25 MB/sec - OverallThroughput: 15.33 MB/sec - PeakMemoryUsage: 76.00 KB (77824) - SerializeBatchTime: 1s055ms - ThriftTransmitTime(*): 638.843ms - UncompressedRowBatchSize: 49.60 MB (52006828) AGGREGATION_NODE (id=2):(Total: 7s348ms, non-child: 1s458ms, % non-child: 19.85%) ExecOption: Codegen Enabled - BuildTime: 588.191ms - GetNewBlockTime: 416.689us - GetResultsTime: 639.932ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.01 MB (321921024) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4333197) - RowsReturnedRate: 589.69 K/sec - SpilledPartitions: 0 (0) - UnpinTime: 2.988us AGGREGATION_NODE (id=5):(Total: 5s889ms, non-child: 1s465ms, % non-child: 24.89%) ExecOption: Codegen Enabled - BuildTime: 973.912ms - GetNewBlockTime: 889.393us - GetResultsTime: 477.632ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.02 MB (321937408) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4333197) - RowsReturnedRate: 735.73 K/sec - SpilledPartitions: 0 (0) - UnpinTime: 8.329us EXCHANGE_NODE (id=4):(Total: 4s424ms, non-child: 508.295ms, % non-child: 11.49%) BytesReceived(500.0ms): 0, 0, 0, 0, 3.52 MB, 7.96 MB, 14.76 MB, 21.18 MB, 27.35 MB, 34.12 MB, 37.85 MB - BytesReceived: 37.85 MB (39691875) - ConvertRowBatchTime: 489.64ms - DeserializeRowBatchTimer: 113.655ms - FirstBatchArrivalWaitTime: 1s668ms - PeakMemoryUsage: 0 - RowsReturned: 4.33M (4333197) - RowsReturnedRate: 979.47 K/sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns Instance 344463ad4603c165:7bf25b60e6357ea7 (host=jbapple-optiplex:22000):(Total: 9s685ms, non-child: 0ns, % non-child: 0.00%) MemoryUsage(500.0ms): 0, 6.30 MB, 6.30 MB, 6.30 MB, 6.30 MB, 43.34 MB, 72.86 MB, 119.30 MB, 157.28 MB, 195.27 MB, 234.25 MB, 272.23 MB, 327.98 MB, 327.62 MB, 307.08 MB, 268.69 MB, 211.14 MB, 172.77 MB, 115.18 MB, 57.63 MB, 19.27 MB ThreadUsage(500.0ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - PeakMemoryUsage: 329.18 MB (345169920) - PerHostPeakMemUsage: 361.41 MB (378966016) - PrepareTime: 223.79ms - RowsProduced: 4.33M (4332707) - TotalCpuTime: 6s264ms - TotalNetworkReceiveTime: 3s802ms - TotalNetworkSendTime: 36.668ms - TotalStorageWaitTime: 0ns CodeGen:(Total: 623.827ms, non-child: 623.827ms, % non-child: 100.00%) - CodegenTime: 9.455ms - CompileTime: 7.736ms - LoadTime: 21.634us - ModuleFileSize: 2.62 MB (2744720) - OptimizationTime: 403.354ms - PrepareTime: 211.781ms DataStreamSender (dst_id=6):(Total: 2s418ms, non-child: 2s418ms, % non-child: 100.00%) - BytesSent: 37.85 MB (39687164) - NetworkThroughput(*): 56.18 MB/sec - OverallThroughput: 15.65 MB/sec - PeakMemoryUsage: 76.00 KB (77824) - SerializeBatchTime: 1s017ms - ThriftTransmitTime(*): 673.742ms - UncompressedRowBatchSize: 49.59 MB (52000948) AGGREGATION_NODE (id=2):(Total: 7s262ms, non-child: 1s434ms, % non-child: 19.75%) ExecOption: Codegen Enabled - BuildTime: 588.289ms - GetNewBlockTime: 457.370us - GetResultsTime: 621.103ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.01 MB (321921024) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4332707) - RowsReturnedRate: 596.55 K/sec - SpilledPartitions: 0 (0) - UnpinTime: 5.73us AGGREGATION_NODE (id=5):(Total: 5s828ms, non-child: 1s502ms, % non-child: 25.78%) ExecOption: Codegen Enabled - BuildTime: 1s001ms - GetNewBlockTime: 604.323us - GetResultsTime: 486.705ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.02 MB (321937408) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4332707) - RowsReturnedRate: 743.38 K/sec - SpilledPartitions: 0 (0) - UnpinTime: 5.501us EXCHANGE_NODE (id=4):(Total: 4s325ms, non-child: 533.985ms, % non-child: 12.34%) BytesReceived(500.0ms): 0, 0, 0, 0, 3.10 MB, 7.57 MB, 12.83 MB, 17.43 MB, 22.52 MB, 28.00 MB, 32.74 MB - BytesReceived: 37.85 MB (39687431) - ConvertRowBatchTime: 514.756ms - DeserializeRowBatchTimer: 113.74ms - FirstBatchArrivalWaitTime: 1s612ms - PeakMemoryUsage: 0 - RowsReturned: 4.33M (4332707) - RowsReturnedRate: 1.00 M/sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns Fragment F00: Instance 344463ad4603c165:7bf25b60e6357eaa (host=jbapple-optiplex:22000):(Total: 5s458ms, non-child: 0ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:1/16.62 MB MemoryUsage(500.0ms): 3.18 MB, 48.90 MB, 134.38 MB, 222.45 MB, 307.08 MB, 268.71 MB, 211.13 MB, 191.96 MB, 134.38 MB, 96.00 MB, 57.64 MB ThreadUsage(500.0ms): 1, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.27 - PeakMemoryUsage: 315.55 MB (330879744) - PerHostPeakMemUsage: 361.41 MB (378966016) - PrepareTime: 312.760ms - RowsProduced: 4.33M (4334096) - TotalCpuTime: 7s255ms - TotalNetworkReceiveTime: 0ns - TotalNetworkSendTime: 26.224ms - TotalStorageWaitTime: 101.821ms CodeGen:(Total: 548.546ms, non-child: 548.546ms, % non-child: 100.00%) - CodegenTime: 5.603ms - CompileTime: 5.634ms - LoadTime: 28.76us - ModuleFileSize: 2.62 MB (2744720) - OptimizationTime: 255.364ms - PrepareTime: 286.591ms DataStreamSender (dst_id=4):(Total: 2s734ms, non-child: 2s734ms, % non-child: 100.00%) - BytesSent: 37.86 MB (39699745) - NetworkThroughput(*): 62.22 MB/sec - OverallThroughput: 13.85 MB/sec - PeakMemoryUsage: 76.00 KB (77824) - SerializeBatchTime: 1s159ms - ThriftTransmitTime(*): 608.486ms - UncompressedRowBatchSize: 49.61 MB (52017620) AGGREGATION_NODE (id=1):(Total: 2s720ms, non-child: 2s348ms, % non-child: 86.34%) ExecOption: Codegen Enabled - BuildTime: 1s305ms - GetNewBlockTime: 742.125us - GetResultsTime: 740.22ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.17 MB (322095176) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4334096) - RowsReturnedRate: 1.59 M/sec - SpilledPartitions: 0 (0) - UnpinTime: 4.994us HDFS_SCAN_NODE (id=0):(Total: 371.490ms, non-child: 371.490ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:1/16.62 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:1 ExecOption: Codegen enabled: 0 out of 1 BytesRead(500.0ms): 0, 16.72 MB, 16.72 MB, 16.72 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 16.72 MB (17529764) - BytesReadDataNodeCache: 0 - BytesReadLocal: 16.72 MB (17529764) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 16.72 MB (17529764) - DecompressionTime: 4.411ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 1 (1) - NumRowGroups: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 17.73 MB (18592024) - PerReadThreadRawHdfsThroughput: 2.58 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 4.33M (4334096) - RowsReturned: 4.33M (4334096) - RowsReturnedRate: 11.67 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 232 (232) - ScannerThreadsTotalWallClockTime: 1s657ms - MaterializeTupleTime(*): 1s546ms - ScannerThreadsSysTime: 0ns - ScannerThreadsUserTime: 1s488ms - ScannerThreadsVoluntaryContextSwitches: 253 (253) - TotalRawHdfsReadTime(*): 6.336ms - TotalReadThroughput: 8.36 MB/sec Instance 344463ad4603c165:7bf25b60e6357ea8 (host=jbapple-optiplex:22002):(Total: 5s416ms, non-child: 0ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:1/16.61 MB MemoryUsage(500.0ms): 3.18 MB, 97.44 MB, 192.68 MB, 288.17 MB, 287.88 MB, 249.52 MB, 211.12 MB, 172.77 MB, 134.39 MB, 96.02 MB, 57.62 MB ThreadUsage(500.0ms): 1, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.27 - PeakMemoryUsage: 315.90 MB (331240184) - PerHostPeakMemUsage: 343.29 MB (359964672) - PrepareTime: 218.265ms - RowsProduced: 4.33M (4332707) - TotalCpuTime: 7s063ms - TotalNetworkReceiveTime: 0ns - TotalNetworkSendTime: 18.139ms - TotalStorageWaitTime: 129.4ms CodeGen:(Total: 470.329ms, non-child: 470.329ms, % non-child: 100.00%) - CodegenTime: 2.552ms - CompileTime: 4.886ms - LoadTime: 24.605us - ModuleFileSize: 2.62 MB (2744720) - OptimizationTime: 259.220ms - PrepareTime: 205.411ms DataStreamSender (dst_id=4):(Total: 2s883ms, non-child: 2s883ms, % non-child: 100.00%) - BytesSent: 37.85 MB (39687431) - NetworkThroughput(*): 61.06 MB/sec - OverallThroughput: 13.13 MB/sec - PeakMemoryUsage: 76.00 KB (77824) - SerializeBatchTime: 1s229ms - ThriftTransmitTime(*): 619.873ms - UncompressedRowBatchSize: 49.59 MB (52000948) AGGREGATION_NODE (id=1):(Total: 2s528ms, non-child: 2s031ms, % non-child: 80.35%) ExecOption: Codegen Enabled - BuildTime: 1s041ms - GetNewBlockTime: 604.34us - GetResultsTime: 771.693ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.23 MB (322158856) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4332707) - RowsReturnedRate: 1.71 M/sec - SpilledPartitions: 0 (0) - UnpinTime: 2.984us HDFS_SCAN_NODE (id=0):(Total: 496.767ms, non-child: 496.767ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:1/16.61 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:1 ExecOption: Codegen enabled: 0 out of 1 BytesRead(500.0ms): 0, 16.71 MB, 16.71 MB, 16.71 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 16.71 MB (17524200) - BytesReadDataNodeCache: 0 - BytesReadLocal: 16.71 MB (17524200) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 16.71 MB (17524200) - DecompressionTime: 4.212ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 1 (1) - NumRowGroups: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 18.27 MB (19157288) - PerReadThreadRawHdfsThroughput: 1.72 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 4.33M (4332707) - RowsReturned: 4.33M (4332707) - RowsReturnedRate: 8.72 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 75 (75) - ScannerThreadsTotalWallClockTime: 1s524ms - MaterializeTupleTime(*): 1s385ms - ScannerThreadsSysTime: 4.0ms - ScannerThreadsUserTime: 1s344ms - ScannerThreadsVoluntaryContextSwitches: 285 (285) - TotalRawHdfsReadTime(*): 9.487ms - TotalReadThroughput: 8.36 MB/sec Instance 344463ad4603c165:7bf25b60e6357ea9 (host=jbapple-optiplex:22001):(Total: 4s837ms, non-child: 0ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:1/16.62 MB MemoryUsage(500.0ms): 3.18 MB, 59.47 MB, 182.57 MB, 278.39 MB, 287.87 MB, 249.50 MB, 191.96 MB, 153.57 MB, 96.02 MB, 38.46 MB ThreadUsage(500.0ms): 1, 2, 2, 2, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.30 - PeakMemoryUsage: 309.35 MB (324375304) - PerHostPeakMemUsage: 343.42 MB (360103936) - PrepareTime: 260.499ms - RowsProduced: 4.33M (4333197) - TotalCpuTime: 6s439ms - TotalNetworkReceiveTime: 0ns - TotalNetworkSendTime: 28.328ms - TotalStorageWaitTime: 112.521ms CodeGen:(Total: 516.320ms, non-child: 516.320ms, % non-child: 100.00%) - CodegenTime: 4.679ms - CompileTime: 4.991ms - LoadTime: 24.139us - ModuleFileSize: 2.62 MB (2744720) - OptimizationTime: 272.358ms - PrepareTime: 238.11ms DataStreamSender (dst_id=4):(Total: 2s453ms, non-child: 2s453ms, % non-child: 100.00%) - BytesSent: 37.85 MB (39691875) - NetworkThroughput(*): 59.14 MB/sec - OverallThroughput: 15.43 MB/sec - PeakMemoryUsage: 76.00 KB (77824) - SerializeBatchTime: 1s028ms - ThriftTransmitTime(*): 640.95ms - UncompressedRowBatchSize: 49.60 MB (52006828) AGGREGATION_NODE (id=1):(Total: 2s380ms, non-child: 1s900ms, % non-child: 79.83%) ExecOption: Codegen Enabled - BuildTime: 995.403ms - GetNewBlockTime: 633.421us - GetResultsTime: 651.323ms - HTResizeTime: 0ns - HashBuckets: 4.19M (4194304) - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 307.23 MB (322158856) - PinTime: 0ns - RowsRepartitioned: 0 (0) - RowsReturned: 4.33M (4333197) - RowsReturnedRate: 1.82 M/sec - SpilledPartitions: 0 (0) - UnpinTime: 4.765us HDFS_SCAN_NODE (id=0):(Total: 480.200ms, non-child: 480.200ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:1/16.62 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:1 ExecOption: Codegen enabled: 0 out of 1 BytesRead(500.0ms): 0, 16.71 MB, 16.71 MB, 16.71 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 16.71 MB (17526158) - BytesReadDataNodeCache: 0 - BytesReadLocal: 16.71 MB (17526158) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 16.71 MB (17526158) - DecompressionTime: 4.147ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 1 (1) - NumRowGroups: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 18.26 MB (19149096) - PerReadThreadRawHdfsThroughput: 2.69 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 4.33M (4333197) - RowsReturned: 4.33M (4333197) - RowsReturnedRate: 9.02 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 58 (58) - ScannerThreadsTotalWallClockTime: 1s459ms - MaterializeTupleTime(*): 1s337ms - ScannerThreadsSysTime: 0ns - ScannerThreadsUserTime: 1s316ms - ScannerThreadsVoluntaryContextSwitches: 168 (168) - TotalRawHdfsReadTime(*): 6.66ms - TotalReadThroughput: 8.36 MB/sec