Query (id=294c6905e4e7a318:11c53b1700000000): Summary: Session ID: 7a4deb1d0715d19e:e389776581fbdb83 Session Type: BEESWAX Start Time: 2016-12-15 00:04:32.192770000 End Time: 2016-12-15 00:04:45.330567000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.8.0-SNAPSHOT RELEASE (build 13897d05ea22d93b4a9f29acaf392611350f189b) User: mmokhtar Connected User: mmokhtar Delegated User: Network Address: 10.17.234.19:60820 Default Db: tpch_300_parquet Sql Statement: select * FROM (SELECT Rank() OVER( partition by l_linenumber ORDER BY l_orderkey) AS rank FROM lineitem WHERE l_shipdate < '1992-05-09') a WHERE rank < 10 Coordinator: vd1309.halxg.cloudera.com:22000 Query Options (non default): Plan: ---------------- Estimated Per-Host Requirements: Memory=616.00MB VCores=2 PLAN-ROOT SINK | 05:EXCHANGE [UNPARTITIONED] | hosts=7 per-host-mem=unavailable | tuple-ids=6,5 row-size=50B cardinality=17999891 | 03:SELECT | predicates: rank() < 10 | hosts=7 per-host-mem=0B | tuple-ids=6,5 row-size=50B cardinality=17999891 | 02:ANALYTIC | functions: rank() | partition by: l_linenumber | order by: l_orderkey ASC | window: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW | hosts=7 per-host-mem=0B | tuple-ids=6,5 row-size=50B cardinality=179998909 | 01:SORT | order by: l_linenumber ASC NULLS FIRST, l_orderkey ASC | hosts=7 per-host-mem=352.00MB | tuple-ids=6 row-size=42B cardinality=179998909 | 04:EXCHANGE [HASH(l_linenumber)] | hosts=7 per-host-mem=0B | tuple-ids=0 row-size=42B cardinality=179998909 | 00:SCAN HDFS [tpch_300_parquet.lineitem, RANDOM] partitions=1/1 files=259 size=63.71GB predicates: l_shipdate < '1992-05-09' table stats: 1799989091 rows total column stats: all hosts=7 per-host-mem=264.00MB tuple-ids=0 row-size=42B cardinality=179998909 ---------------- Estimated Per-Host Mem: 645922816 Estimated Per-Host VCores: 2 Request Pool: root.mmokhtar Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail --------------------------------------------------------------------------------------------------------------------- 05:EXCHANGE 1 156.063us 156.063us 63 18.00M 0 -1.00 B UNPARTITIONED 03:SELECT 7 63.424ms 126.486ms 63 18.00M 9.02 MB 0 02:ANALYTIC 7 1s582ms 3s140ms 50.88M 180.00M 25.03 MB 0 01:SORT 7 4s099ms 8s368ms 50.88M 180.00M 616.07 MB 352.00 MB 04:EXCHANGE 7 497.343ms 1s304ms 50.88M 180.00M 0 0 HASH(l_linenumber) 00:SCAN HDFS 7 89.198ms 94.103ms 50.88M 180.00M 1.21 GB 264.00 MB tpch_300_parquet.lineitem Planner Timeline: 14.041ms - Analysis finished: 2.081ms (2.081ms) - Equivalence classes computed: 2.286ms (205.178us) - Single node plan created: 8.255ms (5.968ms) - Runtime filters computed: 8.308ms (52.728us) - Distributed plan created: 8.455ms (147.135us) - Lineage info computed: 8.650ms (195.300us) - Planning finished: 14.041ms (5.390ms) Query Timeline: 13s183ms - Query submitted: 37.469us (37.469us) - Planning finished: 18.856ms (18.819ms) - Submit for admission: 20.020ms (1.163ms) - Completed admission: 20.794ms (773.463us) - Ready to start 15 fragment instances: 21.443ms (648.888us) - All 15 fragment instances started: 33.322ms (11.879ms) - Rows available: 5s904ms (5s871ms) - First row fetched: 5s985ms (80.483ms) - Unregister query: 13s137ms (7s152ms) - ComputeScanRangeAssignmentTimer: 399.044us ImpalaServer: - ClientFetchWaitTimer: 89.069ms - RowMaterializationTimer: 7s144ms Execution Profile 294c6905e4e7a318:11c53b1700000000:(Total: 13s026ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed Enabled ---------------------------------------------------------------------------------------------------------------------------- Fragment instance start latencies: Count: 15, 25th %-ile: 3ms, 50th %-ile: 4ms, 75th %-ile: 4ms, 90th %-ile: 5ms, 95th %-ile: 5ms, 99.9th %-ile: 5ms Per Node Peak Memory Usage: vd1310.halxg.cloudera.com:22000(1.55 GB) vd1313.halxg.cloudera.com:22000(1.33 GB) vd1312.halxg.cloudera.com:22000(1.44 GB) vd1311.halxg.cloudera.com:22000(1.22 GB) vd1314.halxg.cloudera.com:22000(1.54 GB) vd1316.halxg.cloudera.com:22000(1.24 GB) vd1309.halxg.cloudera.com:22000(1.39 GB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Coordinator Fragment F02: Instance 294c6905e4e7a318:11c53b1700000000 (host=vd1309.halxg.cloudera.com:22000):(Total: 13s098ms, non-child: 1.982ms, % non-child: 0.02%) MemoryUsage(500.000ms): 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 20.14 KB, 20.14 KB, 24.14 KB, 28.14 KB, 28.14 KB, 28.14 KB, 28.14 KB, 28.14 KB, 36.14 KB, 36.14 KB, 36.14 KB, 36.14 KB, 36.14 KB, 40.14 KB, 44.14 KB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 44.28 KB (45344) - PerHostPeakMemUsage: 1.39 GB (1492956815) - RowsProduced: 63 (63) - TotalNetworkReceiveTime: 13s069ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 13s069ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 1.000ms - TotalThreadsVoluntaryContextSwitches: 10 (10) Fragment Instance Lifecycle Timings: - ExecTime: 7s220ms - ExecTreeExecTime: 7s220ms - OpenTime: 5s849ms - ExecTreeOpenTime: 5s848ms - PrepareTime: 28.210ms - ExecTreePrepareTime: 62.951us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 41 (41) - BlocksRecycled: 36 (36) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 392.00 MB (411041792) - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 27.050ms, non-child: 27.050ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 26.306ms EXCHANGE_NODE (id=5):(Total: 13s069ms, non-child: 156.063us, % non-child: 0.00%) BytesReceived(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 280.00 B, 280.00 B, 561.00 B, 836.00 B, 836.00 B, 836.00 B, 836.00 B, 836.00 B, 1.35 KB, 1.35 KB, 1.35 KB, 1.35 KB, 1.35 KB, 1.63 KB, 1.91 KB - BytesReceived: 1.91 KB (1958) - ConvertRowBatchTime: 20.855us - DeserializeRowBatchTimer: 66.957us - FirstBatchArrivalWaitTime: 5s848ms - PeakMemoryUsage: 0 - RowsReturned: 63 (63) - RowsReturnedRate: 4.00 /sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F01:(Total: 8s311ms, non-child: 10.758ms, % non-child: 0.13%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:3s680ms max:13s101ms mean: 8s313ms stddev:3s575ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 7 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 331.05 MB (347131969) - PerHostPeakMemUsage: 1.39 GB (1490223747) - RowsProduced: 9 (9) - TotalNetworkReceiveTime: 2s282ms - TotalNetworkSendTime: 484.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 738 (738) - TotalThreadsTotalWallClockTime: 8s285ms - TotalThreadsSysTime: 256.960ms - TotalThreadsUserTime: 5s846ms - TotalThreadsVoluntaryContextSwitches: 8.06K (8061) Fragment Instance Lifecycle Timings: - ExecTime: 1s672ms - ExecTreeExecTime: 1s665ms - OpenTime: 6s613ms - ExecTreeOpenTime: 6s564ms - PrepareTime: 26.030ms - ExecTreePrepareTime: 216.366us DataStreamSender (dst_id=5):(Total: 80.815us, non-child: 80.815us, % non-child: 100.00%) - BytesSent: 279.00 B (279) - NetworkThroughput(*): 5.89 MB/sec - OverallThroughput: 2.63 MB/sec - PeakMemoryUsage: 6.09 KB (6240) - RowsReturned: 9 (9) - SerializeBatchTime: 26.851us - TransmitDataRPCTime: 32.052us - UncompressedRowBatchSize: 548.00 B (548) CodeGen:(Total: 71.064ms, non-child: 71.064ms, % non-child: 100.00%) - CodegenTime: 561.662us - CompileTime: 8.060ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 56 (56) - NumInstructions: 943 (943) - OptimizationTime: 39.673ms - PeakMemoryUsage: 471.50 KB (482816) - PrepareTime: 22.599ms SELECT_NODE (id=3):(Total: 8s229ms, non-child: 63.424ms, % non-child: 0.77%) - PeakMemoryUsage: 6.45 MB (6758400) - RowsReturned: 9 (9) - RowsReturnedRate: 0 ANALYTIC_EVAL_NODE (id=2):(Total: 8s166ms, non-child: 1s582ms, % non-child: 19.38%) - EvaluationTime: 1s576ms - GetNewBlockTime: 243.099us - PeakMemoryUsage: 20.17 MB (21148818) - PinTime: 0.000ns - RowsReturned: 7.27M (7269256) - RowsReturnedRate: 704.21 K/sec - UnpinTime: 14.876us SORT_NODE (id=1):(Total: 6s583ms, non-child: 4s099ms, % non-child: 62.26%) - InMemorySortTime: 2s959ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 316.62 MB (332003097) - RowsReturned: 7.27M (7269256) - RowsReturnedRate: 902.93 K/sec - SortDataSize: 298.10 MB (312578014) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) EXCHANGE_NODE (id=4):(Total: 2s484ms, non-child: 2s484ms, % non-child: 100.00%) - BytesReceived: 123.10 MB (129084738) - ConvertRowBatchTime: 182.093ms - DeserializeRowBatchTimer: 705.012ms - FirstBatchArrivalWaitTime: 1s052ms - PeakMemoryUsage: 0 - RowsReturned: 7.27M (7269256) - RowsReturnedRate: 4.29 M/sec - SendersBlockedTimer: 320.362ms - SendersBlockedTotalTimer(*): 1s359ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 63 (63) - BlocksRecycled: 57 (57) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 570.67 MB (598387370) - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns Fragment F01: Instance 294c6905e4e7a318:11c53b170000000c (host=vd1314.halxg.cloudera.com:22000):(Total: 13s099ms, non-child: 16.649ms, % non-child: 0.13%) MemoryUsage(500.000ms): 46.09 MB, 145.23 MB, 221.65 MB, 304.20 MB, 384.13 MB, 472.30 MB, 560.02 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 562.07 MB, 493.07 MB, 424.07 MB, 354.07 MB, 276.07 MB, 219.07 MB ThreadUsage(500.000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 636.08 MB (666974876) - PerHostPeakMemUsage: 1.54 GB (1654844496) - RowsProduced: 18 (18) - TotalNetworkReceiveTime: 966.975ms - TotalNetworkSendTime: 586.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 1.32K (1321) - TotalThreadsTotalWallClockTime: 13s070ms - TotalThreadsSysTime: 508.923ms - TotalThreadsUserTime: 11s746ms - TotalThreadsVoluntaryContextSwitches: 11.79K (11793) Fragment Instance Lifecycle Timings: - ExecTime: 3s319ms - ExecTreeExecTime: 3s306ms - OpenTime: 9s751ms - ExecTreeOpenTime: 9s701ms - PrepareTime: 28.376ms - ExecTreePrepareTime: 214.690us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 79 (79) - BlocksRecycled: 73 (73) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 696.00 MB (729808896) - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=5):(Total: 153.686us, non-child: 153.686us, % non-child: 100.00%) - BytesSent: 562.00 B (562) - NetworkThroughput(*): 10.48 MB/sec - OverallThroughput: 3.49 MB/sec - PeakMemoryUsage: 6.09 KB (6240) - RowsReturned: 18 (18) - SerializeBatchTime: 80.762us - TransmitDataRPCTime: 51.122us - UncompressedRowBatchSize: 1.07 KB (1096) CodeGen:(Total: 74.427ms, non-child: 74.427ms, % non-child: 100.00%) - CodegenTime: 560.704us - CompileTime: 8.076ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 56 (56) - NumInstructions: 943 (943) - OptimizationTime: 40.086ms - PeakMemoryUsage: 471.50 KB (482816) - PrepareTime: 25.588ms SELECT_NODE (id=3):(Total: 13s008ms, non-child: 126.486ms, % non-child: 0.97%) - PeakMemoryUsage: 9.02 MB (9453568) - RowsReturned: 18 (18) - RowsReturnedRate: 1.00 /sec ANALYTIC_EVAL_NODE (id=2):(Total: 12s881ms, non-child: 3s140ms, % non-child: 24.38%) - EvaluationTime: 3s129ms - GetNewBlockTime: 489.190us - PeakMemoryUsage: 25.03 MB (26243072) - PinTime: 0.000ns - RowsReturned: 14.54M (14537487) - RowsReturnedRate: 1.13 M/sec - UnpinTime: 32.264us SORT_NODE (id=1):(Total: 9s741ms, non-child: 8s368ms, % non-child: 85.90%) ExecOption: Codegen Enabled - InMemorySortTime: 6s098ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 616.07 MB (645994992) - RowsReturned: 14.54M (14537487) - RowsReturnedRate: 1.49 M/sec - SortDataSize: 596.15 MB (625111941) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) EXCHANGE_NODE (id=4):(Total: 1s373ms, non-child: 1s138ms, % non-child: 82.90%) BytesReceived(500.000ms): 12.12 MB, 51.80 MB, 87.16 MB, 121.43 MB, 154.78 MB, 189.46 MB, 226.80 MB, 248.73 MB, 248.73 MB, 248.73 MB, 248.73 MB, 248.73 MB, 248.73 MB, 248.73 MB, 248.73 MB, 248.73 MB, 248.73 MB, 248.73 MB, 248.73 MB, 248.73 MB - BytesReceived: 248.73 MB (260817416) - ConvertRowBatchTime: 366.970ms - DeserializeRowBatchTimer: 1s524ms - FirstBatchArrivalWaitTime: 27.607ms - PeakMemoryUsage: 0 - RowsReturned: 14.54M (14537487) - RowsReturnedRate: 10.59 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Instance 294c6905e4e7a318:11c53b170000000e (host=vd1310.halxg.cloudera.com:22000):(Total: 12s905ms, non-child: 16.914ms, % non-child: 0.13%) MemoryUsage(500.000ms): 75.29 MB, 164.98 MB, 245.03 MB, 333.03 MB, 413.03 MB, 498.45 MB, 580.85 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 616.07 MB, 595.08 MB, 526.07 MB, 457.07 MB, 379.07 MB, 310.07 MB, 241.07 MB, 182.07 MB ThreadUsage(500.000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 636.08 MB (666974876) - PerHostPeakMemUsage: 1.55 GB (1664404467) - RowsProduced: 18 (18) - TotalNetworkReceiveTime: 980.280ms - TotalNetworkSendTime: 806.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 1.31K (1309) - TotalThreadsTotalWallClockTime: 12s886ms - TotalThreadsSysTime: 524.921ms - TotalThreadsUserTime: 11s618ms - TotalThreadsVoluntaryContextSwitches: 14.32K (14322) Fragment Instance Lifecycle Timings: - ExecTime: 3s280ms - ExecTreeExecTime: 3s266ms - OpenTime: 9s606ms - ExecTreeOpenTime: 9s553ms - PrepareTime: 18.607ms - ExecTreePrepareTime: 222.532us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 79 (79) - BlocksRecycled: 73 (73) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 696.00 MB (729808896) - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=5):(Total: 147.536us, non-child: 147.536us, % non-child: 100.00%) - BytesSent: 560.00 B (560) - NetworkThroughput(*): 9.03 MB/sec - OverallThroughput: 3.62 MB/sec - PeakMemoryUsage: 6.09 KB (6240) - RowsReturned: 18 (18) - SerializeBatchTime: 70.204us - TransmitDataRPCTime: 59.118us - UncompressedRowBatchSize: 1.07 KB (1096) CodeGen:(Total: 68.013ms, non-child: 68.013ms, % non-child: 100.00%) - CodegenTime: 575.692us - CompileTime: 9.937ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 56 (56) - NumInstructions: 943 (943) - OptimizationTime: 41.214ms - PeakMemoryUsage: 471.50 KB (482816) - PrepareTime: 16.110ms SELECT_NODE (id=3):(Total: 12s820ms, non-child: 126.213ms, % non-child: 0.98%) - PeakMemoryUsage: 9.02 MB (9453568) - RowsReturned: 18 (18) - RowsReturnedRate: 1.00 /sec ANALYTIC_EVAL_NODE (id=2):(Total: 12s693ms, non-child: 3s101ms, % non-child: 24.43%) - EvaluationTime: 3s089ms - GetNewBlockTime: 658.122us - PeakMemoryUsage: 25.03 MB (26243072) - PinTime: 0.000ns - RowsReturned: 14.54M (14537452) - RowsReturnedRate: 1.15 M/sec - UnpinTime: 28.390us SORT_NODE (id=1):(Total: 9s592ms, non-child: 8s205ms, % non-child: 85.54%) ExecOption: Codegen Enabled - InMemorySortTime: 5s942ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 616.07 MB (645994992) - RowsReturned: 14.54M (14537452) - RowsReturnedRate: 1.52 M/sec - SortDataSize: 596.15 MB (625110436) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) EXCHANGE_NODE (id=4):(Total: 1s386ms, non-child: 1s304ms, % non-child: 94.09%) BytesReceived(500.000ms): 22.89 MB, 59.62 MB, 94.18 MB, 127.49 MB, 160.47 MB, 196.57 MB, 230.74 MB, 244.42 MB, 244.42 MB, 244.42 MB, 244.42 MB, 244.42 MB, 244.42 MB, 244.42 MB, 244.42 MB, 244.42 MB, 244.42 MB, 244.42 MB, 244.42 MB - BytesReceived: 244.42 MB (256287961) - ConvertRowBatchTime: 361.523ms - DeserializeRowBatchTimer: 1s411ms - FirstBatchArrivalWaitTime: 34.574ms - PeakMemoryUsage: 0 - RowsReturned: 14.54M (14537452) - RowsReturnedRate: 10.48 M/sec - SendersBlockedTimer: 2s242ms - SendersBlockedTotalTimer(*): 9s514ms Instance 294c6905e4e7a318:11c53b170000000b (host=vd1312.halxg.cloudera.com:22000):(Total: 9s301ms, non-child: 11.739ms, % non-child: 0.13%) MemoryUsage(500.000ms): 64.10 MB, 112.06 MB, 168.06 MB, 216.15 MB, 272.10 MB, 328.06 MB, 376.08 MB, 384.08 MB, 384.08 MB, 384.08 MB, 384.08 MB, 384.08 MB, 384.08 MB, 384.08 MB, 374.07 MB, 304.07 MB, 227.07 MB, 158.07 MB ThreadUsage(500.000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 404.08 MB (423705244) - PerHostPeakMemUsage: 1.44 GB (1550180241) - RowsProduced: 9 (9) - TotalNetworkReceiveTime: 2s173ms - TotalNetworkSendTime: 394.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 923 (923) - TotalThreadsTotalWallClockTime: 9s274ms - TotalThreadsSysTime: 291.955ms - TotalThreadsUserTime: 6s930ms - TotalThreadsVoluntaryContextSwitches: 12.37K (12372) Fragment Instance Lifecycle Timings: - ExecTime: 1s994ms - ExecTreeExecTime: 1s985ms - OpenTime: 7s280ms - ExecTreeOpenTime: 7s234ms - PrepareTime: 26.961ms - ExecTreePrepareTime: 200.908us DataStreamSender (dst_id=5):(Total: 61.618us, non-child: 61.618us, % non-child: 100.00%) - BytesSent: 275.00 B (275) - NetworkThroughput(*): 9.23 MB/sec - OverallThroughput: 4.26 MB/sec - PeakMemoryUsage: 6.09 KB (6240) - RowsReturned: 9 (9) - SerializeBatchTime: 10.884us - TransmitDataRPCTime: 28.428us - UncompressedRowBatchSize: 548.00 B (548) CodeGen:(Total: 69.972ms, non-child: 69.972ms, % non-child: 100.00%) - CodegenTime: 555.988us - CompileTime: 7.102ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 56 (56) - NumInstructions: 943 (943) - OptimizationTime: 38.167ms - PeakMemoryUsage: 471.50 KB (482816) - PrepareTime: 23.940ms SELECT_NODE (id=3):(Total: 9s219ms, non-child: 79.470ms, % non-child: 0.86%) - PeakMemoryUsage: 9.02 MB (9453568) - RowsReturned: 9 (9) - RowsReturnedRate: 0 ANALYTIC_EVAL_NODE (id=2):(Total: 9s140ms, non-child: 1s882ms, % non-child: 20.59%) - EvaluationTime: 1s874ms - GetNewBlockTime: 205.906us - PeakMemoryUsage: 25.03 MB (26243072) - PinTime: 0.000ns - RowsReturned: 9.09M (9085834) - RowsReturnedRate: 994.03 K/sec - UnpinTime: 19.386us SORT_NODE (id=1):(Total: 7s258ms, non-child: 4s858ms, % non-child: 66.94%) ExecOption: Codegen Enabled - InMemorySortTime: 3s627ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 384.07 MB (402725360) - RowsReturned: 9.09M (9085834) - RowsReturnedRate: 1.25 M/sec - SortDataSize: 372.59 MB (390690862) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) EXCHANGE_NODE (id=4):(Total: 2s399ms, non-child: 434.262ms, % non-child: 18.10%) BytesReceived(500.000ms): 19.97 MB, 41.64 MB, 62.95 MB, 83.63 MB, 104.21 MB, 127.44 MB, 148.49 MB, 152.25 MB, 152.25 MB, 152.25 MB, 152.25 MB, 152.25 MB, 152.25 MB, 152.25 MB - BytesReceived: 152.25 MB (159647156) - ConvertRowBatchTime: 203.707ms - DeserializeRowBatchTimer: 844.807ms - FirstBatchArrivalWaitTime: 34.050ms - PeakMemoryUsage: 0 - RowsReturned: 9.09M (9085834) - RowsReturnedRate: 3.79 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Instance 294c6905e4e7a318:11c53b170000000a (host=vd1309.halxg.cloudera.com:22000):(Total: 8s450ms, non-child: 15.477ms, % non-child: 0.18%) MemoryUsage(500.000ms): 48.08 MB, 96.08 MB, 136.08 MB, 176.13 MB, 216.10 MB, 264.17 MB, 304.11 MB, 312.05 MB, 312.05 MB, 312.05 MB, 312.05 MB, 312.05 MB, 312.05 MB, 286.07 MB, 220.07 MB, 169.07 MB, 102.07 MB ThreadUsage(500.000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 332.08 MB (348207772) - PerHostPeakMemUsage: 1.39 GB (1492956815) - RowsProduced: 9 (9) - TotalNetworkReceiveTime: 2s033ms - TotalNetworkSendTime: 579.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 915 (915) - TotalThreadsTotalWallClockTime: 8s417ms - TotalThreadsSysTime: 279.957ms - TotalThreadsUserTime: 6s204ms - TotalThreadsVoluntaryContextSwitches: 9.46K (9457) Fragment Instance Lifecycle Timings: - ExecTime: 1s875ms - ExecTreeExecTime: 1s867ms - OpenTime: 6s542ms - ExecTreeOpenTime: 6s492ms - PrepareTime: 32.077ms - ExecTreePrepareTime: 212.943us DataStreamSender (dst_id=5):(Total: 77.335us, non-child: 77.335us, % non-child: 100.00%) - BytesSent: 281.00 B (281) - NetworkThroughput(*): 6.12 MB/sec - OverallThroughput: 3.47 MB/sec - PeakMemoryUsage: 6.09 KB (6240) - RowsReturned: 9 (9) - SerializeBatchTime: 13.437us - TransmitDataRPCTime: 43.778us - UncompressedRowBatchSize: 548.00 B (548) CodeGen:(Total: 73.974ms, non-child: 73.974ms, % non-child: 100.00%) - CodegenTime: 591.532us - CompileTime: 8.203ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 56 (56) - NumInstructions: 943 (943) - OptimizationTime: 40.739ms - PeakMemoryUsage: 471.50 KB (482816) - PrepareTime: 24.302ms SELECT_NODE (id=3):(Total: 8s360ms, non-child: 63.946ms, % non-child: 0.76%) - PeakMemoryUsage: 9.02 MB (9453568) - RowsReturned: 9 (9) - RowsReturnedRate: 1.00 /sec ANALYTIC_EVAL_NODE (id=2):(Total: 8s296ms, non-child: 1s784ms, % non-child: 21.50%) - EvaluationTime: 1s778ms - GetNewBlockTime: 104.139us - PeakMemoryUsage: 25.03 MB (26243072) - PinTime: 0.000ns - RowsReturned: 7.27M (7270783) - RowsReturnedRate: 876.33 K/sec - UnpinTime: 14.265us SORT_NODE (id=1):(Total: 6s512ms, non-child: 4s255ms, % non-child: 65.34%) ExecOption: Codegen Enabled - InMemorySortTime: 2s897ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 312.07 MB (327227888) - RowsReturned: 7.27M (7270783) - RowsReturnedRate: 1.12 M/sec - SortDataSize: 298.16 MB (312643669) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) EXCHANGE_NODE (id=4):(Total: 2s256ms, non-child: 351.693ms, % non-child: 15.58%) BytesReceived(500.000ms): 13.14 MB, 31.27 MB, 48.52 MB, 65.28 MB, 81.90 MB, 100.55 MB, 117.51 MB, 122.98 MB, 122.98 MB, 122.98 MB, 122.98 MB, 122.98 MB, 122.98 MB - BytesReceived: 122.98 MB (128957920) - ConvertRowBatchTime: 202.862ms - DeserializeRowBatchTimer: 638.693ms - FirstBatchArrivalWaitTime: 26.134ms - PeakMemoryUsage: 0 - RowsReturned: 7.27M (7270783) - RowsReturnedRate: 3.22 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Instance 294c6905e4e7a318:11c53b170000000d (host=vd1313.halxg.cloudera.com:22000):(Total: 7s069ms, non-child: 8.091ms, % non-child: 0.11%) MemoryUsage(500.000ms): 32.10 MB, 64.09 MB, 96.08 MB, 128.13 MB, 160.08 MB, 184.06 MB, 216.08 MB, 240.06 MB, 240.06 MB, 240.06 MB, 240.06 MB, 240.06 MB, 198.07 MB, 128.07 MB ThreadUsage(500.000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 260.08 MB (272710300) - PerHostPeakMemUsage: 1.33 GB (1430088899) - RowsProduced: 9 (9) - TotalNetworkReceiveTime: 2s615ms - TotalNetworkSendTime: 478.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 686 (686) - TotalThreadsTotalWallClockTime: 7s043ms - TotalThreadsSysTime: 190.971ms - TotalThreadsUserTime: 4s333ms - TotalThreadsVoluntaryContextSwitches: 8.47K (8474) Fragment Instance Lifecycle Timings: - ExecTime: 1s236ms - ExecTreeExecTime: 1s231ms - OpenTime: 5s806ms - ExecTreeOpenTime: 5s759ms - PrepareTime: 25.552ms - ExecTreePrepareTime: 242.650us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 32 (32) - BlocksRecycled: 26 (26) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 320.00 MB (335544320) - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=5):(Total: 74.160us, non-child: 74.160us, % non-child: 100.00%) - BytesSent: 280.00 B (280) - NetworkThroughput(*): 6.37 MB/sec - OverallThroughput: 3.60 MB/sec - PeakMemoryUsage: 6.09 KB (6240) - RowsReturned: 9 (9) - SerializeBatchTime: 12.674us - TransmitDataRPCTime: 41.918us - UncompressedRowBatchSize: 548.00 B (548) CodeGen:(Total: 69.748ms, non-child: 69.748ms, % non-child: 100.00%) - CodegenTime: 503.292us - CompileTime: 7.646ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 56 (56) - NumInstructions: 943 (943) - OptimizationTime: 38.232ms - PeakMemoryUsage: 471.50 KB (482816) - PrepareTime: 23.194ms SELECT_NODE (id=3):(Total: 6s991ms, non-child: 47.774ms, % non-child: 0.68%) - PeakMemoryUsage: 9.02 MB (9453568) - RowsReturned: 9 (9) - RowsReturnedRate: 1.00 /sec ANALYTIC_EVAL_NODE (id=2):(Total: 6s943ms, non-child: 1s169ms, % non-child: 16.84%) - EvaluationTime: 1s165ms - GetNewBlockTime: 226.004us - PeakMemoryUsage: 25.03 MB (26243072) - PinTime: 0.000ns - RowsReturned: 5.45M (5453237) - RowsReturnedRate: 785.37 K/sec - UnpinTime: 9.828us SORT_NODE (id=1):(Total: 5s774ms, non-child: 3s003ms, % non-child: 52.02%) ExecOption: Codegen Enabled - InMemorySortTime: 2s150ms - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 240.07 MB (251730416) - RowsReturned: 5.45M (5453237) - RowsReturnedRate: 944.43 K/sec - SortDataSize: 223.63 MB (234489191) - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) EXCHANGE_NODE (id=4):(Total: 2s770ms, non-child: 252.043ms, % non-child: 9.10%) BytesReceived(500.000ms): 4.52 MB, 19.43 MB, 32.68 MB, 45.54 MB, 58.05 MB, 71.05 MB, 85.07 MB, 93.35 MB, 93.35 MB, 93.35 MB, 93.35 MB, 93.35 MB - BytesReceived: 93.35 MB (97882717) - ConvertRowBatchTime: 139.589ms - DeserializeRowBatchTimer: 515.943ms - FirstBatchArrivalWaitTime: 38.751ms - PeakMemoryUsage: 0 - RowsReturned: 5.45M (5453237) - RowsReturnedRate: 1.97 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Instance 294c6905e4e7a318:11c53b1700000008 (host=vd1316.halxg.cloudera.com:22000):(Total: 3s678ms, non-child: 3.228ms, % non-child: 0.09%) MemoryUsage(500.000ms): 24.03 MB, 24.03 MB, 24.03 MB, 24.03 MB, 24.03 MB, 24.03 MB, 24.03 MB ThreadUsage(500.000ms): 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 24.49 MB (25675360) - PerHostPeakMemUsage: 1.24 GB (1328663452) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 3s605ms - TotalNetworkSendTime: 312.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 6 (6) - TotalThreadsTotalWallClockTime: 3s653ms - TotalThreadsSysTime: 1.000ms - TotalThreadsUserTime: 46.993ms - TotalThreadsVoluntaryContextSwitches: 9 (9) Fragment Instance Lifecycle Timings: - ExecTime: 157.068us - ExecTreeExecTime: 23.406us - OpenTime: 3s653ms - ExecTreeOpenTime: 3s606ms - PrepareTime: 24.894ms - ExecTreePrepareTime: 203.110us DataStreamSender (dst_id=5):(Total: 23.566us, non-child: 23.566us, % non-child: 100.00%) - BytesSent: 0 - NetworkThroughput(*): 0.00 /sec - OverallThroughput: 0.00 /sec - PeakMemoryUsage: 6.09 KB (6240) - RowsReturned: 0 (0) - SerializeBatchTime: 0.000ns - TransmitDataRPCTime: 0.000ns - UncompressedRowBatchSize: 0 CodeGen:(Total: 68.909ms, non-child: 68.909ms, % non-child: 100.00%) - CodegenTime: 528.968us - CompileTime: 7.263ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 56 (56) - NumInstructions: 943 (943) - OptimizationTime: 38.843ms - PeakMemoryUsage: 471.50 KB (482816) - PrepareTime: 21.972ms SELECT_NODE (id=3):(Total: 3s606ms, non-child: 44.696us, % non-child: 0.00%) - PeakMemoryUsage: 20.00 KB (20480) - RowsReturned: 0 (0) - RowsReturnedRate: 0 ANALYTIC_EVAL_NODE (id=2):(Total: 3s606ms, non-child: 134.688us, % non-child: 0.00%) - EvaluationTime: 884.000ns - GetNewBlockTime: 8.448us - PeakMemoryUsage: 8.02 MB (8413184) - PinTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 - UnpinTime: 0.000ns SORT_NODE (id=1):(Total: 3s606ms, non-child: 813.766us, % non-child: 0.02%) ExecOption: Codegen Enabled - InMemorySortTime: 676.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 24.01 MB (25174016) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) EXCHANGE_NODE (id=4):(Total: 3s605ms, non-child: 5.394us, % non-child: 0.00%) BytesReceived(500.000ms): 0, 0, 0, 0, 0, 0, 0 - BytesReceived: 0 - ConvertRowBatchTime: 966.000ns - DeserializeRowBatchTimer: 0.000ns - FirstBatchArrivalWaitTime: 3s605ms - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Instance 294c6905e4e7a318:11c53b1700000009 (host=vd1311.halxg.cloudera.com:22000):(Total: 3s678ms, non-child: 3.211ms, % non-child: 0.09%) MemoryUsage(500.000ms): 24.03 MB, 24.03 MB, 24.03 MB, 24.03 MB, 24.03 MB, 24.03 MB, 24.03 MB ThreadUsage(500.000ms): 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 24.49 MB (25675360) - PerHostPeakMemUsage: 1.22 GB (1310427859) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 3s601ms - TotalNetworkSendTime: 234.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 3s651ms - TotalThreadsSysTime: 999.000us - TotalThreadsUserTime: 48.992ms - TotalThreadsVoluntaryContextSwitches: 6 (6) Fragment Instance Lifecycle Timings: - ExecTime: 122.094us - ExecTreeExecTime: 11.150us - OpenTime: 3s652ms - ExecTreeOpenTime: 3s602ms - PrepareTime: 25.742ms - ExecTreePrepareTime: 217.732us DataStreamSender (dst_id=5):(Total: 27.806us, non-child: 27.806us, % non-child: 100.00%) - BytesSent: 0 - NetworkThroughput(*): 0.00 /sec - OverallThroughput: 0.00 /sec - PeakMemoryUsage: 6.09 KB (6240) - RowsReturned: 0 (0) - SerializeBatchTime: 0.000ns - TransmitDataRPCTime: 0.000ns - UncompressedRowBatchSize: 0 CodeGen:(Total: 72.409ms, non-child: 72.409ms, % non-child: 100.00%) - CodegenTime: 615.458us - CompileTime: 8.193ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 56 (56) - NumInstructions: 943 (943) - OptimizationTime: 40.429ms - PeakMemoryUsage: 471.50 KB (482816) - PrepareTime: 23.089ms SELECT_NODE (id=3):(Total: 3s602ms, non-child: 38.172us, % non-child: 0.00%) - PeakMemoryUsage: 20.00 KB (20480) - RowsReturned: 0 (0) - RowsReturnedRate: 0 ANALYTIC_EVAL_NODE (id=2):(Total: 3s602ms, non-child: 135.912us, % non-child: 0.00%) - EvaluationTime: 866.000ns - GetNewBlockTime: 9.884us - PeakMemoryUsage: 8.02 MB (8413184) - PinTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 - UnpinTime: 0.000ns SORT_NODE (id=1):(Total: 3s602ms, non-child: 790.962us, % non-child: 0.02%) ExecOption: Codegen Enabled - InMemorySortTime: 738.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 24.01 MB (25174016) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) EXCHANGE_NODE (id=4):(Total: 3s601ms, non-child: 5.330us, % non-child: 0.00%) BytesReceived(500.000ms): 0, 0, 0, 0, 0, 0, 0 - BytesReceived: 0 - ConvertRowBatchTime: 1.354us - DeserializeRowBatchTimer: 0.000ns - FirstBatchArrivalWaitTime: 3s601ms - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 3s546ms, non-child: 2s131ms, % non-child: 60.10%) split sizes: min: 8.79 GB, max: 9.33 GB, avg: 9.10 GB, stddev: 152.13 MB completion times: min:3s359ms max:3s681ms mean: 3s548ms stddev:93.993ms execution rates: min:2.45 GB/sec max:2.71 GB/sec mean:2.57 GB/sec stddev:74.50 MB/sec num instances: 7 - AverageThreadTokens: 36.50 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.19 GB (1277917371) - PerHostPeakMemUsage: 1.39 GB (1490223747) - RowsProduced: 7.27M (7269256) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 491.629ms - TotalStorageWaitTime: 3s118ms - TotalThreadsInvoluntaryContextSwitches: 7.91K (7912) - TotalThreadsTotalWallClockTime: 2m6s - TotalThreadsSysTime: 2s509ms - TotalThreadsUserTime: 30s651ms - TotalThreadsVoluntaryContextSwitches: 9.60K (9603) Fragment Instance Lifecycle Timings: - ExecTime: 3s487ms - ExecTreeExecTime: 93.226ms - OpenTime: 33.753ms - ExecTreeOpenTime: 34.735us - PrepareTime: 26.140ms - ExecTreePrepareTime: 346.680us DataStreamSender (dst_id=4):(Total: 1s271ms, non-child: 1s271ms, % non-child: 100.00%) - BytesSent: 123.10 MB (129084738) - NetworkThroughput(*): 323.73 MB/sec - OverallThroughput: 96.98 MB/sec - PeakMemoryUsage: 167.12 KB (171136) - RowsReturned: 7.27M (7269256) - SerializeBatchTime: 888.125ms - TransmitDataRPCTime: 380.658ms - UncompressedRowBatchSize: 325.88 MB (341713672) CodeGen:(Total: 54.815ms, non-child: 54.815ms, % non-child: 100.00%) - CodegenTime: 668.242us - CompileTime: 8.308ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 19 (19) - NumInstructions: 303 (303) - OptimizationTime: 24.221ms - PeakMemoryUsage: 151.50 KB (155136) - PrepareTime: 21.609ms HDFS_SCAN_NODE (id=0):(Total: 89.198ms, non-child: 89.198ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 35.50 - BytesRead: 821.07 MB (860950004) - BytesReadDataNodeCache: 0 - BytesReadLocal: 821.07 MB (860950004) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 821.07 MB (860950004) - DecompressionTime: 4s132ms - MaxCompressedTextFileLength: 0 - NumColumns: 3 (3) - NumDisksAccessed: 7 (7) - NumRowGroups: 37 (37) - NumScannerThreadsStarted: 37 (37) - PeakMemoryUsage: 1.19 GB (1276243840) - PerReadThreadRawHdfsThroughput: 1.42 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 46.192ms - RowBatchQueuePutWaitTime: 1m29s - RowsRead: 257.14M (257141298) - RowsReturned: 7.27M (7269256) - RowsReturnedRate: 81.53 M/sec - ScanRangesComplete: 37 (37) - ScannerThreadsInvoluntaryContextSwitches: 7.54K (7542) - ScannerThreadsTotalWallClockTime: 2m3s - MaterializeTupleTime(*): 25s518ms - ScannerThreadsSysTime: 1s919ms - ScannerThreadsUserTime: 28s251ms - ScannerThreadsVoluntaryContextSwitches: 7.92K (7925) - TotalRawHdfsReadTime(*): 567.275ms - TotalReadThroughput: 240.17 MB/sec BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 18 (18) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 146.67 MB (153791146) - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns Fragment F00: Instance 294c6905e4e7a318:11c53b1700000001 (host=vd1316.halxg.cloudera.com:22000):(Total: 3s680ms, non-child: 2s260ms, % non-child: 61.41%) Hdfs split stats (:<# splits>/): 5:3/759.72 MB 7:5/1.22 GB 6:5/1.19 GB 4:4/1012.96 MB 3:6/1.48 GB 2:6/1.48 GB 1:4/1012.99 MB 0:5/1.24 GB MemoryUsage(500.000ms): 1.20 GB, 1.21 GB, 1.21 GB, 1.21 GB, 1.21 GB, 1.02 GB, 627.54 MB ThreadUsage(500.000ms): 39, 39, 39, 39, 39, 39, 38 - AverageThreadTokens: 38.86 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.21 GB (1303466240) - PerHostPeakMemUsage: 1.24 GB (1328663452) - RowsProduced: 7.45M (7452417) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 598.509ms - TotalStorageWaitTime: 2s977ms - TotalThreadsInvoluntaryContextSwitches: 8.62K (8625) - TotalThreadsTotalWallClockTime: 2m16s - TotalThreadsSysTime: 2s599ms - TotalThreadsUserTime: 31s552ms - TotalThreadsVoluntaryContextSwitches: 9.85K (9850) Fragment Instance Lifecycle Timings: - ExecTime: 3s621ms - ExecTreeExecTime: 97.874ms - OpenTime: 34.448ms - ExecTreeOpenTime: 30.112us - PrepareTime: 24.175ms - ExecTreePrepareTime: 334.094us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 3 (3) - BlocksRecycled: 1 (1) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 32.00 MB (33554432) - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 1s271ms, non-child: 1s271ms, % non-child: 100.00%) - BytesSent: 126.19 MB (132317560) - NetworkThroughput(*): 314.06 MB/sec - OverallThroughput: 99.22 MB/sec - PeakMemoryUsage: 167.12 KB (171136) - RowsReturned: 7.45M (7452417) - SerializeBatchTime: 868.107ms - TransmitDataRPCTime: 401.801ms - UncompressedRowBatchSize: 334.09 MB (350323707) CodeGen:(Total: 54.292ms, non-child: 54.292ms, % non-child: 100.00%) - CodegenTime: 559.734us - CompileTime: 8.527ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 19 (19) - NumInstructions: 303 (303) - OptimizationTime: 24.759ms - PeakMemoryUsage: 151.50 KB (155136) - PrepareTime: 20.203ms HDFS_SCAN_NODE (id=0):(Total: 94.103ms, non-child: 94.103ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:3/759.72 MB 7:5/1.22 GB 6:5/1.19 GB 4:4/1012.96 MB 3:6/1.48 GB 2:6/1.48 GB 1:4/1012.99 MB 0:5/1.24 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 38 out of 38 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/SNAPPY:114 BytesRead(500.000ms): 841.93 MB, 841.93 MB, 841.93 MB, 841.93 MB, 841.93 MB, 841.93 MB, 841.93 MB - FooterProcessingTime: (Avg: 24.520ms ; Min: 8.715ms ; Max: 74.083ms ; Number of samples: 38) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 37.86 - BytesRead: 841.93 MB (882825140) - BytesReadDataNodeCache: 0 - BytesReadLocal: 841.93 MB (882825140) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 841.93 MB (882825140) - DecompressionTime: 4s302ms - MaxCompressedTextFileLength: 0 - NumColumns: 3 (3) - NumDisksAccessed: 8 (8) - NumRowGroups: 38 (38) - NumScannerThreadsStarted: 38 (38) - PeakMemoryUsage: 1.21 GB (1301717369) - PerReadThreadRawHdfsThroughput: 1.46 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 49.299ms - RowBatchQueuePutWaitTime: 1m38s - RowsRead: 263.66M (263661591) - RowsReturned: 7.45M (7452417) - RowsReturnedRate: 79.19 M/sec - ScanRangesComplete: 38 (38) - ScannerThreadsInvoluntaryContextSwitches: 8.26K (8255) - ScannerThreadsTotalWallClockTime: 2m12s - MaterializeTupleTime(*): 26s262ms - ScannerThreadsSysTime: 2s006ms - ScannerThreadsUserTime: 29s128ms - ScannerThreadsVoluntaryContextSwitches: 8.05K (8049) - TotalRawHdfsReadTime(*): 563.462ms - TotalReadThroughput: 240.55 MB/sec Instance 294c6905e4e7a318:11c53b1700000003 (host=vd1309.halxg.cloudera.com:22000):(Total: 3s606ms, non-child: 2s183ms, % non-child: 60.53%) Hdfs split stats (:<# splits>/): 0:5/1.24 GB 5:5/1.24 GB 2:4/1012.95 MB 4:11/2.72 GB 3:5/1.24 GB 1:7/1.73 GB MemoryUsage(500.000ms): 1.19 GB, 1.19 GB, 1.19 GB, 1.19 GB, 1.15 GB, 773.27 MB, 488.42 MB ThreadUsage(500.000ms): 38, 38, 38, 38, 38, 38, 28 - AverageThreadTokens: 36.57 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.20 GB (1284480011) - PerHostPeakMemUsage: 1.39 GB (1492956815) - RowsProduced: 7.31M (7309592) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 442.812ms - TotalStorageWaitTime: 3s622ms - TotalThreadsInvoluntaryContextSwitches: 8.07K (8072) - TotalThreadsTotalWallClockTime: 2m8s - TotalThreadsSysTime: 2s522ms - TotalThreadsUserTime: 30s964ms - TotalThreadsVoluntaryContextSwitches: 9.60K (9600) Fragment Instance Lifecycle Timings: - ExecTime: 3s540ms - ExecTreeExecTime: 91.596ms - OpenTime: 34.805ms - ExecTreeOpenTime: 30.762us - PrepareTime: 31.640ms - ExecTreePrepareTime: 342.982us DataStreamSender (dst_id=4):(Total: 1s280ms, non-child: 1s280ms, % non-child: 100.00%) - BytesSent: 123.80 MB (129814778) - NetworkThroughput(*): 335.63 MB/sec - OverallThroughput: 96.70 MB/sec - PeakMemoryUsage: 167.12 KB (171136) - RowsReturned: 7.31M (7309592) - SerializeBatchTime: 909.402ms - TransmitDataRPCTime: 368.859ms - UncompressedRowBatchSize: 327.69 MB (343609784) CodeGen:(Total: 56.923ms, non-child: 56.923ms, % non-child: 100.00%) - CodegenTime: 592.796us - CompileTime: 8.618ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 19 (19) - NumInstructions: 303 (303) - OptimizationTime: 24.848ms - PeakMemoryUsage: 151.50 KB (155136) - PrepareTime: 22.744ms HDFS_SCAN_NODE (id=0):(Total: 86.333ms, non-child: 86.333ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:5/1.24 GB 5:5/1.24 GB 2:4/1012.95 MB 4:11/2.72 GB 3:5/1.24 GB 1:7/1.73 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 37 out of 37 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/SNAPPY:111 BytesRead(500.000ms): 825.39 MB, 825.39 MB, 825.39 MB, 825.39 MB, 825.39 MB, 825.39 MB, 825.39 MB - FooterProcessingTime: (Avg: 32.029ms ; Min: 6.757ms ; Max: 83.627ms ; Number of samples: 37) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 35.57 - BytesRead: 825.39 MB (865484414) - BytesReadDataNodeCache: 0 - BytesReadLocal: 825.39 MB (865484414) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 825.39 MB (865484414) - DecompressionTime: 4s212ms - MaxCompressedTextFileLength: 0 - NumColumns: 3 (3) - NumDisksAccessed: 6 (6) - NumRowGroups: 37 (37) - NumScannerThreadsStarted: 37 (37) - PeakMemoryUsage: 1.19 GB (1282989156) - PerReadThreadRawHdfsThroughput: 1.45 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 43.979ms - RowBatchQueuePutWaitTime: 1m30s - RowsRead: 258.52M (258520412) - RowsReturned: 7.31M (7309592) - RowsReturnedRate: 84.67 M/sec - ScanRangesComplete: 37 (37) - ScannerThreadsInvoluntaryContextSwitches: 7.69K (7694) - ScannerThreadsTotalWallClockTime: 2m4s - MaterializeTupleTime(*): 25s609ms - ScannerThreadsSysTime: 1s941ms - ScannerThreadsUserTime: 28s450ms - ScannerThreadsVoluntaryContextSwitches: 7.98K (7978) - TotalRawHdfsReadTime(*): 557.404ms - TotalReadThroughput: 235.83 MB/sec Instance 294c6905e4e7a318:11c53b1700000005 (host=vd1314.halxg.cloudera.com:22000):(Total: 3s588ms, non-child: 2s081ms, % non-child: 58.00%) Hdfs split stats (:<# splits>/): 6:1/253.23 MB 7:5/1.18 GB 0:4/1012.95 MB 5:4/1012.97 MB 4:6/1.48 GB 3:4/1012.97 MB 2:6/1.48 GB 1:6/1.43 GB MemoryUsage(500.000ms): 1.14 GB, 1.15 GB, 1.15 GB, 1.15 GB, 1.15 GB, 938.40 MB, 580.59 MB ThreadUsage(500.000ms): 37, 37, 37, 37, 37, 37, 34 - AverageThreadTokens: 36.57 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.15 GB (1238679239) - PerHostPeakMemUsage: 1.54 GB (1654844496) - RowsProduced: 7.02M (7023583) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 446.956ms - TotalStorageWaitTime: 2s849ms - TotalThreadsInvoluntaryContextSwitches: 7.19K (7192) - TotalThreadsTotalWallClockTime: 2m4s - TotalThreadsSysTime: 2s433ms - TotalThreadsUserTime: 29s350ms - TotalThreadsVoluntaryContextSwitches: 9.20K (9196) Fragment Instance Lifecycle Timings: - ExecTime: 3s529ms - ExecTreeExecTime: 92.729ms - OpenTime: 32.993ms - ExecTreeOpenTime: 42.736us - PrepareTime: 26.404ms - ExecTreePrepareTime: 364.300us DataStreamSender (dst_id=4):(Total: 1s363ms, non-child: 1s363ms, % non-child: 100.00%) - BytesSent: 118.98 MB (124754705) - NetworkThroughput(*): 320.79 MB/sec - OverallThroughput: 87.23 MB/sec - PeakMemoryUsage: 167.12 KB (171136) - RowsReturned: 7.02M (7023583) - SerializeBatchTime: 990.855ms - TransmitDataRPCTime: 370.879ms - UncompressedRowBatchSize: 314.87 MB (330165053) CodeGen:(Total: 55.181ms, non-child: 55.181ms, % non-child: 100.00%) - CodegenTime: 706.164us - CompileTime: 8.287ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 19 (19) - NumInstructions: 303 (303) - OptimizationTime: 23.443ms - PeakMemoryUsage: 151.50 KB (155136) - PrepareTime: 22.809ms HDFS_SCAN_NODE (id=0):(Total: 88.307ms, non-child: 88.307ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:1/253.23 MB 7:5/1.18 GB 0:4/1012.95 MB 5:4/1012.97 MB 4:6/1.48 GB 3:4/1012.97 MB 2:6/1.48 GB 1:6/1.43 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 36 out of 36 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/SNAPPY:108 BytesRead(500.000ms): 793.13 MB, 793.13 MB, 793.13 MB, 793.13 MB, 793.13 MB, 793.13 MB, 793.13 MB - FooterProcessingTime: (Avg: 24.666ms ; Min: 6.505ms ; Max: 76.048ms ; Number of samples: 36) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 35.57 - BytesRead: 793.13 MB (831652042) - BytesReadDataNodeCache: 0 - BytesReadLocal: 793.13 MB (831652042) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 793.13 MB (831652042) - DecompressionTime: 3s833ms - MaxCompressedTextFileLength: 0 - NumColumns: 3 (3) - NumDisksAccessed: 8 (8) - NumRowGroups: 36 (36) - NumScannerThreadsStarted: 36 (36) - PeakMemoryUsage: 1.15 GB (1237071688) - PerReadThreadRawHdfsThroughput: 1.42 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 43.896ms - RowBatchQueuePutWaitTime: 1m29s - RowsRead: 248.37M (248372793) - RowsReturned: 7.02M (7023583) - RowsReturnedRate: 79.54 M/sec - ScanRangesComplete: 36 (36) - ScannerThreadsInvoluntaryContextSwitches: 6.77K (6772) - ScannerThreadsTotalWallClockTime: 2m1s - MaterializeTupleTime(*): 24s433ms - ScannerThreadsSysTime: 1s806ms - ScannerThreadsUserTime: 26s901ms - ScannerThreadsVoluntaryContextSwitches: 7.55K (7551) - TotalRawHdfsReadTime(*): 544.600ms - TotalReadThroughput: 226.61 MB/sec Instance 294c6905e4e7a318:11c53b1700000002 (host=vd1311.halxg.cloudera.com:22000):(Total: 3s564ms, non-child: 2s147ms, % non-child: 60.26%) Hdfs split stats (:<# splits>/): 7:3/759.72 MB 4:3/759.71 MB 0:2/506.48 MB 6:4/1012.94 MB 5:5/1.24 GB 3:5/1.24 GB 1:5/1.24 GB 2:10/2.47 GB MemoryUsage(500.000ms): 1.19 GB, 1.19 GB, 1.19 GB, 1.19 GB, 1.18 GB, 755.25 MB, 416.52 MB ThreadUsage(500.000ms): 38, 38, 38, 38, 38, 38, 25 - AverageThreadTokens: 36.14 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.20 GB (1285230647) - PerHostPeakMemUsage: 1.22 GB (1310427859) - RowsProduced: 7.31M (7312355) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 552.945ms - TotalStorageWaitTime: 3s201ms - TotalThreadsInvoluntaryContextSwitches: 8.25K (8247) - TotalThreadsTotalWallClockTime: 2m8s - TotalThreadsSysTime: 2s563ms - TotalThreadsUserTime: 30s565ms - TotalThreadsVoluntaryContextSwitches: 9.64K (9645) Fragment Instance Lifecycle Timings: - ExecTime: 3s505ms - ExecTreeExecTime: 92.553ms - OpenTime: 33.576ms - ExecTreeOpenTime: 31.032us - PrepareTime: 25.535ms - ExecTreePrepareTime: 335.148us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 3 (3) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 24.00 MB (25165824) - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 1s272ms, non-child: 1s272ms, % non-child: 100.00%) - BytesSent: 123.79 MB (129807296) - NetworkThroughput(*): 320.79 MB/sec - OverallThroughput: 97.25 MB/sec - PeakMemoryUsage: 167.12 KB (171136) - RowsReturned: 7.31M (7312355) - SerializeBatchTime: 885.001ms - TransmitDataRPCTime: 385.905ms - UncompressedRowBatchSize: 327.82 MB (343739665) CodeGen:(Total: 54.802ms, non-child: 54.802ms, % non-child: 100.00%) - CodegenTime: 619.360us - CompileTime: 8.253ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 19 (19) - NumInstructions: 303 (303) - OptimizationTime: 24.020ms - PeakMemoryUsage: 151.50 KB (155136) - PrepareTime: 21.826ms HDFS_SCAN_NODE (id=0):(Total: 88.675ms, non-child: 88.675ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:3/759.72 MB 4:3/759.71 MB 0:2/506.48 MB 6:4/1012.94 MB 5:5/1.24 GB 3:5/1.24 GB 1:5/1.24 GB 2:10/2.47 GB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 37 out of 37 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/SNAPPY:111 BytesRead(500.000ms): 825.38 MB, 825.38 MB, 825.38 MB, 825.38 MB, 825.38 MB, 825.38 MB, 825.38 MB - FooterProcessingTime: (Avg: 25.968ms ; Min: 7.652ms ; Max: 82.633ms ; Number of samples: 37) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 35.14 - BytesRead: 825.38 MB (865473125) - BytesReadDataNodeCache: 0 - BytesReadLocal: 825.38 MB (865473125) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 825.38 MB (865473125) - DecompressionTime: 4s200ms - MaxCompressedTextFileLength: 0 - NumColumns: 3 (3) - NumDisksAccessed: 8 (8) - NumRowGroups: 37 (37) - NumScannerThreadsStarted: 37 (37) - PeakMemoryUsage: 1.20 GB (1283565786) - PerReadThreadRawHdfsThroughput: 1.36 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 46.891ms - RowBatchQueuePutWaitTime: 1m31s - RowsRead: 258.52M (258520143) - RowsReturned: 7.31M (7312355) - RowsReturnedRate: 82.46 M/sec - ScanRangesComplete: 37 (37) - ScannerThreadsInvoluntaryContextSwitches: 7.89K (7889) - ScannerThreadsTotalWallClockTime: 2m4s - MaterializeTupleTime(*): 25s482ms - ScannerThreadsSysTime: 2s009ms - ScannerThreadsUserTime: 28s175ms - ScannerThreadsVoluntaryContextSwitches: 7.87K (7869) - TotalRawHdfsReadTime(*): 593.509ms - TotalReadThroughput: 235.82 MB/sec Instance 294c6905e4e7a318:11c53b1700000006 (host=vd1313.halxg.cloudera.com:22000):(Total: 3s530ms, non-child: 2s159ms, % non-child: 61.15%) Hdfs split stats (:<# splits>/): 1:1/253.25 MB 6:3/759.71 MB 7:4/1012.97 MB 2:3/759.71 MB 4:11/2.66 GB 5:5/1.24 GB 3:6/1.48 GB 0:4/1012.97 MB MemoryUsage(500.000ms): 1.03 GB, 1.18 GB, 1.19 GB, 1.18 GB, 1.17 GB, 894.27 MB, 549.11 MB ThreadUsage(500.000ms): 38, 38, 38, 38, 38, 37, 33 - AverageThreadTokens: 37.14 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.19 GB (1277756472) - PerHostPeakMemUsage: 1.33 GB (1430088899) - RowsProduced: 7.26M (7259246) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 534.308ms - TotalStorageWaitTime: 3s340ms - TotalThreadsInvoluntaryContextSwitches: 8.24K (8236) - TotalThreadsTotalWallClockTime: 2m5s - TotalThreadsSysTime: 2s541ms - TotalThreadsUserTime: 31s000ms - TotalThreadsVoluntaryContextSwitches: 9.64K (9637) Fragment Instance Lifecycle Timings: - ExecTime: 3s474ms - ExecTreeExecTime: 91.254ms - OpenTime: 32.820ms - ExecTreeOpenTime: 30.486us - PrepareTime: 23.841ms - ExecTreePrepareTime: 324.578us DataStreamSender (dst_id=4):(Total: 1s231ms, non-child: 1s231ms, % non-child: 100.00%) - BytesSent: 122.94 MB (128907462) - NetworkThroughput(*): 314.38 MB/sec - OverallThroughput: 99.84 MB/sec - PeakMemoryUsage: 167.12 KB (171136) - RowsReturned: 7.26M (7259246) - SerializeBatchTime: 835.380ms - TransmitDataRPCTime: 391.046ms - UncompressedRowBatchSize: 325.43 MB (341243118) CodeGen:(Total: 52.542ms, non-child: 52.542ms, % non-child: 100.00%) - CodegenTime: 841.940us - CompileTime: 8.173ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 19 (19) - NumInstructions: 303 (303) - OptimizationTime: 23.592ms - PeakMemoryUsage: 151.50 KB (155136) - PrepareTime: 20.229ms HDFS_SCAN_NODE (id=0):(Total: 87.941ms, non-child: 87.941ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:1/253.25 MB 6:3/759.71 MB 7:4/1012.97 MB 2:3/759.71 MB 4:11/2.66 GB 5:5/1.24 GB 3:6/1.48 GB 0:4/1012.97 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 37 out of 37 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/SNAPPY:111 BytesRead(500.000ms): 704.64 MB, 819.96 MB, 819.96 MB, 819.96 MB, 819.96 MB, 819.96 MB, 819.96 MB - FooterProcessingTime: (Avg: 28.255ms ; Min: 7.607ms ; Max: 80.511ms ; Number of samples: 37) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 36.14 - BytesRead: 819.96 MB (859785817) - BytesReadDataNodeCache: 0 - BytesReadLocal: 819.96 MB (859785817) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 819.96 MB (859785817) - DecompressionTime: 4s170ms - MaxCompressedTextFileLength: 0 - NumColumns: 3 (3) - NumDisksAccessed: 8 (8) - NumRowGroups: 37 (37) - NumScannerThreadsStarted: 37 (37) - PeakMemoryUsage: 1.19 GB (1276066977) - PerReadThreadRawHdfsThroughput: 1.45 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 45.284ms - RowBatchQueuePutWaitTime: 1m27s - RowsRead: 256.77M (256768045) - RowsReturned: 7.26M (7259246) - RowsReturnedRate: 82.55 M/sec - ScanRangesComplete: 37 (37) - ScannerThreadsInvoluntaryContextSwitches: 7.88K (7881) - ScannerThreadsTotalWallClockTime: 2m2s - MaterializeTupleTime(*): 25s906ms - ScannerThreadsSysTime: 1s931ms - ScannerThreadsUserTime: 28s672ms - ScannerThreadsVoluntaryContextSwitches: 7.84K (7836) - TotalRawHdfsReadTime(*): 552.545ms - TotalReadThroughput: 234.27 MB/sec Instance 294c6905e4e7a318:11c53b1700000004 (host=vd1312.halxg.cloudera.com:22000):(Total: 3s500ms, non-child: 2s137ms, % non-child: 61.06%) Hdfs split stats (:<# splits>/): 5:3/759.73 MB 3:4/1012.95 MB 6:6/1.48 GB 2:5/1.24 GB 7:4/962.49 MB 4:8/1.98 GB 0:3/759.68 MB 1:4/1012.94 MB MemoryUsage(500.000ms): 1.18 GB, 1.19 GB, 1.18 GB, 1.18 GB, 1.11 GB, 624.20 MB, 42.51 MB ThreadUsage(500.000ms): 38, 38, 38, 38, 38, 37, 1 - AverageThreadTokens: 32.57 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.19 GB (1279036495) - PerHostPeakMemUsage: 1.44 GB (1550180241) - RowsProduced: 7.26M (7257029) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 498.205ms - TotalStorageWaitTime: 2s912ms - TotalThreadsInvoluntaryContextSwitches: 7.83K (7826) - TotalThreadsTotalWallClockTime: 2m5s - TotalThreadsSysTime: 2s489ms - TotalThreadsUserTime: 30s614ms - TotalThreadsVoluntaryContextSwitches: 9.83K (9831) Fragment Instance Lifecycle Timings: - ExecTime: 3s439ms - ExecTreeExecTime: 92.882ms - OpenTime: 34.243ms - ExecTreeOpenTime: 30.796us - PrepareTime: 26.153ms - ExecTreePrepareTime: 329.682us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 48 (48) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 384.00 MB (402653184) - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=4):(Total: 1s217ms, non-child: 1s217ms, % non-child: 100.00%) - BytesSent: 122.90 MB (128868689) - NetworkThroughput(*): 317.69 MB/sec - OverallThroughput: 100.91 MB/sec - PeakMemoryUsage: 167.12 KB (171136) - RowsReturned: 7.26M (7257029) - SerializeBatchTime: 829.289ms - TransmitDataRPCTime: 386.851ms - UncompressedRowBatchSize: 325.34 MB (341138899) CodeGen:(Total: 55.611ms, non-child: 55.611ms, % non-child: 100.00%) - CodegenTime: 617.434us - CompileTime: 8.172ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 19 (19) - NumInstructions: 303 (303) - OptimizationTime: 24.811ms - PeakMemoryUsage: 151.50 KB (155136) - PrepareTime: 21.932ms HDFS_SCAN_NODE (id=0):(Total: 89.268ms, non-child: 89.268ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:3/759.73 MB 3:4/1012.95 MB 6:6/1.48 GB 2:5/1.24 GB 7:4/962.49 MB 4:8/1.98 GB 0:3/759.68 MB 1:4/1012.94 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 37 out of 37 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/SNAPPY:111 BytesRead(500.000ms): 821.04 MB, 821.04 MB, 821.04 MB, 821.04 MB, 821.04 MB, 821.04 MB, 821.04 MB - FooterProcessingTime: (Avg: 24.431ms ; Min: 7.521ms ; Max: 74.033ms ; Number of samples: 37) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 31.57 - BytesRead: 821.04 MB (860924617) - BytesReadDataNodeCache: 0 - BytesReadLocal: 821.04 MB (860924617) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 821.04 MB (860924617) - DecompressionTime: 4s169ms - MaxCompressedTextFileLength: 0 - NumColumns: 3 (3) - NumDisksAccessed: 8 (8) - NumRowGroups: 37 (37) - NumScannerThreadsStarted: 37 (37) - PeakMemoryUsage: 1.19 GB (1277270216) - PerReadThreadRawHdfsThroughput: 1.45 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 49.350ms - RowBatchQueuePutWaitTime: 1m28s - RowsRead: 257.13M (257125846) - RowsReturned: 7.26M (7257029) - RowsReturnedRate: 81.29 M/sec - ScanRangesComplete: 37 (37) - ScannerThreadsInvoluntaryContextSwitches: 7.48K (7476) - ScannerThreadsTotalWallClockTime: 2m1s - MaterializeTupleTime(*): 25s455ms - ScannerThreadsSysTime: 1s883ms - ScannerThreadsUserTime: 28s283ms - ScannerThreadsVoluntaryContextSwitches: 8.14K (8137) - TotalRawHdfsReadTime(*): 554.436ms - TotalReadThroughput: 234.58 MB/sec Instance 294c6905e4e7a318:11c53b1700000007 (host=vd1310.halxg.cloudera.com:22000):(Total: 3s357ms, non-child: 1s953ms, % non-child: 58.18%) Hdfs split stats (:<# splits>/): 2:4/1012.98 MB 5:5/1.24 GB 1:8/1.98 GB 6:4/1013.00 MB 3:3/759.74 MB 7:4/1012.96 MB 4:5/1.24 GB 0:4/958.69 MB MemoryUsage(500.000ms): 1.18 GB, 1.18 GB, 1.18 GB, 1.18 GB, 1.06 GB, 614.23 MB ThreadUsage(500.000ms): 38, 38, 38, 38, 37, 37 - AverageThreadTokens: 37.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.19 GB (1276772498) - PerHostPeakMemUsage: 1.55 GB (1664404467) - RowsProduced: 7.27M (7270571) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 367.667ms - TotalStorageWaitTime: 2s927ms - TotalThreadsInvoluntaryContextSwitches: 7.19K (7186) - TotalThreadsTotalWallClockTime: 1m59s - TotalThreadsSysTime: 2s416ms - TotalThreadsUserTime: 30s510ms - TotalThreadsVoluntaryContextSwitches: 9.46K (9463) Fragment Instance Lifecycle Timings: - ExecTime: 3s298ms - ExecTreeExecTime: 93.693ms - OpenTime: 33.389ms - ExecTreeOpenTime: 47.226us - PrepareTime: 25.233ms - ExecTreePrepareTime: 395.976us DataStreamSender (dst_id=4):(Total: 1s260ms, non-child: 1s260ms, % non-child: 100.00%) - BytesSent: 123.14 MB (129122680) - NetworkThroughput(*): 342.76 MB/sec - OverallThroughput: 97.73 MB/sec - PeakMemoryUsage: 167.12 KB (171136) - RowsReturned: 7.27M (7270571) - SerializeBatchTime: 898.840ms - TransmitDataRPCTime: 359.263ms - UncompressedRowBatchSize: 325.94 MB (341775481) CodeGen:(Total: 54.356ms, non-child: 54.356ms, % non-child: 100.00%) - CodegenTime: 740.272us - CompileTime: 8.128ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1993976) - NumFunctions: 19 (19) - NumInstructions: 303 (303) - OptimizationTime: 24.071ms - PeakMemoryUsage: 151.50 KB (155136) - PrepareTime: 21.521ms HDFS_SCAN_NODE (id=0):(Total: 89.758ms, non-child: 89.758ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:4/1012.98 MB 5:5/1.24 GB 1:8/1.98 GB 6:4/1013.00 MB 3:3/759.74 MB 7:4/1012.96 MB 4:5/1.24 GB 0:4/958.69 MB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 37 out of 37 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/SNAPPY:111 BytesRead(500.000ms): 820.64 MB, 820.64 MB, 820.64 MB, 820.64 MB, 820.64 MB, 820.64 MB - FooterProcessingTime: (Avg: 21.470ms ; Min: 7.748ms ; Max: 52.652ms ; Number of samples: 37) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 36.67 - BytesRead: 820.64 MB (860504874) - BytesReadDataNodeCache: 0 - BytesReadLocal: 820.64 MB (860504874) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 820.64 MB (860504874) - DecompressionTime: 4s042ms - MaxCompressedTextFileLength: 0 - NumColumns: 3 (3) - NumDisksAccessed: 8 (8) - NumRowGroups: 37 (37) - NumScannerThreadsStarted: 37 (37) - PeakMemoryUsage: 1.19 GB (1275025691) - PerReadThreadRawHdfsThroughput: 1.32 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 44.649ms - RowBatchQueuePutWaitTime: 1m22s - RowsRead: 257.02M (257020261) - RowsReturned: 7.27M (7270571) - RowsReturnedRate: 81.00 M/sec - ScanRangesComplete: 37 (37) - ScannerThreadsInvoluntaryContextSwitches: 6.83K (6828) - ScannerThreadsTotalWallClockTime: 1m55s - MaterializeTupleTime(*): 25s479ms - ScannerThreadsSysTime: 1s859ms - ScannerThreadsUserTime: 28s145ms - ScannerThreadsVoluntaryContextSwitches: 8.06K (8059) - TotalRawHdfsReadTime(*): 604.969ms - TotalReadThroughput: 273.55 MB/sec