impalad /backends /catalog /hadoop-varz /logs /memz /metrics /queries /rpcz /sessions /threadz /varz Query c34cc30188b6a8c9:93c2cb1e00000000 Plan Query Text plan Summary Profile Memory Download Thrift Profile Query (id=c34cc30188b6a8c9:93c2cb1e00000000): Summary: Session ID: 794a311f06f46c42:6d475c5e16d1bf88 Session Type: BEESWAX Start Time: 2017-02-27 23:51:15.745707000 End Time: 2017-02-27 23:52:13.351827000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.7.0-cdh5.10.0 RELEASE (build 785a073cd07e2540d521ecebb8b38161ccbd2aa2) User: yo@foo.fa.COM Connected User: yo@foo.fa.COM Delegated User: Network Address: 10.17.187.36:38086 Default Db: tpch_30000_parquet Sql Statement: select count(*) from orders_p Coordinator: vhs026.foo.fa.com:22000 Query Options (non default): Plan: ---------------- Estimated Per-Host Requirements: Memory=10.00MB VCores=1 WARNING: The following tables are missing relevant table and/or column statistics. tpch_30000_parquet.orders_p PLAN-ROOT SINK | 03:AGGREGATE [FINALIZE] | output: count:merge(*) | hosts=134 per-host-mem=unavailable | tuple-ids=1 row-size=8B cardinality=1 | 02:EXCHANGE [UNPARTITIONED] | hosts=134 per-host-mem=unavailable | tuple-ids=1 row-size=8B cardinality=1 | 01:AGGREGATE | output: count(*) | hosts=134 per-host-mem=10.00MB | tuple-ids=1 row-size=8B cardinality=1 | 00:SCAN HDFS [tpch_30000_parquet.orders_p, RANDOM] partitions=249998/249998 files=249998 size=332.84MB table stats: unavailable column stats: all hosts=134 per-host-mem=0B tuple-ids=0 row-size=0B cardinality=unavailable ---------------- Estimated Per-Host Mem: 10485760 Estimated Per-Host VCores: 1 Tables Missing Stats: tpch_30000_parquet.orders_p Request Pool: root.yo Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ------------------------------------------------------------------------------------------------------------------------ 03:AGGREGATE 1 352.123us 352.123us 1 1 552.00 KB -1.00 B FINALIZE 02:EXCHANGE 1 2.481ms 2.481ms 134 1 0 -1.00 B UNPARTITIONED 01:AGGREGATE 134 4.518ms 7.256ms 134 1 16.00 KB 10.00 MB 00:SCAN HDFS 134 1s556ms 2s265ms 250.00K -1 264.00 KB 0 tpch_30000_parquet.orders_p Planner Timeline: 537.706ms - Analysis finished: 2.045ms (2.045ms) - Equivalence classes computed: 2.229ms (183.536us) - Single node plan created: 318.980ms (316.751ms) - Runtime filters computed: 319.022ms (41.895us) - Distributed plan created: 319.258ms (235.947us) - Lineage info computed: 319.306ms (48.056us) - Planning finished: 537.706ms (218.400ms) Query Timeline: 57s686ms - Query submitted: 305.864us (305.864us) - Planning finished: 3s874ms (3s874ms) - Submit for admission: 4s781ms (906.874ms) - Completed admission: 4s781ms (300.926us) - Ready to start 135 fragment instances: 5s168ms (387.022ms) - All 135 fragment instances started: 56s767ms (51s598ms) - Rows available: 57s534ms (766.534ms) - First row fetched: 57s576ms (41.999ms) - Unregister query: 57s605ms (29.687ms) - ComputeScanRangeAssignmentTimer: 329.865ms ImpalaServer: - ClientFetchWaitTimer: 52.846ms - RowMaterializationTimer: 18.830ms Execution Profile c34cc30188b6a8c9:93c2cb1e00000000:(Total: 51s980ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed Enabled ---------------------------------------------------------------------------------------------------------------------------- Fragment instance start latencies: Count: 135, 25th %-ile: 1s503ms, 50th %-ile: 1s593ms, 75th %-ile: 1s735ms, 90th %-ile: 1s820ms, 95th %-ile: 1s854ms, 99.9th %-ile: 1s944ms Per Node Peak Memory Usage: ve1142.foo.fa.com:22000(226.36 KB) vd1342.foo.fa.com:22000(250.36 KB) vd1118.foo.fa.com:22000(232.36 KB) vc1513.foo.fa.com:22000(248.36 KB) vhs036.foo.fa.com:22000(270.36 KB) vhs031.foo.fa.com:22000(248.36 KB) vc1523.foo.fa.com:22000(248.36 KB) vc1516.foo.fa.com:22000(248.36 KB) vd1136.foo.fa.com:22000(280.36 KB) vc1530.foo.fa.com:22000(248.36 KB) vc1537.foo.fa.com:22000(226.36 KB) ve1134.foo.fa.com:22000(226.36 KB) ve1129.foo.fa.com:22000(248.36 KB) vd1134.foo.fa.com:22000(248.36 KB) vd1127.foo.fa.com:22000(248.36 KB) vd1341.foo.fa.com:22000(248.36 KB) ve1139.foo.fa.com:22000(218.36 KB) vd1102.foo.fa.com:22000(248.36 KB) vhs042.foo.fa.com:22000(240.36 KB) vc1535.foo.fa.com:22000(248.36 KB) vc1510.foo.fa.com:22000(248.36 KB) vd1338.foo.fa.com:22000(248.36 KB) vd1336.foo.fa.com:22000(248.36 KB) vd1142.foo.fa.com:22000(226.36 KB) vc1533.foo.fa.com:22000(248.36 KB) vd1139.foo.fa.com:22000(250.36 KB) ve1125.foo.fa.com:22000(248.36 KB) vc1509.foo.fa.com:22000(248.36 KB) vd1329.foo.fa.com:22000(240.36 KB) vc1532.foo.fa.com:22000(248.36 KB) vc1525.foo.fa.com:22000(248.36 KB) vc1527.foo.fa.com:22000(248.36 KB) ve1140.foo.fa.com:22000(240.36 KB) ve1131.foo.fa.com:22000(258.36 KB) vhs045.foo.fa.com:22000(272.36 KB) ve1127.foo.fa.com:22000(248.36 KB) ve1126.foo.fa.com:22000(248.36 KB) ve1130.foo.fa.com:22000(248.36 KB) vd1135.foo.fa.com:22000(248.36 KB) vd1337.foo.fa.com:22000(248.36 KB) vd1140.foo.fa.com:22000(248.36 KB) vhs029.foo.fa.com:22000(248.36 KB) vc1522.foo.fa.com:22000(248.36 KB) ve1141.foo.fa.com:22000(248.36 KB) vc1518.foo.fa.com:22000(240.36 KB) vc1508.foo.fa.com:22000(250.36 KB) vd1126.foo.fa.com:22000(248.36 KB) vd1110.foo.fa.com:22000(262.36 KB) vhs030.foo.fa.com:22000(248.36 KB) vd1120.foo.fa.com:22000(248.36 KB) vc1501.foo.fa.com:22000(248.36 KB) vd1133.foo.fa.com:22000(280.36 KB) vd1334.foo.fa.com:22000(258.36 KB) vd1111.foo.fa.com:22000(248.36 KB) vd1101.foo.fa.com:22000(230.36 KB) vd1326.foo.fa.com:22000(248.36 KB) vc1536.foo.fa.com:22000(248.36 KB) vhs046.foo.fa.com:22000(248.36 KB) vc1521.foo.fa.com:22000(248.36 KB) vc1529.foo.fa.com:22000(248.36 KB) vc1505.foo.fa.com:22000(240.36 KB) vd1112.foo.fa.com:22000(260.36 KB) vc1503.foo.fa.com:22000(248.36 KB) vc1539.foo.fa.com:22000(248.36 KB) vd1123.foo.fa.com:22000(240.36 KB) vc1519.foo.fa.com:22000(248.36 KB) vd1333.foo.fa.com:22000(248.36 KB) vc1514.foo.fa.com:22000(248.36 KB) vc1531.foo.fa.com:22000(240.36 KB) vd1129.foo.fa.com:22000(248.36 KB) vc1541.foo.fa.com:22000(280.36 KB) vhs044.foo.fa.com:22000(248.36 KB) vhs043.foo.fa.com:22000(250.36 KB) vc1502.foo.fa.com:22000(240.36 KB) vd1107.foo.fa.com:22000(248.36 KB) vd1117.foo.fa.com:22000(248.36 KB) vd1332.foo.fa.com:22000(248.36 KB) vc1524.foo.fa.com:22000(268.36 KB) vhs028.foo.fa.com:22000(248.36 KB) vd1116.foo.fa.com:22000(248.36 KB) vd1103.foo.fa.com:22000(268.36 KB) vhs039.foo.fa.com:22000(248.36 KB) vd1119.foo.fa.com:22000(248.36 KB) vhs027.foo.fa.com:22000(248.36 KB) vd1114.foo.fa.com:22000(240.36 KB) vhs038.foo.fa.com:22000(236.36 KB) vc1520.foo.fa.com:22000(240.36 KB) vd1104.foo.fa.com:22000(232.36 KB) vhs035.foo.fa.com:22000(248.36 KB) vhs032.foo.fa.com:22000(248.36 KB) vd1132.foo.fa.com:22000(248.36 KB) ve1132.foo.fa.com:22000(248.36 KB) vd1109.foo.fa.com:22000(232.36 KB) vd1335.foo.fa.com:22000(248.36 KB) vd1124.foo.fa.com:22000(248.36 KB) vc1507.foo.fa.com:22000(304.36 KB) vc1526.foo.fa.com:22000(248.36 KB) vd1113.foo.fa.com:22000(266.36 KB) ve1138.foo.fa.com:22000(248.36 KB) vd1108.foo.fa.com:22000(240.36 KB) vhs041.foo.fa.com:22000(248.36 KB) vd1125.foo.fa.com:22000(258.36 KB) vd1115.foo.fa.com:22000(248.36 KB) vd1105.foo.fa.com:22000(250.36 KB) vc1538.foo.fa.com:22000(248.36 KB) vd1122.foo.fa.com:22000(240.36 KB) vd1339.foo.fa.com:22000(248.36 KB) vc1517.foo.fa.com:22000(248.36 KB) vd1130.foo.fa.com:22000(258.36 KB) vc1528.foo.fa.com:22000(248.36 KB) ve1136.foo.fa.com:22000(240.36 KB) ve1128.foo.fa.com:22000(248.36 KB) vc1542.foo.fa.com:22000(248.36 KB) vd1121.foo.fa.com:22000(242.36 KB) ve1135.foo.fa.com:22000(248.36 KB) vd1138.foo.fa.com:22000(248.36 KB) vd1327.foo.fa.com:22000(248.36 KB) ve1137.foo.fa.com:22000(248.36 KB) vd1141.foo.fa.com:22000(248.36 KB) vd1331.foo.fa.com:22000(264.36 KB) vhs037.foo.fa.com:22000(248.36 KB) vd1330.foo.fa.com:22000(240.36 KB) vd1328.foo.fa.com:22000(258.36 KB) vc1515.foo.fa.com:22000(248.36 KB) vc1506.foo.fa.com:22000(248.36 KB) vhs034.foo.fa.com:22000(248.36 KB) vd1340.foo.fa.com:22000(236.36 KB) vhs033.foo.fa.com:22000(248.36 KB) vhs026.foo.fa.com:22000(584.37 KB) vc1534.foo.fa.com:22000(248.36 KB) vc1540.foo.fa.com:22000(248.36 KB) vd1131.foo.fa.com:22000(240.36 KB) vc1504.foo.fa.com:22000(288.36 KB) vc1511.foo.fa.com:22000(230.36 KB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Coordinator Fragment F01: Instance c34cc30188b6a8c9:93c2cb1e00000000 (host=vhs026.foo.fa.com:22000):(Total: 52s406ms, non-child: 42.961ms, % non-child: 0.08%) MemoryUsage(1s000ms): 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 28.01 KB, 40.01 KB, 62.01 KB, 72.01 KB, 88.01 KB, 94.01 KB, 112.01 KB, 120.01 KB, 138.01 KB, 148.01 KB, 166.01 KB, 170.01 KB, 190.01 KB, 210.01 KB, 216.01 KB, 226.01 KB, 242.01 KB, 264.01 KB, 264.01 KB, 282.01 KB, 300.01 KB, 312.01 KB, 312.01 KB, 322.01 KB, 348.01 KB, 360.01 KB, 360.01 KB, 370.01 KB, 400.01 KB, 408.01 KB, 408.01 KB, 416.01 KB, 446.01 KB, 456.01 KB, 456.01 KB, 470.01 KB, 498.01 KB, 504.01 KB, 504.01 KB, 518.01 KB, 544.01 KB, 552.01 KB, 552.01 KB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 560.02 KB (573456) - PerHostPeakMemUsage: 584.37 KB (598398) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 52s342ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 52s386ms - TotalThreadsSysTime: 1.999ms - TotalThreadsUserTime: 4.999ms - TotalThreadsVoluntaryContextSwitches: 138 (138) Fragment Instance Lifecycle Timings: - ExecTime: 42.102ms - ExecTreeExecTime: 4.799us - OpenTime: 52s344ms - ExecTreeOpenTime: 52s343ms - PrepareTime: 19.794ms - ExecTreePrepareTime: 182.454us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 19.098ms, non-child: 19.098ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 18.564ms AGGREGATION_NODE (id=3):(Total: 52s344ms, non-child: 352.123us, % non-child: 0.00%) - BuildTime: 89.141us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 552.00 KB (565248) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=2):(Total: 52s343ms, non-child: 2.481ms, % non-child: 0.00%) BytesReceived(1s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 16.00 B, 64.00 B, 152.00 B, 192.00 B, 256.00 B, 280.00 B, 352.00 B, 384.00 B, 456.00 B, 496.00 B, 568.00 B, 584.00 B, 664.00 B, 744.00 B, 768.00 B, 808.00 B, 872.00 B, 960.00 B, 960.00 B, 1.01 KB, 1.08 KB, 1.12 KB, 1.12 KB, 1.16 KB, 1.27 KB, 1.31 KB, 1.31 KB, 1.35 KB, 1.47 KB, 1.50 KB, 1.50 KB, 1.53 KB, 1.65 KB, 1.69 KB, 1.69 KB, 1.74 KB, 1.85 KB, 1.88 KB, 1.88 KB, 1.93 KB, 2.03 KB, 2.06 KB, 2.06 KB - BytesReceived: 2.09 KB (2144) - ConvertRowBatchTime: 348.745us - DeserializeRowBatchTimer: 2.516ms - FirstBatchArrivalWaitTime: 9s265ms - PeakMemoryUsage: 0 - RowsReturned: 134 (134) - RowsReturnedRate: 2.00 /sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 1s968ms, non-child: 381.990ms, % non-child: 19.41%) split sizes: min: 1.81 MB, max: 2.59 MB, avg: 2.48 MB, stddev: 270.91 KB completion times: min:1s896ms max:43s920ms mean: 23s977ms stddev:12s255ms execution rates: min:42.28 KB/sec max:1.36 MB/sec mean:175.54 KB/sec stddev:191.89 KB/sec num instances: 134 - AverageThreadTokens: 68.21 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.70 KB (254669) - PerHostPeakMemUsage: 249.06 KB (255036) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 428.670us - TotalStorageWaitTime: 23s528ms - TotalThreadsInvoluntaryContextSwitches: 22 (22) - TotalThreadsTotalWallClockTime: 2m13s - TotalThreadsSysTime: 74.884ms - TotalThreadsUserTime: 404.873ms - TotalThreadsVoluntaryContextSwitches: 9.06K (9060) Fragment Instance Lifecycle Timings: - ExecTime: 634.894us - ExecTreeExecTime: 5.757us - OpenTime: 1s602ms - ExecTreeOpenTime: 1s554ms - PrepareTime: 364.904ms - ExecTreePrepareTime: 6.652ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 85.785us, non-child: 85.785us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 262.77 KB/sec - OverallThroughput: 186.39 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.532us - TransmitDataRPCTime: 61.239us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.752ms, non-child: 24.752ms, % non-child: 100.00%) - CodegenTime: 594.292us - CompileTime: 4.107ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.723ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.533ms AGGREGATION_NODE (id=1):(Total: 1s561ms, non-child: 4.518ms, % non-child: 0.29%) - BuildTime: 1.501ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s556ms, non-child: 1s556ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.02 - AverageScannerThreadConcurrency: 84.38 - BytesRead: 2.48 MB (2604512) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.48 MB (2604512) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.48 MB (2604512) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 7 (7) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 85 (85) - PeakMemoryUsage: 208.40 KB (213404) - PerReadThreadRawHdfsThroughput: 104.62 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s512ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.86K (1865) - RowsReturned: 1.86K (1865) - RowsReturnedRate: 1.22 K/sec - ScanRangesComplete: 1.86K (1865) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 2m11s - MaterializeTupleTime(*): 2.234ms - ScannerThreadsSysTime: 48.022ms - ScannerThreadsUserTime: 305.828ms - ScannerThreadsVoluntaryContextSwitches: 7.05K (7048) - TotalRawHdfsReadTime(*): 23.757ms - TotalReadThroughput: 1.30 MB/sec Fragment F00: Instance c34cc30188b6a8c9:63 (host=vc1508.foo.fa.com:22000):(Total: 2s651ms, non-child: 356.113ms, % non-child: 13.43%) Hdfs split stats (:<# splits>/): 7:251/342.25 KB 0:233/318.03 KB 3:259/352.93 KB 4:249/339.56 KB 1:226/307.88 KB 2:247/337.46 KB 5:249/339.16 KB 6:229/312.25 KB MemoryUsage(500.000ms): 0, 194.36 KB, 184.36 KB, 184.36 KB, 184.36 KB, 130.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89, 89 - AverageThreadTokens: 74.33 - BloomFilterBytes: 0 - PeakMemoryUsage: 250.36 KB (256366) - PerHostPeakMemUsage: 250.36 KB (256366) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 435.925us - TotalStorageWaitTime: 34s627ms - TotalThreadsInvoluntaryContextSwitches: 28 (28) - TotalThreadsTotalWallClockTime: 3m20s - TotalThreadsSysTime: 78.971ms - TotalThreadsUserTime: 449.902ms - TotalThreadsVoluntaryContextSwitches: 9.21K (9210) Fragment Instance Lifecycle Timings: - ExecTime: 648.469us - ExecTreeExecTime: 5.442us - OpenTime: 2s310ms - ExecTreeOpenTime: 2s262ms - PrepareTime: 340.041ms - ExecTreePrepareTime: 7.079ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 94.394us, non-child: 94.394us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 270.69 KB/sec - OverallThroughput: 165.53 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.141us - TransmitDataRPCTime: 57.723us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.815ms, non-child: 25.815ms, % non-child: 100.00%) - CodegenTime: 659.071us - CompileTime: 4.238ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.984ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.203ms AGGREGATION_NODE (id=1):(Total: 2s269ms, non-child: 3.681ms, % non-child: 0.16%) ExecOption: Codegen Enabled - BuildTime: 1.343ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 2s265ms, non-child: 2s265ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:251/342.25 KB 0:233/318.03 KB 3:259/352.93 KB 4:249/339.56 KB 1:226/307.88 KB 2:247/337.46 KB 5:249/339.16 KB 6:229/312.25 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 318.69 KB, 1019.65 KB, 1.35 MB, 2.00 MB, 2.47 MB - FooterProcessingTime: (Avg: 17.892ms ; Min: 5.403ms ; Max: 413.689ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713105) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713105) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713105) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 210.00 KB (215040) - PerReadThreadRawHdfsThroughput: 109.39 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 2s220ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 857.00 /sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 3m17s - MaterializeTupleTime(*): 2.296ms - ScannerThreadsSysTime: 43.976ms - ScannerThreadsUserTime: 348.917ms - ScannerThreadsVoluntaryContextSwitches: 7.09K (7088) - TotalRawHdfsReadTime(*): 23.653ms - TotalReadThroughput: 1013.44 KB/sec Instance c34cc30188b6a8c9:62 (host=vc1527.foo.fa.com:22000):(Total: 2s561ms, non-child: 350.387ms, % non-child: 13.68%) Hdfs split stats (:<# splits>/): 5:273/372.13 KB 1:236/321.67 KB 3:244/332.61 KB 4:230/313.17 KB 6:242/329.72 KB 7:238/324.13 KB 0:243/331.06 KB 2:240/326.88 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB, 192.36 KB, 58.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89, 89 - AverageThreadTokens: 74.33 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 399.287us - TotalStorageWaitTime: 32s735ms - TotalThreadsInvoluntaryContextSwitches: 18 (18) - TotalThreadsTotalWallClockTime: 3m12s - TotalThreadsSysTime: 76.963ms - TotalThreadsUserTime: 425.900ms - TotalThreadsVoluntaryContextSwitches: 9.40K (9403) Fragment Instance Lifecycle Timings: - ExecTime: 597.006us - ExecTreeExecTime: 5.229us - OpenTime: 2s226ms - ExecTreeOpenTime: 2s178ms - PrepareTime: 334.198ms - ExecTreePrepareTime: 6.798ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 89.511us, non-child: 89.511us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 224.56 KB/sec - OverallThroughput: 174.56 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.213us - TransmitDataRPCTime: 69.580us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.839ms, non-child: 24.839ms, % non-child: 100.00%) - CodegenTime: 630.398us - CompileTime: 4.119ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.763ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.555ms AGGREGATION_NODE (id=1):(Total: 2s185ms, non-child: 4.575ms, % non-child: 0.21%) ExecOption: Codegen Enabled - BuildTime: 1.560ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 2s181ms, non-child: 2s181ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:273/372.13 KB 1:236/321.67 KB 3:244/332.61 KB 4:230/313.17 KB 6:242/329.72 KB 7:238/324.13 KB 0:243/331.06 KB 2:240/326.88 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1946 out of 1946 Hdfs Read Thread Concurrency Bucket: 0:80% 1:20% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1946 BytesRead(500.000ms): 242.59 KB, 775.27 KB, 1.28 MB, 1.98 MB, 2.57 MB - FooterProcessingTime: (Avg: 16.888ms ; Min: 5.260ms ; Max: 486.678ms ; Number of samples: 1946) - AverageHdfsReadThreadConcurrency: 0.20 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2715008) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2715008) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2715008) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 111.00 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 2s135ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1946) - RowsReturned: 1.95K (1946) - RowsReturnedRate: 892.00 /sec - ScanRangesComplete: 1.95K (1946) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 3m10s - MaterializeTupleTime(*): 2.269ms - ScannerThreadsSysTime: 55.967ms - ScannerThreadsUserTime: 317.916ms - ScannerThreadsVoluntaryContextSwitches: 7.28K (7282) - TotalRawHdfsReadTime(*): 23.326ms - TotalReadThroughput: 1.03 MB/sec Instance c34cc30188b6a8c9:51 (host=vc1509.foo.fa.com:22000):(Total: 2s537ms, non-child: 406.820ms, % non-child: 16.03%) Hdfs split stats (:<# splits>/): 6:230/313.33 KB 2:249/339.51 KB 4:238/324.47 KB 0:260/354.36 KB 3:232/316.28 KB 1:251/341.80 KB 7:258/351.64 KB 5:226/307.56 KB MemoryUsage(500.000ms): 0, 184.36 KB, 194.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 385.575us - TotalStorageWaitTime: 32s575ms - TotalThreadsInvoluntaryContextSwitches: 40 (40) - TotalThreadsTotalWallClockTime: 3m5s - TotalThreadsSysTime: 67.968ms - TotalThreadsUserTime: 426.901ms - TotalThreadsVoluntaryContextSwitches: 9.52K (9516) Fragment Instance Lifecycle Timings: - ExecTime: 596.778us - ExecTreeExecTime: 4.584us - OpenTime: 2s145ms - ExecTreeOpenTime: 2s099ms - PrepareTime: 390.944ms - ExecTreePrepareTime: 6.958ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 63.687us, non-child: 63.687us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 329.53 KB/sec - OverallThroughput: 245.34 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.319us - TransmitDataRPCTime: 47.416us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.909ms, non-child: 23.909ms, % non-child: 100.00%) - CodegenTime: 571.092us - CompileTime: 4.060ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.550ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 12.950ms AGGREGATION_NODE (id=1):(Total: 2s106ms, non-child: 4.031ms, % non-child: 0.19%) ExecOption: Codegen Enabled - BuildTime: 1.479ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 2s102ms, non-child: 2s102ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:230/313.33 KB 2:249/339.51 KB 4:238/324.47 KB 0:260/354.36 KB 3:232/316.28 KB 1:251/341.80 KB 7:258/351.64 KB 5:226/307.56 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 262.65 KB, 809.36 KB, 1.03 MB, 1.85 MB - FooterProcessingTime: (Avg: 16.822ms ; Min: 5.438ms ; Max: 416.043ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2712521) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712521) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712521) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 108.39 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 2s057ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 924.00 /sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 7 (7) - ScannerThreadsTotalWallClockTime: 3m3s - MaterializeTupleTime(*): 2.276ms - ScannerThreadsSysTime: 45.971ms - ScannerThreadsUserTime: 318.917ms - ScannerThreadsVoluntaryContextSwitches: 7.43K (7433) - TotalRawHdfsReadTime(*): 23.865ms - TotalReadThroughput: 944.95 KB/sec Instance c34cc30188b6a8c9:4b (host=vd1115.foo.fa.com:22000):(Total: 2s521ms, non-child: 411.212ms, % non-child: 16.31%) Hdfs split stats (:<# splits>/): 2:239/326.30 KB 0:236/322.22 KB 3:266/362.76 KB 6:249/339.21 KB 7:246/335.62 KB 1:241/328.60 KB 4:229/312.04 KB 5:237/323.71 KB MemoryUsage(500.000ms): 0, 200.36 KB, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 419.902us - TotalStorageWaitTime: 31s671ms - TotalThreadsInvoluntaryContextSwitches: 15 (15) - TotalThreadsTotalWallClockTime: 3m3s - TotalThreadsSysTime: 54.973ms - TotalThreadsUserTime: 433.896ms - TotalThreadsVoluntaryContextSwitches: 9.44K (9440) Fragment Instance Lifecycle Timings: - ExecTime: 591.058us - ExecTreeExecTime: 5.283us - OpenTime: 2s126ms - ExecTreeOpenTime: 2s079ms - PrepareTime: 394.507ms - ExecTreePrepareTime: 6.809ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 84.453us, non-child: 84.453us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 286.83 KB/sec - OverallThroughput: 185.01 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.839us - TransmitDataRPCTime: 54.475us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.245ms, non-child: 24.245ms, % non-child: 100.00%) - CodegenTime: 599.657us - CompileTime: 3.907ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.712ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.270ms AGGREGATION_NODE (id=1):(Total: 2s085ms, non-child: 5.323ms, % non-child: 0.26%) ExecOption: Codegen Enabled - BuildTime: 1.574ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 2s080ms, non-child: 2s080ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:239/326.30 KB 0:236/322.22 KB 3:266/362.76 KB 6:249/339.21 KB 7:246/335.62 KB 1:241/328.60 KB 4:229/312.04 KB 5:237/323.71 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 236.01 KB, 1.01 MB, 1.72 MB, 2.26 MB - FooterProcessingTime: (Avg: 16.365ms ; Min: 5.320ms ; Max: 190.714ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714067) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714067) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714067) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.91 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 2s035ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 933.00 /sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 3m1s - MaterializeTupleTime(*): 2.255ms - ScannerThreadsSysTime: 29.977ms - ScannerThreadsUserTime: 327.912ms - ScannerThreadsVoluntaryContextSwitches: 7.34K (7344) - TotalRawHdfsReadTime(*): 25.150ms - TotalReadThroughput: 1.13 MB/sec Instance c34cc30188b6a8c9:7c (host=vhs036.foo.fa.com:22000):(Total: 2s498ms, non-child: 337.453ms, % non-child: 13.51%) Hdfs split stats (:<# splits>/): 7:236/322.04 KB 0:231/314.85 KB 2:267/364.19 KB 6:256/349.11 KB 1:237/324.00 KB 4:221/301.07 KB 5:252/343.51 KB 3:244/332.45 KB MemoryUsage(500.000ms): 184.36 KB, 208.36 KB, 184.36 KB, 184.36 KB, 86.36 KB ThreadUsage(500.000ms): 89, 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 270.36 KB (276846) - PerHostPeakMemUsage: 270.36 KB (276846) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 437.732us - TotalStorageWaitTime: 32s368ms - TotalThreadsInvoluntaryContextSwitches: 22 (22) - TotalThreadsTotalWallClockTime: 3m8s - TotalThreadsSysTime: 98.956ms - TotalThreadsUserTime: 411.901ms - TotalThreadsVoluntaryContextSwitches: 9.31K (9309) Fragment Instance Lifecycle Timings: - ExecTime: 626.304us - ExecTreeExecTime: 4.336us - OpenTime: 2s175ms - ExecTreeOpenTime: 2s129ms - PrepareTime: 322.360ms - ExecTreePrepareTime: 6.535ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 93.971us, non-child: 93.971us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 207.71 KB/sec - OverallThroughput: 166.27 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.487us - TransmitDataRPCTime: 75.224us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.803ms, non-child: 24.803ms, % non-child: 100.00%) - CodegenTime: 599.163us - CompileTime: 4.094ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.821ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.475ms AGGREGATION_NODE (id=1):(Total: 2s136ms, non-child: 3.436ms, % non-child: 0.16%) ExecOption: Codegen Enabled - BuildTime: 1.274ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 2s132ms, non-child: 2s132ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:236/322.04 KB 0:231/314.85 KB 2:267/364.19 KB 6:256/349.11 KB 1:237/324.00 KB 4:221/301.07 KB 5:252/343.51 KB 3:244/332.45 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 0, 870.04 KB, 1.53 MB, 1.61 MB, 2.56 MB - FooterProcessingTime: (Avg: 16.716ms ; Min: 5.699ms ; Max: 525.922ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714854) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714854) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714854) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 230.00 KB (235520) - PerReadThreadRawHdfsThroughput: 105.41 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 2s088ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 911.00 /sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 3m6s - MaterializeTupleTime(*): 2.187ms - ScannerThreadsSysTime: 68.961ms - ScannerThreadsUserTime: 307.917ms - ScannerThreadsVoluntaryContextSwitches: 7.25K (7248) - TotalRawHdfsReadTime(*): 24.562ms - TotalReadThroughput: 1.02 MB/sec Instance c34cc30188b6a8c9:4 (host=vc1511.foo.fa.com:22000):(Total: 2s461ms, non-child: 431.851ms, % non-child: 17.54%) Hdfs split stats (:<# splits>/): 5:248/338.13 KB 0:312/425.57 KB 1:275/374.99 KB 3:260/354.35 KB 4:286/389.68 KB 2:289/394.32 KB 6:274/373.23 KB MemoryUsage(500.000ms): 0, 166.36 KB, 166.36 KB, 166.36 KB, 84.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 230.36 KB (235886) - PerHostPeakMemUsage: 230.36 KB (235886) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 534.869us - TotalStorageWaitTime: 26s220ms - TotalThreadsInvoluntaryContextSwitches: 29 (29) - TotalThreadsTotalWallClockTime: 2m56s - TotalThreadsSysTime: 72.965ms - TotalThreadsUserTime: 436.895ms - TotalThreadsVoluntaryContextSwitches: 9.47K (9472) Fragment Instance Lifecycle Timings: - ExecTime: 738.885us - ExecTreeExecTime: 5.977us - OpenTime: 2s046ms - ExecTreeOpenTime: 1s998ms - PrepareTime: 414.392ms - ExecTreePrepareTime: 6.907ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 80.769us, non-child: 80.769us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 260.52 KB/sec - OverallThroughput: 193.45 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.001us - TransmitDataRPCTime: 59.975us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.181ms, non-child: 24.181ms, % non-child: 100.00%) - CodegenTime: 575.598us - CompileTime: 3.932ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.708ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.174ms AGGREGATION_NODE (id=1):(Total: 2s005ms, non-child: 5.428ms, % non-child: 0.27%) ExecOption: Codegen Enabled - BuildTime: 1.539ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 2s000ms, non-child: 2s000ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:248/338.13 KB 0:312/425.57 KB 1:275/374.99 KB 3:260/354.35 KB 4:286/389.68 KB 2:289/394.32 KB 6:274/373.23 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 499.62 KB, 1.15 MB, 1.64 MB, 2.55 MB - FooterProcessingTime: (Avg: 13.557ms ; Min: 5.453ms ; Max: 248.832ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713875) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713875) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713875) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 7 (7) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 190.00 KB (194560) - PerReadThreadRawHdfsThroughput: 99.20 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s955ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 971.00 /sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 2m54s - MaterializeTupleTime(*): 2.459ms - ScannerThreadsSysTime: 41.969ms - ScannerThreadsUserTime: 324.912ms - ScannerThreadsVoluntaryContextSwitches: 7.43K (7428) - TotalRawHdfsReadTime(*): 26.090ms - TotalReadThroughput: 1.27 MB/sec Instance c34cc30188b6a8c9:4e (host=vc1532.foo.fa.com:22000):(Total: 2s413ms, non-child: 341.066ms, % non-child: 14.13%) Hdfs split stats (:<# splits>/): 2:236/322.35 KB 3:244/332.45 KB 6:269/366.46 KB 7:229/312.61 KB 0:235/320.44 KB 4:264/359.97 KB 1:236/321.72 KB 5:231/314.65 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 440.036us - TotalStorageWaitTime: 31s130ms - TotalThreadsInvoluntaryContextSwitches: 31 (31) - TotalThreadsTotalWallClockTime: 3m - TotalThreadsSysTime: 73.965ms - TotalThreadsUserTime: 442.897ms - TotalThreadsVoluntaryContextSwitches: 9.37K (9366) Fragment Instance Lifecycle Timings: - ExecTime: 666.014us - ExecTreeExecTime: 12.580us - OpenTime: 2s088ms - ExecTreeOpenTime: 2s042ms - PrepareTime: 324.529ms - ExecTreePrepareTime: 6.512ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 105.187us, non-child: 105.187us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 210.38 KB/sec - OverallThroughput: 148.54 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 14.627us - TransmitDataRPCTime: 74.271us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.102ms, non-child: 24.102ms, % non-child: 100.00%) - CodegenTime: 570.975us - CompileTime: 4.074ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.715ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 12.936ms AGGREGATION_NODE (id=1):(Total: 2s048ms, non-child: 6.407ms, % non-child: 0.31%) ExecOption: Codegen Enabled - BuildTime: 1.694ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 2s042ms, non-child: 2s042ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:236/322.35 KB 3:244/332.45 KB 6:269/366.46 KB 7:229/312.61 KB 0:235/320.44 KB 4:264/359.97 KB 1:236/321.72 KB 5:231/314.65 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 99.68 KB, 643.59 KB, 1.20 MB, 1.87 MB - FooterProcessingTime: (Avg: 16.082ms ; Min: 5.764ms ; Max: 246.876ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714273) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714273) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714273) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.48 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s996ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 951.00 /sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 7 (7) - ScannerThreadsTotalWallClockTime: 2m58s - MaterializeTupleTime(*): 2.365ms - ScannerThreadsSysTime: 48.969ms - ScannerThreadsUserTime: 325.915ms - ScannerThreadsVoluntaryContextSwitches: 7.30K (7301) - TotalRawHdfsReadTime(*): 25.259ms - TotalReadThroughput: 956.49 KB/sec Instance c34cc30188b6a8c9:c (host=vd1122.foo.fa.com:22000):(Total: 2s389ms, non-child: 421.806ms, % non-child: 17.65%) Hdfs split stats (:<# splits>/): 7:243/330.95 KB 2:238/324.49 KB 4:250/340.59 KB 5:235/320.04 KB 0:246/334.93 KB 3:238/324.41 KB 6:257/350.04 KB 1:239/325.94 KB MemoryUsage(500.000ms): 20.00 KB, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 470.973us - TotalStorageWaitTime: 30s054ms - TotalThreadsInvoluntaryContextSwitches: 49 (49) - TotalThreadsTotalWallClockTime: 2m51s - TotalThreadsSysTime: 90.965ms - TotalThreadsUserTime: 419.902ms - TotalThreadsVoluntaryContextSwitches: 9.46K (9464) Fragment Instance Lifecycle Timings: - ExecTime: 671.355us - ExecTreeExecTime: 5.560us - OpenTime: 1s984ms - ExecTreeOpenTime: 1s936ms - PrepareTime: 404.290ms - ExecTreePrepareTime: 6.808ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 80.406us, non-child: 80.406us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 255.33 KB/sec - OverallThroughput: 194.33 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.001us - TransmitDataRPCTime: 61.195us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.328ms, non-child: 24.328ms, % non-child: 100.00%) - CodegenTime: 606.736us - CompileTime: 4.062ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.752ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.146ms AGGREGATION_NODE (id=1):(Total: 1s943ms, non-child: 3.951ms, % non-child: 0.20%) ExecOption: Codegen Enabled - BuildTime: 1.321ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s939ms, non-child: 1s939ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:243/330.95 KB 2:238/324.49 KB 4:250/340.59 KB 5:235/320.04 KB 0:246/334.93 KB 3:238/324.41 KB 6:257/350.04 KB 1:239/325.94 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1946 out of 1946 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1946 BytesRead(500.000ms): 0, 692.45 KB, 1.33 MB, 1.75 MB - FooterProcessingTime: (Avg: 15.511ms ; Min: 5.198ms ; Max: 250.252ms ; Number of samples: 1946) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2715029) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2715029) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2715029) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 200.00 KB (204800) - PerReadThreadRawHdfsThroughput: 104.00 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s893ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1946) - RowsReturned: 1.95K (1946) - RowsReturnedRate: 1.00 K/sec - ScanRangesComplete: 1.95K (1946) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 2m49s - MaterializeTupleTime(*): 2.248ms - ScannerThreadsSysTime: 63.969ms - ScannerThreadsUserTime: 309.919ms - ScannerThreadsVoluntaryContextSwitches: 7.36K (7358) - TotalRawHdfsReadTime(*): 24.897ms - TotalReadThroughput: 895.60 KB/sec Instance c34cc30188b6a8c9:57 (host=vhs034.foo.fa.com:22000):(Total: 2s387ms, non-child: 469.148ms, % non-child: 19.65%) Hdfs split stats (:<# splits>/): 1:237/323.07 KB 7:221/301.12 KB 0:282/385.25 KB 3:240/327.28 KB 4:243/331.15 KB 5:219/298.53 KB 2:241/328.16 KB 6:261/355.96 KB MemoryUsage(500.000ms): 0, 194.36 KB, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 552.833us - TotalStorageWaitTime: 27s641ms - TotalThreadsInvoluntaryContextSwitches: 29 (29) - TotalThreadsTotalWallClockTime: 2m46s - TotalThreadsSysTime: 85.960ms - TotalThreadsUserTime: 420.900ms - TotalThreadsVoluntaryContextSwitches: 9.08K (9076) Fragment Instance Lifecycle Timings: - ExecTime: 710.635us - ExecTreeExecTime: 4.935us - OpenTime: 1s946ms - ExecTreeOpenTime: 1s884ms - PrepareTime: 440.240ms - ExecTreePrepareTime: 7.730ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 66.891us, non-child: 66.891us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 320.93 KB/sec - OverallThroughput: 233.59 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.090us - TransmitDataRPCTime: 48.686us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.758ms, non-child: 25.758ms, % non-child: 100.00%) - CodegenTime: 622.628us - CompileTime: 4.401ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.693ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.293ms AGGREGATION_NODE (id=1):(Total: 1s892ms, non-child: 2.455ms, % non-child: 0.13%) ExecOption: Codegen Enabled - BuildTime: 1.462ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s889ms, non-child: 1s889ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:237/323.07 KB 7:221/301.12 KB 0:282/385.25 KB 3:240/327.28 KB 4:243/331.15 KB 5:219/298.53 KB 2:241/328.16 KB 6:261/355.96 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 100.72 KB, 965.56 KB, 1.35 MB, 2.19 MB - FooterProcessingTime: (Avg: 14.284ms ; Min: 5.884ms ; Max: 214.882ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714117) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714117) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714117) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 117.64 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s846ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.03 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m44s - MaterializeTupleTime(*): 2.077ms - ScannerThreadsSysTime: 64.964ms - ScannerThreadsUserTime: 303.918ms - ScannerThreadsVoluntaryContextSwitches: 6.99K (6991) - TotalRawHdfsReadTime(*): 22.002ms - TotalReadThroughput: 1.09 MB/sec Instance c34cc30188b6a8c9:68 (host=vd1119.foo.fa.com:22000):(Total: 2s382ms, non-child: 437.125ms, % non-child: 18.35%) Hdfs split stats (:<# splits>/): 1:260/354.73 KB 4:225/306.96 KB 2:229/312.27 KB 3:264/359.41 KB 6:252/343.05 KB 5:242/330.26 KB 0:222/302.41 KB 7:251/342.11 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 513.006us - TotalStorageWaitTime: 29s181ms - TotalThreadsInvoluntaryContextSwitches: 24 (24) - TotalThreadsTotalWallClockTime: 2m49s - TotalThreadsSysTime: 66.968ms - TotalThreadsUserTime: 432.898ms - TotalThreadsVoluntaryContextSwitches: 9.36K (9357) Fragment Instance Lifecycle Timings: - ExecTime: 742.643us - ExecTreeExecTime: 5.303us - OpenTime: 1s962ms - ExecTreeOpenTime: 1s914ms - PrepareTime: 418.793ms - ExecTreePrepareTime: 6.605ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 74.903us, non-child: 74.903us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 278.59 KB/sec - OverallThroughput: 208.60 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.751us - TransmitDataRPCTime: 56.086us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.253ms, non-child: 24.253ms, % non-child: 100.00%) - CodegenTime: 592.404us - CompileTime: 3.926ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.656ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.267ms AGGREGATION_NODE (id=1):(Total: 1s920ms, non-child: 5.060ms, % non-child: 0.26%) ExecOption: Codegen Enabled - BuildTime: 1.591ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s915ms, non-child: 1s915ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:260/354.73 KB 4:225/306.96 KB 2:229/312.27 KB 3:264/359.41 KB 6:252/343.05 KB 5:242/330.26 KB 0:222/302.41 KB 7:251/342.11 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 0, 836.65 KB, 1.48 MB, 2.04 MB - FooterProcessingTime: (Avg: 15.070ms ; Min: 5.579ms ; Max: 211.921ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714822) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714822) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714822) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 101.50 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s870ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.01 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 6 (6) - ScannerThreadsTotalWallClockTime: 2m47s - MaterializeTupleTime(*): 2.310ms - ScannerThreadsSysTime: 38.972ms - ScannerThreadsUserTime: 334.913ms - ScannerThreadsVoluntaryContextSwitches: 7.25K (7246) - TotalRawHdfsReadTime(*): 25.509ms - TotalReadThroughput: 1.02 MB/sec Instance c34cc30188b6a8c9:1 (host=vhs026.foo.fa.com:22000):(Total: 2s366ms, non-child: 489.898ms, % non-child: 20.70%) Hdfs split stats (:<# splits>/): 3:231/315.29 KB 1:256/349.33 KB 2:243/331.42 KB 0:243/331.39 KB 4:244/332.72 KB 7:239/326.00 KB 5:232/316.66 KB 6:256/349.56 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 296.37 KB (303478) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 312.760us - TotalStorageWaitTime: 28s385ms - TotalThreadsInvoluntaryContextSwitches: 23 (23) - TotalThreadsTotalWallClockTime: 2m42s - TotalThreadsSysTime: 94.957ms - TotalThreadsUserTime: 460.894ms - TotalThreadsVoluntaryContextSwitches: 9.59K (9589) Fragment Instance Lifecycle Timings: - ExecTime: 593.794us - ExecTreeExecTime: 4.832us - OpenTime: 1s911ms - ExecTreeOpenTime: 1s841ms - PrepareTime: 454.775ms - ExecTreePrepareTime: 8.894ms DataStreamSender (dst_id=2):(Total: 80.837us, non-child: 80.837us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 252.52 KB/sec - OverallThroughput: 193.29 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.773us - TransmitDataRPCTime: 61.876us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 26.656ms, non-child: 26.656ms, % non-child: 100.00%) - CodegenTime: 583.158us - CompileTime: 4.270ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 7.073ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.904ms AGGREGATION_NODE (id=1):(Total: 1s850ms, non-child: 7.256ms, % non-child: 0.39%) ExecOption: Codegen Enabled - BuildTime: 1.928ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s842ms, non-child: 1s842ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:231/315.29 KB 1:256/349.33 KB 2:243/331.42 KB 0:243/331.39 KB 4:244/332.72 KB 7:239/326.00 KB 5:232/316.66 KB 6:256/349.56 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 73.65 KB, 774.58 KB, 1.17 MB, 2.11 MB - FooterProcessingTime: (Avg: 14.681ms ; Min: 4.812ms ; Max: 250.943ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2716021) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2716021) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2716021) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 98.25 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s787ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.05 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 11 (11) - ScannerThreadsTotalWallClockTime: 2m40s - MaterializeTupleTime(*): 2.543ms - ScannerThreadsSysTime: 64.961ms - ScannerThreadsUserTime: 340.913ms - ScannerThreadsVoluntaryContextSwitches: 7.21K (7212) - TotalRawHdfsReadTime(*): 26.363ms - TotalReadThroughput: 1.05 MB/sec Instance c34cc30188b6a8c9:66 (host=vd1127.foo.fa.com:22000):(Total: 2s338ms, non-child: 346.955ms, % non-child: 14.84%) Hdfs split stats (:<# splits>/): 3:222/302.75 KB 5:242/329.68 KB 6:250/340.57 KB 4:262/357.02 KB 0:240/327.25 KB 1:269/366.66 KB 7:228/310.50 KB 2:231/314.90 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 194.36 KB, 166.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81, 81 - AverageThreadTokens: 65.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 318.363us - TotalStorageWaitTime: 29s587ms - TotalThreadsInvoluntaryContextSwitches: 22 (22) - TotalThreadsTotalWallClockTime: 2m37s - TotalThreadsSysTime: 102.960ms - TotalThreadsUserTime: 397.908ms - TotalThreadsVoluntaryContextSwitches: 9.22K (9216) Fragment Instance Lifecycle Timings: - ExecTime: 504.702us - ExecTreeExecTime: 3.976us - OpenTime: 2s008ms - ExecTreeOpenTime: 1s957ms - PrepareTime: 329.231ms - ExecTreePrepareTime: 7.093ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 65.257us, non-child: 65.257us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 322.69 KB/sec - OverallThroughput: 239.44 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.495us - TransmitDataRPCTime: 48.421us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 26.096ms, non-child: 26.096ms, % non-child: 100.00%) - CodegenTime: 756.636us - CompileTime: 4.103ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 7.933ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.673ms AGGREGATION_NODE (id=1):(Total: 1s964ms, non-child: 5.511ms, % non-child: 0.28%) ExecOption: Codegen Enabled - BuildTime: 1.625ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s959ms, non-child: 1s959ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:222/302.75 KB 5:242/329.68 KB 6:250/340.57 KB 4:262/357.02 KB 0:240/327.25 KB 1:269/366.66 KB 7:228/310.50 KB 2:231/314.90 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 519.59 KB, 1.28 MB, 1.91 MB, 2.38 MB - FooterProcessingTime: (Avg: 15.285ms ; Min: 5.989ms ; Max: 211.986ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2712915) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712915) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712915) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.81 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s916ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 992.00 /sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 2m35s - MaterializeTupleTime(*): 2.289ms - ScannerThreadsSysTime: 67.965ms - ScannerThreadsUserTime: 298.923ms - ScannerThreadsVoluntaryContextSwitches: 7.16K (7155) - TotalRawHdfsReadTime(*): 25.165ms - TotalReadThroughput: 1.19 MB/sec Instance c34cc30188b6a8c9:56 (host=vd1142.foo.fa.com:22000):(Total: 2s328ms, non-child: 414.375ms, % non-child: 17.80%) Hdfs split stats (:<# splits>/): 3:258/351.97 KB 0:262/357.43 KB 5:239/326.13 KB 1:234/318.81 KB 4:245/333.95 KB 7:232/316.44 KB 6:228/310.37 KB 2:247/336.65 KB MemoryUsage(500.000ms): 28.00 KB, 186.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 226.36 KB (231790) - PerHostPeakMemUsage: 226.36 KB (231790) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 287.974us - TotalStorageWaitTime: 28s828ms - TotalThreadsInvoluntaryContextSwitches: 22 (22) - TotalThreadsTotalWallClockTime: 2m31s - TotalThreadsSysTime: 61.968ms - TotalThreadsUserTime: 403.906ms - TotalThreadsVoluntaryContextSwitches: 9.40K (9400) Fragment Instance Lifecycle Timings: - ExecTime: 573.720us - ExecTreeExecTime: 5.382us - OpenTime: 1s929ms - ExecTreeOpenTime: 1s882ms - PrepareTime: 398.635ms - ExecTreePrepareTime: 7.083ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 69.932us, non-child: 69.932us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 296.91 KB/sec - OverallThroughput: 223.43 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.514us - TransmitDataRPCTime: 52.625us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.147ms, non-child: 24.147ms, % non-child: 100.00%) - CodegenTime: 545.500us - CompileTime: 4.152ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.523ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.103ms AGGREGATION_NODE (id=1):(Total: 1s890ms, non-child: 3.808ms, % non-child: 0.20%) ExecOption: Codegen Enabled - BuildTime: 1.593ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s886ms, non-child: 1s886ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:258/351.97 KB 0:262/357.43 KB 5:239/326.13 KB 1:234/318.81 KB 4:245/333.95 KB 7:232/316.44 KB 6:228/310.37 KB 2:247/336.65 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 0, 964.06 KB, 1.72 MB, 2.12 MB - FooterProcessingTime: (Avg: 14.881ms ; Min: 3.608ms ; Max: 214.933ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2715385) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2715385) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2715385) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 186.00 KB (190464) - PerReadThreadRawHdfsThroughput: 115.69 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s840ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.03 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m29s - MaterializeTupleTime(*): 2.043ms - ScannerThreadsSysTime: 36.972ms - ScannerThreadsUserTime: 305.921ms - ScannerThreadsVoluntaryContextSwitches: 7.33K (7327) - TotalRawHdfsReadTime(*): 22.383ms - TotalReadThroughput: 1.06 MB/sec Instance c34cc30188b6a8c9:7 (host=vhs033.foo.fa.com:22000):(Total: 2s320ms, non-child: 352.670ms, % non-child: 15.20%) Hdfs split stats (:<# splits>/): 5:240/327.48 KB 4:226/308.41 KB 3:232/316.35 KB 7:252/343.92 KB 2:246/335.52 KB 1:254/346.60 KB 6:250/340.79 KB 0:243/331.64 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 535.482us - TotalStorageWaitTime: 30s102ms - TotalThreadsInvoluntaryContextSwitches: 20 (20) - TotalThreadsTotalWallClockTime: 2m51s - TotalThreadsSysTime: 70.966ms - TotalThreadsUserTime: 423.895ms - TotalThreadsVoluntaryContextSwitches: 9.32K (9318) Fragment Instance Lifecycle Timings: - ExecTime: 714.877us - ExecTreeExecTime: 5.867us - OpenTime: 1s981ms - ExecTreeOpenTime: 1s936ms - PrepareTime: 337.639ms - ExecTreePrepareTime: 6.464ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 80.446us, non-child: 80.446us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 256.36 KB/sec - OverallThroughput: 194.23 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.198us - TransmitDataRPCTime: 60.950us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.949ms, non-child: 23.949ms, % non-child: 100.00%) - CodegenTime: 573.890us - CompileTime: 3.970ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.535ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.089ms AGGREGATION_NODE (id=1):(Total: 1s943ms, non-child: 6.040ms, % non-child: 0.31%) ExecOption: Codegen Enabled - BuildTime: 1.619ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s937ms, non-child: 1s937ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:240/327.48 KB 4:226/308.41 KB 3:232/316.35 KB 7:252/343.92 KB 2:246/335.52 KB 1:254/346.60 KB 6:250/340.79 KB 0:243/331.64 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 19.10 KB, 680.94 KB, 1.27 MB, 1.84 MB - FooterProcessingTime: (Avg: 15.559ms ; Min: 5.173ms ; Max: 250.562ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714323) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714323) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714323) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 112.99 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s892ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.00 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 2m49s - MaterializeTupleTime(*): 2.324ms - ScannerThreadsSysTime: 38.970ms - ScannerThreadsUserTime: 314.912ms - ScannerThreadsVoluntaryContextSwitches: 7.23K (7233) - TotalRawHdfsReadTime(*): 22.908ms - TotalReadThroughput: 943.44 KB/sec Instance c34cc30188b6a8c9:55 (host=vd1133.foo.fa.com:22000):(Total: 2s318ms, non-child: 417.211ms, % non-child: 18.00%) Hdfs split stats (:<# splits>/): 0:242/330.00 KB 1:245/333.98 KB 3:223/304.12 KB 4:235/320.60 KB 5:278/378.66 KB 7:243/330.99 KB 6:236/321.59 KB 2:243/331.15 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB, 166.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81, 81 - AverageThreadTokens: 65.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 280.36 KB (287086) - PerHostPeakMemUsage: 280.36 KB (287086) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 352.241us - TotalStorageWaitTime: 27s610ms - TotalThreadsInvoluntaryContextSwitches: 40 (40) - TotalThreadsTotalWallClockTime: 2m30s - TotalThreadsSysTime: 89.961ms - TotalThreadsUserTime: 447.905ms - TotalThreadsVoluntaryContextSwitches: 9.16K (9158) Fragment Instance Lifecycle Timings: - ExecTime: 591.258us - ExecTreeExecTime: 5.963us - OpenTime: 1s916ms - ExecTreeOpenTime: 1s868ms - PrepareTime: 400.806ms - ExecTreePrepareTime: 7.050ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 89.623us, non-child: 89.623us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 223.08 KB/sec - OverallThroughput: 174.34 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.925us - TransmitDataRPCTime: 70.042us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.126ms, non-child: 25.126ms, % non-child: 100.00%) - CodegenTime: 614.550us - CompileTime: 4.219ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.671ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.828ms AGGREGATION_NODE (id=1):(Total: 1s875ms, non-child: 5.611ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.766ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s870ms, non-child: 1s870ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:242/330.00 KB 1:245/333.98 KB 3:223/304.12 KB 4:235/320.60 KB 5:278/378.66 KB 7:243/330.99 KB 6:236/321.59 KB 2:243/331.15 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 280.78 KB, 1.27 MB, 1.89 MB, 2.43 MB - FooterProcessingTime: (Avg: 14.267ms ; Min: 5.909ms ; Max: 214.205ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2714711) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714711) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714711) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 240.00 KB (245760) - PerReadThreadRawHdfsThroughput: 103.88 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s823ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.04 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 10 (10) - ScannerThreadsTotalWallClockTime: 2m28s - MaterializeTupleTime(*): 2.612ms - ScannerThreadsSysTime: 54.966ms - ScannerThreadsUserTime: 342.921ms - ScannerThreadsVoluntaryContextSwitches: 7.08K (7078) - TotalRawHdfsReadTime(*): 24.923ms - TotalReadThroughput: 1.22 MB/sec Instance c34cc30188b6a8c9:b (host=ve1136.foo.fa.com:22000):(Total: 2s316ms, non-child: 341.222ms, % non-child: 14.73%) Hdfs split stats (:<# splits>/): 3:248/337.98 KB 1:232/316.15 KB 5:247/336.89 KB 6:239/325.23 KB 0:217/295.46 KB 7:263/358.17 KB 2:262/357.53 KB 4:237/322.85 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB, 194.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81, 81 - AverageThreadTokens: 65.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 527.450us - TotalStorageWaitTime: 29s818ms - TotalThreadsInvoluntaryContextSwitches: 24 (24) - TotalThreadsTotalWallClockTime: 2m36s - TotalThreadsSysTime: 83.964ms - TotalThreadsUserTime: 425.907ms - TotalThreadsVoluntaryContextSwitches: 9.39K (9390) Fragment Instance Lifecycle Timings: - ExecTime: 715.151us - ExecTreeExecTime: 5.115us - OpenTime: 1s988ms - ExecTreeOpenTime: 1s943ms - PrepareTime: 326.738ms - ExecTreePrepareTime: 6.912ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 85.540us, non-child: 85.540us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 235.28 KB/sec - OverallThroughput: 182.66 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.280us - TransmitDataRPCTime: 66.409us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.990ms, non-child: 24.990ms, % non-child: 100.00%) - CodegenTime: 631.133us - CompileTime: 4.157ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.863ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.558ms AGGREGATION_NODE (id=1):(Total: 1s950ms, non-child: 5.874ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.699ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s944ms, non-child: 1s944ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:248/337.98 KB 1:232/316.15 KB 5:247/336.89 KB 6:239/325.23 KB 0:217/295.46 KB 7:263/358.17 KB 2:262/357.53 KB 4:237/322.85 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:75% 1:25% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 191.97 KB, 808.79 KB, 1.22 MB, 2.08 MB - FooterProcessingTime: (Avg: 15.400ms ; Min: 4.962ms ; Max: 250.567ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.25 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2713865) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713865) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713865) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 200.00 KB (204800) - PerReadThreadRawHdfsThroughput: 103.10 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s897ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.00 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m34s - MaterializeTupleTime(*): 2.412ms - ScannerThreadsSysTime: 55.968ms - ScannerThreadsUserTime: 321.923ms - ScannerThreadsVoluntaryContextSwitches: 7.30K (7299) - TotalRawHdfsReadTime(*): 25.102ms - TotalReadThroughput: 1.04 MB/sec Instance c34cc30188b6a8c9:6a (host=vd1109.foo.fa.com:22000):(Total: 2s312ms, non-child: 432.945ms, % non-child: 18.73%) Hdfs split stats (:<# splits>/): 7:236/321.49 KB 1:242/330.15 KB 4:252/343.82 KB 2:241/328.88 KB 3:245/333.74 KB 0:240/326.93 KB 5:256/348.47 KB 6:232/316.48 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 232.36 KB (237934) - PerHostPeakMemUsage: 232.36 KB (237934) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 401.681us - TotalStorageWaitTime: 28s735ms - TotalThreadsInvoluntaryContextSwitches: 27 (27) - TotalThreadsTotalWallClockTime: 2m43s - TotalThreadsSysTime: 74.966ms - TotalThreadsUserTime: 453.898ms - TotalThreadsVoluntaryContextSwitches: 9.21K (9206) Fragment Instance Lifecycle Timings: - ExecTime: 570.079us - ExecTreeExecTime: 5.255us - OpenTime: 1s895ms - ExecTreeOpenTime: 1s846ms - PrepareTime: 416.395ms - ExecTreePrepareTime: 7.074ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 93.475us, non-child: 93.475us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 212.56 KB/sec - OverallThroughput: 167.16 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.001us - TransmitDataRPCTime: 73.510us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.222ms, non-child: 25.222ms, % non-child: 100.00%) - CodegenTime: 576.053us - CompileTime: 4.180ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.816ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.797ms AGGREGATION_NODE (id=1):(Total: 1s853ms, non-child: 5.024ms, % non-child: 0.27%) ExecOption: Codegen Enabled - BuildTime: 1.600ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s848ms, non-child: 1s848ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:236/321.49 KB 1:242/330.15 KB 4:252/343.82 KB 2:241/328.88 KB 3:245/333.74 KB 0:240/326.93 KB 5:256/348.47 KB 6:232/316.48 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 522.94 KB, 1.17 MB, 1.72 MB - FooterProcessingTime: (Avg: 14.851ms ; Min: 4.952ms ; Max: 211.781ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713554) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713554) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713554) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 192.00 KB (196608) - PerReadThreadRawHdfsThroughput: 103.27 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s800ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.05 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m41s - MaterializeTupleTime(*): 2.483ms - ScannerThreadsSysTime: 45.971ms - ScannerThreadsUserTime: 342.915ms - ScannerThreadsVoluntaryContextSwitches: 7.12K (7120) - TotalRawHdfsReadTime(*): 25.059ms - TotalReadThroughput: 1.15 MB/sec Instance c34cc30188b6a8c9:65 (host=vd1107.foo.fa.com:22000):(Total: 2s309ms, non-child: 341.325ms, % non-child: 14.78%) Hdfs split stats (:<# splits>/): 7:225/306.62 KB 6:221/301.10 KB 0:240/327.04 KB 1:250/340.69 KB 2:275/375.20 KB 4:246/334.72 KB 5:243/331.63 KB 3:243/331.39 KB MemoryUsage(500.000ms): 0, 200.36 KB, 184.36 KB, 184.36 KB, 58.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 506.050us - TotalStorageWaitTime: 26s101ms - TotalThreadsInvoluntaryContextSwitches: 13 (13) - TotalThreadsTotalWallClockTime: 2m50s - TotalThreadsSysTime: 87.963ms - TotalThreadsUserTime: 462.898ms - TotalThreadsVoluntaryContextSwitches: 9.00K (8998) Fragment Instance Lifecycle Timings: - ExecTime: 841.560us - ExecTreeExecTime: 5.146us - OpenTime: 1s982ms - ExecTreeOpenTime: 1s934ms - PrepareTime: 325.829ms - ExecTreePrepareTime: 6.850ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 82.282us, non-child: 82.282us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 249.46 KB/sec - OverallThroughput: 189.90 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.710us - TransmitDataRPCTime: 62.634us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 26.794ms, non-child: 26.794ms, % non-child: 100.00%) - CodegenTime: 563.241us - CompileTime: 4.826ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 7.710ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.814ms AGGREGATION_NODE (id=1):(Total: 1s941ms, non-child: 6.026ms, % non-child: 0.31%) ExecOption: Codegen Enabled - BuildTime: 1.744ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s935ms, non-child: 1s935ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:225/306.62 KB 6:221/301.10 KB 0:240/327.04 KB 1:250/340.69 KB 2:275/375.20 KB 4:246/334.72 KB 5:243/331.63 KB 3:243/331.39 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 582.02 KB, 1.40 MB, 2.06 MB, 2.56 MB - FooterProcessingTime: (Avg: 13.503ms ; Min: 5.987ms ; Max: 195.329ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2711951) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2711951) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2711951) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 112.25 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s884ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.00 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 8 (8) - ScannerThreadsTotalWallClockTime: 2m48s - MaterializeTupleTime(*): 2.284ms - ScannerThreadsSysTime: 56.967ms - ScannerThreadsUserTime: 360.913ms - ScannerThreadsVoluntaryContextSwitches: 6.89K (6891) - TotalRawHdfsReadTime(*): 23.040ms - TotalReadThroughput: 1.28 MB/sec Instance c34cc30188b6a8c9:69 (host=vd1108.foo.fa.com:22000):(Total: 2s297ms, non-child: 417.448ms, % non-child: 18.17%) Hdfs split stats (:<# splits>/): 7:231/315.28 KB 6:250/340.64 KB 4:239/325.70 KB 5:249/339.30 KB 3:224/305.51 KB 0:252/343.30 KB 1:241/328.28 KB 2:258/351.73 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB, 58.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 3 - AverageThreadTokens: 54.20 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 528.380us - TotalStorageWaitTime: 28s509ms - TotalThreadsInvoluntaryContextSwitches: 15 (15) - TotalThreadsTotalWallClockTime: 2m43s - TotalThreadsSysTime: 75.965ms - TotalThreadsUserTime: 403.898ms - TotalThreadsVoluntaryContextSwitches: 9.52K (9515) Fragment Instance Lifecycle Timings: - ExecTime: 715.395us - ExecTreeExecTime: 5.767us - OpenTime: 1s895ms - ExecTreeOpenTime: 1s847ms - PrepareTime: 401.372ms - ExecTreePrepareTime: 7.137ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 89.544us, non-child: 89.544us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 230.15 KB/sec - OverallThroughput: 174.50 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.261us - TransmitDataRPCTime: 67.890us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.454ms, non-child: 25.454ms, % non-child: 100.00%) - CodegenTime: 673.575us - CompileTime: 4.183ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.834ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.014ms AGGREGATION_NODE (id=1):(Total: 1s854ms, non-child: 4.153ms, % non-child: 0.22%) ExecOption: Codegen Enabled - BuildTime: 1.524ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s850ms, non-child: 1s850ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:231/315.28 KB 6:250/340.64 KB 4:239/325.70 KB 5:249/339.30 KB 3:224/305.51 KB 0:252/343.30 KB 1:241/328.28 KB 2:258/351.73 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:75% 1:25% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 425.36 KB, 1.06 MB, 1.56 MB, 2.58 MB - FooterProcessingTime: (Avg: 14.727ms ; Min: 4.878ms ; Max: 211.747ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.25 - AverageScannerThreadConcurrency: 66.50 - BytesRead: 2.59 MB (2713336) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713336) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713336) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 200.00 KB (204800) - PerReadThreadRawHdfsThroughput: 104.02 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s805ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.05 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m41s - MaterializeTupleTime(*): 2.367ms - ScannerThreadsSysTime: 46.970ms - ScannerThreadsUserTime: 302.914ms - ScannerThreadsVoluntaryContextSwitches: 7.45K (7453) - TotalRawHdfsReadTime(*): 24.875ms - TotalReadThroughput: 1.29 MB/sec Instance c34cc30188b6a8c9:8 (host=ve1137.foo.fa.com:22000):(Total: 2s297ms, non-child: 337.347ms, % non-child: 14.68%) Hdfs split stats (:<# splits>/): 0:237/323.11 KB 5:243/331.78 KB 1:231/315.10 KB 3:241/329.05 KB 2:257/350.16 KB 4:225/306.19 KB 7:249/339.60 KB 6:258/351.88 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 81, 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 469.881us - TotalStorageWaitTime: 29s784ms - TotalThreadsInvoluntaryContextSwitches: 20 (20) - TotalThreadsTotalWallClockTime: 2m35s - TotalThreadsSysTime: 66.970ms - TotalThreadsUserTime: 439.904ms - TotalThreadsVoluntaryContextSwitches: 9.24K (9243) Fragment Instance Lifecycle Timings: - ExecTime: 692.637us - ExecTreeExecTime: 6.706us - OpenTime: 1s973ms - ExecTreeOpenTime: 1s928ms - PrepareTime: 323.233ms - ExecTreePrepareTime: 6.801ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 84.996us, non-child: 84.996us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 251.88 KB/sec - OverallThroughput: 183.83 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 14.356us - TransmitDataRPCTime: 62.033us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.954ms, non-child: 24.954ms, % non-child: 100.00%) - CodegenTime: 556.446us - CompileTime: 4.233ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.850ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.485ms AGGREGATION_NODE (id=1):(Total: 1s935ms, non-child: 6.333ms, % non-child: 0.33%) ExecOption: Codegen Enabled - BuildTime: 1.802ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s928ms, non-child: 1s928ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:237/323.11 KB 5:243/331.78 KB 1:231/315.10 KB 3:241/329.05 KB 2:257/350.16 KB 4:225/306.19 KB 7:249/339.60 KB 6:258/351.88 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1941 out of 1941 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1941 BytesRead(500.000ms): 0, 614.87 KB, 1.12 MB, 1.83 MB - FooterProcessingTime: (Avg: 15.414ms ; Min: 5.450ms ; Max: 250.663ms ; Number of samples: 1941) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.58 MB (2710409) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.58 MB (2710409) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.58 MB (2710409) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 105.15 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s883ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1941) - RowsReturned: 1.94K (1941) - RowsReturnedRate: 1.01 K/sec - ScanRangesComplete: 1.94K (1941) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m33s - MaterializeTupleTime(*): 2.414ms - ScannerThreadsSysTime: 42.973ms - ScannerThreadsUserTime: 331.920ms - ScannerThreadsVoluntaryContextSwitches: 7.16K (7164) - TotalRawHdfsReadTime(*): 24.583ms - TotalReadThroughput: 936.79 KB/sec Instance c34cc30188b6a8c9:67 (host=vd1134.foo.fa.com:22000):(Total: 2s284ms, non-child: 352.375ms, % non-child: 15.42%) Hdfs split stats (:<# splits>/): 6:260/354.92 KB 1:252/343.73 KB 7:230/313.84 KB 2:250/340.52 KB 4:235/320.37 KB 5:238/324.36 KB 3:243/331.02 KB 0:236/321.83 KB MemoryUsage(500.000ms): 28.00 KB, 194.36 KB, 194.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 324.410us - TotalStorageWaitTime: 29s400ms - TotalThreadsInvoluntaryContextSwitches: 21 (21) - TotalThreadsTotalWallClockTime: 2m32s - TotalThreadsSysTime: 78.968ms - TotalThreadsUserTime: 411.909ms - TotalThreadsVoluntaryContextSwitches: 9.34K (9335) Fragment Instance Lifecycle Timings: - ExecTime: 522.725us - ExecTreeExecTime: 5.717us - OpenTime: 1s948ms - ExecTreeOpenTime: 1s901ms - PrepareTime: 335.790ms - ExecTreePrepareTime: 6.528ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 93.320us, non-child: 93.320us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 254.58 KB/sec - OverallThroughput: 167.43 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.097us - TransmitDataRPCTime: 61.375us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.940ms, non-child: 23.940ms, % non-child: 100.00%) - CodegenTime: 591.255us - CompileTime: 3.974ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.589ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.016ms AGGREGATION_NODE (id=1):(Total: 1s908ms, non-child: 4.694ms, % non-child: 0.25%) ExecOption: Codegen Enabled - BuildTime: 1.595ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s903ms, non-child: 1s903ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:260/354.92 KB 1:252/343.73 KB 7:230/313.84 KB 2:250/340.52 KB 4:235/320.37 KB 5:238/324.36 KB 3:243/331.02 KB 0:236/321.83 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 0, 704.95 KB, 1.32 MB, 1.78 MB - FooterProcessingTime: (Avg: 15.190ms ; Min: 5.632ms ; Max: 212.091ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2714211) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714211) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714211) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 96.47 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s860ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.02 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 2m31s - MaterializeTupleTime(*): 2.278ms - ScannerThreadsSysTime: 51.972ms - ScannerThreadsUserTime: 312.924ms - ScannerThreadsVoluntaryContextSwitches: 7.28K (7275) - TotalRawHdfsReadTime(*): 26.830ms - TotalReadThroughput: 913.31 KB/sec Instance c34cc30188b6a8c9:21 (host=vc1531.foo.fa.com:22000):(Total: 2s279ms, non-child: 422.142ms, % non-child: 18.52%) Hdfs split stats (:<# splits>/): 1:230/313.74 KB 0:246/335.42 KB 7:231/314.83 KB 2:246/335.08 KB 4:225/306.66 KB 5:280/382.07 KB 3:236/321.83 KB 6:250/340.44 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB, 58.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 389.387us - TotalStorageWaitTime: 27s240ms - TotalThreadsInvoluntaryContextSwitches: 21 (21) - TotalThreadsTotalWallClockTime: 2m41s - TotalThreadsSysTime: 66.969ms - TotalThreadsUserTime: 424.899ms - TotalThreadsVoluntaryContextSwitches: 9.56K (9561) Fragment Instance Lifecycle Timings: - ExecTime: 584.488us - ExecTreeExecTime: 5.179us - OpenTime: 1s872ms - ExecTreeOpenTime: 1s824ms - PrepareTime: 406.120ms - ExecTreePrepareTime: 7.130ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 95.605us, non-child: 95.605us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 224.13 KB/sec - OverallThroughput: 163.43 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.535us - TransmitDataRPCTime: 69.715us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.308ms, non-child: 25.308ms, % non-child: 100.00%) - CodegenTime: 601.269us - CompileTime: 4.167ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.929ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.802ms AGGREGATION_NODE (id=1):(Total: 1s831ms, non-child: 3.796ms, % non-child: 0.21%) ExecOption: Codegen Enabled - BuildTime: 1.474ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s828ms, non-child: 1s828ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:230/313.74 KB 0:246/335.42 KB 7:231/314.83 KB 2:246/335.08 KB 4:225/306.66 KB 5:280/382.07 KB 3:236/321.83 KB 6:250/340.44 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 473.07 KB, 541.32 KB, 1.57 MB, 2.57 MB - FooterProcessingTime: (Avg: 14.080ms ; Min: 5.277ms ; Max: 506.556ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713673) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713673) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713673) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 200.00 KB (204800) - PerReadThreadRawHdfsThroughput: 105.22 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s782ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.06 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 2m39s - MaterializeTupleTime(*): 2.299ms - ScannerThreadsSysTime: 40.973ms - ScannerThreadsUserTime: 326.914ms - ScannerThreadsVoluntaryContextSwitches: 7.47K (7473) - TotalRawHdfsReadTime(*): 24.595ms - TotalReadThroughput: 1.29 MB/sec Instance c34cc30188b6a8c9:49 (host=vd1140.foo.fa.com:22000):(Total: 2s273ms, non-child: 335.216ms, % non-child: 14.74%) Hdfs split stats (:<# splits>/): 1:250/340.47 KB 5:227/308.90 KB 3:244/332.68 KB 2:243/331.66 KB 4:256/349.35 KB 6:253/344.49 KB 0:229/312.21 KB 7:243/331.24 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 186.36 KB, 142.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81, 81 - AverageThreadTokens: 65.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 335.001us - TotalStorageWaitTime: 29s001ms - TotalThreadsInvoluntaryContextSwitches: 30 (30) - TotalThreadsTotalWallClockTime: 2m33s - TotalThreadsSysTime: 81.963ms - TotalThreadsUserTime: 441.907ms - TotalThreadsVoluntaryContextSwitches: 9.31K (9310) Fragment Instance Lifecycle Timings: - ExecTime: 556.303us - ExecTreeExecTime: 5.951us - OpenTime: 1s953ms - ExecTreeOpenTime: 1s907ms - PrepareTime: 319.417ms - ExecTreePrepareTime: 6.586ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 75.686us, non-child: 75.686us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 279.04 KB/sec - OverallThroughput: 206.44 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.141us - TransmitDataRPCTime: 55.995us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.316ms, non-child: 24.316ms, % non-child: 100.00%) - CodegenTime: 554.807us - CompileTime: 3.943ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.638ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.278ms AGGREGATION_NODE (id=1):(Total: 1s914ms, non-child: 3.105ms, % non-child: 0.16%) ExecOption: Codegen Enabled - BuildTime: 1.336ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s911ms, non-child: 1s911ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:250/340.47 KB 5:227/308.90 KB 3:244/332.68 KB 2:243/331.66 KB 4:256/349.35 KB 6:253/344.49 KB 0:229/312.21 KB 7:243/331.24 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 240.17 KB, 1.25 MB, 2.00 MB, 2.54 MB - FooterProcessingTime: (Avg: 14.980ms ; Min: 5.812ms ; Max: 208.970ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2714629) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714629) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714629) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 100.74 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s865ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.02 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 2m31s - MaterializeTupleTime(*): 2.501ms - ScannerThreadsSysTime: 54.968ms - ScannerThreadsUserTime: 349.921ms - ScannerThreadsVoluntaryContextSwitches: 7.20K (7204) - TotalRawHdfsReadTime(*): 25.699ms - TotalReadThroughput: 1.27 MB/sec Instance c34cc30188b6a8c9:1e (host=vc1507.foo.fa.com:22000):(Total: 2s258ms, non-child: 419.179ms, % non-child: 18.56%) Hdfs split stats (:<# splits>/): 7:244/333.27 KB 3:253/345.75 KB 4:265/361.07 KB 6:247/336.80 KB 5:250/340.66 KB 1:238/324.60 KB 0:215/292.92 KB 2:231/315.20 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB, 58.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 304.36 KB (311662) - PerHostPeakMemUsage: 304.36 KB (311662) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 551.126us - TotalStorageWaitTime: 27s138ms - TotalThreadsInvoluntaryContextSwitches: 14 (14) - TotalThreadsTotalWallClockTime: 2m39s - TotalThreadsSysTime: 80.964ms - TotalThreadsUserTime: 416.900ms - TotalThreadsVoluntaryContextSwitches: 9.54K (9539) Fragment Instance Lifecycle Timings: - ExecTime: 774.074us - ExecTreeExecTime: 5.236us - OpenTime: 1s854ms - ExecTreeOpenTime: 1s806ms - PrepareTime: 402.723ms - ExecTreePrepareTime: 7.172ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 94.870us, non-child: 94.870us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 212.60 KB/sec - OverallThroughput: 164.70 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.318us - TransmitDataRPCTime: 73.493us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.210ms, non-child: 25.210ms, % non-child: 100.00%) - CodegenTime: 596.689us - CompileTime: 4.167ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.867ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.768ms AGGREGATION_NODE (id=1):(Total: 1s813ms, non-child: 3.854ms, % non-child: 0.21%) ExecOption: Codegen Enabled - BuildTime: 1.485ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s809ms, non-child: 1s809ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:244/333.27 KB 3:253/345.75 KB 4:265/361.07 KB 6:247/336.80 KB 5:250/340.66 KB 1:238/324.60 KB 0:215/292.92 KB 2:231/315.20 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 84.47 KB, 587.99 KB, 1.63 MB, 2.56 MB - FooterProcessingTime: (Avg: 14.034ms ; Min: 4.798ms ; Max: 456.420ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713880) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713880) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713880) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 264.00 KB (270336) - PerReadThreadRawHdfsThroughput: 101.95 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s761ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.07 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m37s - MaterializeTupleTime(*): 2.266ms - ScannerThreadsSysTime: 51.968ms - ScannerThreadsUserTime: 321.915ms - ScannerThreadsVoluntaryContextSwitches: 7.44K (7439) - TotalRawHdfsReadTime(*): 25.386ms - TotalReadThroughput: 1.28 MB/sec Instance c34cc30188b6a8c9:4d (host=vc1525.foo.fa.com:22000):(Total: 2s252ms, non-child: 417.753ms, % non-child: 18.55%) Hdfs split stats (:<# splits>/): 7:241/328.62 KB 2:232/316.36 KB 4:238/324.42 KB 5:244/332.87 KB 6:230/313.50 KB 1:243/331.56 KB 0:264/359.84 KB 3:253/344.78 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 386.589us - TotalStorageWaitTime: 27s239ms - TotalThreadsInvoluntaryContextSwitches: 34 (34) - TotalThreadsTotalWallClockTime: 2m39s - TotalThreadsSysTime: 68.974ms - TotalThreadsUserTime: 468.897ms - TotalThreadsVoluntaryContextSwitches: 9.18K (9181) Fragment Instance Lifecycle Timings: - ExecTime: 630.378us - ExecTreeExecTime: 5.696us - OpenTime: 1s850ms - ExecTreeOpenTime: 1s802ms - PrepareTime: 401.614ms - ExecTreePrepareTime: 7.255ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 88.829us, non-child: 88.829us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 266.67 KB/sec - OverallThroughput: 175.90 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 17.203us - TransmitDataRPCTime: 58.594us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.114ms, non-child: 25.114ms, % non-child: 100.00%) - CodegenTime: 639.249us - CompileTime: 4.158ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.825ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.735ms AGGREGATION_NODE (id=1):(Total: 1s809ms, non-child: 5.231ms, % non-child: 0.29%) ExecOption: Codegen Enabled - BuildTime: 1.582ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s804ms, non-child: 1s804ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:241/328.62 KB 2:232/316.36 KB 4:238/324.42 KB 5:244/332.87 KB 6:230/313.50 KB 1:243/331.56 KB 0:264/359.84 KB 3:253/344.78 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 145.74 KB, 1.15 MB, 1.84 MB, 2.38 MB - FooterProcessingTime: (Avg: 14.074ms ; Min: 6.495ms ; Max: 34.636ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2715602) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2715602) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2715602) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 100.71 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s756ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.08 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m37s - MaterializeTupleTime(*): 2.385ms - ScannerThreadsSysTime: 36.979ms - ScannerThreadsUserTime: 355.914ms - ScannerThreadsVoluntaryContextSwitches: 7.09K (7093) - TotalRawHdfsReadTime(*): 25.714ms - TotalReadThroughput: 1.19 MB/sec Instance c34cc30188b6a8c9:59 (host=vc1522.foo.fa.com:22000):(Total: 2s237ms, non-child: 395.886ms, % non-child: 17.70%) Hdfs split stats (:<# splits>/): 5:232/316.07 KB 0:232/316.45 KB 1:239/325.76 KB 2:249/339.45 KB 7:236/321.52 KB 3:262/357.08 KB 4:243/331.81 KB 6:251/342.33 KB MemoryUsage(500.000ms): 0, 184.36 KB, 194.36 KB, 184.36 KB, 94.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 260.750us - TotalStorageWaitTime: 27s726ms - TotalThreadsInvoluntaryContextSwitches: 32 (32) - TotalThreadsTotalWallClockTime: 2m40s - TotalThreadsSysTime: 79.967ms - TotalThreadsUserTime: 429.900ms - TotalThreadsVoluntaryContextSwitches: 9.27K (9265) Fragment Instance Lifecycle Timings: - ExecTime: 471.453us - ExecTreeExecTime: 4.959us - OpenTime: 1s856ms - ExecTreeOpenTime: 1s810ms - PrepareTime: 379.958ms - ExecTreePrepareTime: 6.469ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 76.103us, non-child: 76.103us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 328.58 KB/sec - OverallThroughput: 205.31 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.331us - TransmitDataRPCTime: 47.553us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.223ms, non-child: 24.223ms, % non-child: 100.00%) - CodegenTime: 639.366us - CompileTime: 4.076ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.549ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.175ms AGGREGATION_NODE (id=1):(Total: 1s816ms, non-child: 5.780ms, % non-child: 0.32%) ExecOption: Codegen Enabled - BuildTime: 1.687ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s811ms, non-child: 1s811ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:232/316.07 KB 0:232/316.45 KB 1:239/325.76 KB 2:249/339.45 KB 7:236/321.52 KB 3:262/357.08 KB 4:243/331.81 KB 6:251/342.33 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 500.88 KB, 1.42 MB, 1.87 MB, 2.56 MB - FooterProcessingTime: (Avg: 14.328ms ; Min: 4.985ms ; Max: 214.644ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714078) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714078) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714078) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 104.31 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s766ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.07 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 12 (12) - ScannerThreadsTotalWallClockTime: 2m38s - MaterializeTupleTime(*): 2.222ms - ScannerThreadsSysTime: 49.971ms - ScannerThreadsUserTime: 332.915ms - ScannerThreadsVoluntaryContextSwitches: 7.19K (7188) - TotalRawHdfsReadTime(*): 24.812ms - TotalReadThroughput: 1.28 MB/sec Instance c34cc30188b6a8c9:5c (host=vc1535.foo.fa.com:22000):(Total: 2s228ms, non-child: 414.767ms, % non-child: 18.62%) Hdfs split stats (:<# splits>/): 5:228/310.99 KB 7:237/323.23 KB 1:243/331.57 KB 3:249/339.07 KB 4:228/310.60 KB 0:258/351.62 KB 6:254/346.33 KB 2:247/337.19 KB MemoryUsage(500.000ms): 113.50 KB, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 410.623us - TotalStorageWaitTime: 27s335ms - TotalThreadsInvoluntaryContextSwitches: 18 (18) - TotalThreadsTotalWallClockTime: 2m37s - TotalThreadsSysTime: 72.966ms - TotalThreadsUserTime: 442.897ms - TotalThreadsVoluntaryContextSwitches: 9.48K (9479) Fragment Instance Lifecycle Timings: - ExecTime: 682.729us - ExecTreeExecTime: 5.619us - OpenTime: 1s828ms - ExecTreeOpenTime: 1s780ms - PrepareTime: 399.277ms - ExecTreePrepareTime: 7.265ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 86.142us, non-child: 86.142us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 244.45 KB/sec - OverallThroughput: 181.39 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 14.259us - TransmitDataRPCTime: 63.920us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.514ms, non-child: 25.514ms, % non-child: 100.00%) - CodegenTime: 623.992us - CompileTime: 4.162ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.951ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.990ms AGGREGATION_NODE (id=1):(Total: 1s787ms, non-child: 4.888ms, % non-child: 0.27%) ExecOption: Codegen Enabled - BuildTime: 1.705ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s782ms, non-child: 1s782ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:228/310.99 KB 7:237/323.23 KB 1:243/331.57 KB 3:249/339.07 KB 4:228/310.60 KB 0:258/351.62 KB 6:254/346.33 KB 2:247/337.19 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 0, 415.72 KB, 1.06 MB, 2.05 MB - FooterProcessingTime: (Avg: 14.131ms ; Min: 5.562ms ; Max: 214.274ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714210) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714210) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714210) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.92 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s735ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.09 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m35s - MaterializeTupleTime(*): 2.427ms - ScannerThreadsSysTime: 39.971ms - ScannerThreadsUserTime: 333.913ms - ScannerThreadsVoluntaryContextSwitches: 7.41K (7408) - TotalRawHdfsReadTime(*): 25.150ms - TotalReadThroughput: 1.03 MB/sec Instance c34cc30188b6a8c9:4f (host=vc1517.foo.fa.com:22000):(Total: 2s210ms, non-child: 410.251ms, % non-child: 18.56%) Hdfs split stats (:<# splits>/): 4:255/348.12 KB 2:239/326.27 KB 1:254/346.60 KB 6:214/291.63 KB 3:247/336.59 KB 0:251/342.27 KB 7:251/341.77 KB 5:231/314.76 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB, 166.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 359.909us - TotalStorageWaitTime: 27s369ms - TotalThreadsInvoluntaryContextSwitches: 18 (18) - TotalThreadsTotalWallClockTime: 2m36s - TotalThreadsSysTime: 73.964ms - TotalThreadsUserTime: 424.899ms - TotalThreadsVoluntaryContextSwitches: 9.42K (9416) Fragment Instance Lifecycle Timings: - ExecTime: 539.989us - ExecTreeExecTime: 5.821us - OpenTime: 1s816ms - ExecTreeOpenTime: 1s768ms - PrepareTime: 393.629ms - ExecTreePrepareTime: 6.835ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 90.858us, non-child: 90.858us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 241.52 KB/sec - OverallThroughput: 171.97 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.609us - TransmitDataRPCTime: 64.694us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.737ms, non-child: 24.737ms, % non-child: 100.00%) - CodegenTime: 563.233us - CompileTime: 3.959ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.611ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.800ms AGGREGATION_NODE (id=1):(Total: 1s775ms, non-child: 4.015ms, % non-child: 0.23%) ExecOption: Codegen Enabled - BuildTime: 1.485ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s771ms, non-child: 1s771ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:255/348.12 KB 2:239/326.27 KB 1:254/346.60 KB 6:214/291.63 KB 3:247/336.59 KB 0:251/342.27 KB 7:251/341.77 KB 5:231/314.76 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1942 out of 1942 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1942 BytesRead(500.000ms): 237.14 KB, 1.13 MB, 1.70 MB, 2.36 MB - FooterProcessingTime: (Avg: 14.160ms ; Min: 5.750ms ; Max: 29.851ms ; Number of samples: 1942) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2711569) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2711569) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2711569) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.51 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s726ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1942) - RowsReturned: 1.94K (1942) - RowsReturnedRate: 1.10 K/sec - ScanRangesComplete: 1.94K (1942) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m34s - MaterializeTupleTime(*): 2.287ms - ScannerThreadsSysTime: 46.969ms - ScannerThreadsUserTime: 326.913ms - ScannerThreadsVoluntaryContextSwitches: 7.28K (7279) - TotalRawHdfsReadTime(*): 25.227ms - TotalReadThroughput: 1.18 MB/sec Instance c34cc30188b6a8c9:4a (host=vhs038.foo.fa.com:22000):(Total: 2s195ms, non-child: 327.561ms, % non-child: 14.92%) Hdfs split stats (:<# splits>/): 0:229/312.38 KB 7:259/352.94 KB 3:244/332.91 KB 2:259/353.33 KB 4:242/330.05 KB 6:232/316.23 KB 1:250/341.09 KB 5:227/309.72 KB MemoryUsage(500.000ms): 28.00 KB, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 236.36 KB (242030) - PerHostPeakMemUsage: 236.36 KB (242030) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 392.736us - TotalStorageWaitTime: 27s872ms - TotalThreadsInvoluntaryContextSwitches: 17 (17) - TotalThreadsTotalWallClockTime: 2m42s - TotalThreadsSysTime: 83.963ms - TotalThreadsUserTime: 422.899ms - TotalThreadsVoluntaryContextSwitches: 9.49K (9492) Fragment Instance Lifecycle Timings: - ExecTime: 569.202us - ExecTreeExecTime: 5.626us - OpenTime: 1s882ms - ExecTreeOpenTime: 1s836ms - PrepareTime: 312.276ms - ExecTreePrepareTime: 6.577ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 85.146us, non-child: 85.146us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 238.21 KB/sec - OverallThroughput: 183.51 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.923us - TransmitDataRPCTime: 65.592us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.247ms, non-child: 24.247ms, % non-child: 100.00%) - CodegenTime: 588.125us - CompileTime: 4.092ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.658ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.141ms AGGREGATION_NODE (id=1):(Total: 1s843ms, non-child: 5.604ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.632ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s837ms, non-child: 1s837ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:229/312.38 KB 7:259/352.94 KB 3:244/332.91 KB 2:259/353.33 KB 4:242/330.05 KB 6:232/316.23 KB 1:250/341.09 KB 5:227/309.72 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1942 out of 1942 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1942 BytesRead(500.000ms): 0, 746.16 KB, 1.64 MB, 2.20 MB - FooterProcessingTime: (Avg: 14.422ms ; Min: 6.361ms ; Max: 120.783ms ; Number of samples: 1942) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2712217) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712217) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712217) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 196.00 KB (200704) - PerReadThreadRawHdfsThroughput: 106.64 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s792ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1942) - RowsReturned: 1.94K (1942) - RowsReturnedRate: 1.06 K/sec - ScanRangesComplete: 1.94K (1942) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m40s - MaterializeTupleTime(*): 2.319ms - ScannerThreadsSysTime: 50.968ms - ScannerThreadsUserTime: 320.914ms - ScannerThreadsVoluntaryContextSwitches: 7.42K (7419) - TotalRawHdfsReadTime(*): 24.254ms - TotalReadThroughput: 1.10 MB/sec Instance c34cc30188b6a8c9:82 (host=ve1142.foo.fa.com:22000):(Total: 2s192ms, non-child: 337.689ms, % non-child: 15.40%) Hdfs split stats (:<# splits>/): 2:218/296.96 KB 6:243/331.19 KB 0:267/363.35 KB 3:236/321.43 KB 5:253/344.68 KB 1:213/290.75 KB 4:256/349.42 KB 7:259/352.62 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 226.36 KB (231790) - PerHostPeakMemUsage: 226.36 KB (231790) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 419.868us - TotalStorageWaitTime: 27s736ms - TotalThreadsInvoluntaryContextSwitches: 26 (26) - TotalThreadsTotalWallClockTime: 2m26s - TotalThreadsSysTime: 79.968ms - TotalThreadsUserTime: 412.909ms - TotalThreadsVoluntaryContextSwitches: 9.49K (9489) Fragment Instance Lifecycle Timings: - ExecTime: 635.210us - ExecTreeExecTime: 4.851us - OpenTime: 1s869ms - ExecTreeOpenTime: 1s823ms - PrepareTime: 322.679ms - ExecTreePrepareTime: 6.964ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 83.482us, non-child: 83.482us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 237.08 KB/sec - OverallThroughput: 187.17 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.888us - TransmitDataRPCTime: 65.905us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.871ms, non-child: 24.871ms, % non-child: 100.00%) - CodegenTime: 643.902us - CompileTime: 4.087ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.741ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.628ms AGGREGATION_NODE (id=1):(Total: 1s830ms, non-child: 4.755ms, % non-child: 0.26%) ExecOption: Codegen Enabled - BuildTime: 1.530ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s825ms, non-child: 1s825ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:218/296.96 KB 6:243/331.19 KB 0:267/363.35 KB 3:236/321.43 KB 5:253/344.68 KB 1:213/290.75 KB 4:256/349.42 KB 7:259/352.62 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 467.67 KB, 1.01 MB, 1.61 MB - FooterProcessingTime: (Avg: 14.327ms ; Min: 5.412ms ; Max: 374.307ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2714016) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714016) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714016) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 186.00 KB (190464) - PerReadThreadRawHdfsThroughput: 110.88 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s779ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.06 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 2m24s - MaterializeTupleTime(*): 2.281ms - ScannerThreadsSysTime: 46.973ms - ScannerThreadsUserTime: 317.923ms - ScannerThreadsVoluntaryContextSwitches: 7.42K (7416) - TotalRawHdfsReadTime(*): 23.342ms - TotalReadThroughput: 1.08 MB/sec Instance c34cc30188b6a8c9:5b (host=vc1510.foo.fa.com:22000):(Total: 2s192ms, non-child: 325.373ms, % non-child: 14.84%) Hdfs split stats (:<# splits>/): 2:221/301.48 KB 4:244/332.21 KB 1:251/342.31 KB 0:207/282.04 KB 7:243/331.30 KB 3:269/366.04 KB 5:249/339.74 KB 6:260/354.46 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 184.36 KB, 166.36 KB ThreadUsage(500.000ms): 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 480.488us - TotalStorageWaitTime: 27s591ms - TotalThreadsInvoluntaryContextSwitches: 33 (33) - TotalThreadsTotalWallClockTime: 2m42s - TotalThreadsSysTime: 77.963ms - TotalThreadsUserTime: 422.901ms - TotalThreadsVoluntaryContextSwitches: 9.21K (9210) Fragment Instance Lifecycle Timings: - ExecTime: 657.480us - ExecTreeExecTime: 6.225us - OpenTime: 1s879ms - ExecTreeOpenTime: 1s835ms - PrepareTime: 312.024ms - ExecTreePrepareTime: 6.603ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 94.778us, non-child: 94.778us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 206.98 KB/sec - OverallThroughput: 164.86 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.643us - TransmitDataRPCTime: 75.491us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.537ms, non-child: 24.537ms, % non-child: 100.00%) - CodegenTime: 576.838us - CompileTime: 4.152ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.659ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.299ms AGGREGATION_NODE (id=1):(Total: 1s842ms, non-child: 4.763ms, % non-child: 0.26%) ExecOption: Codegen Enabled - BuildTime: 1.550ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s837ms, non-child: 1s837ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:221/301.48 KB 4:244/332.21 KB 1:251/342.31 KB 0:207/282.04 KB 7:243/331.30 KB 3:269/366.04 KB 5:249/339.74 KB 6:260/354.46 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 136.55 KB, 740.07 KB, 1.29 MB, 2.25 MB - FooterProcessingTime: (Avg: 14.258ms ; Min: 5.169ms ; Max: 214.735ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713166) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713166) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713166) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 112.24 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s793ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.06 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 7 (7) - ScannerThreadsTotalWallClockTime: 2m40s - MaterializeTupleTime(*): 2.279ms - ScannerThreadsSysTime: 53.967ms - ScannerThreadsUserTime: 322.916ms - ScannerThreadsVoluntaryContextSwitches: 7.14K (7136) - TotalRawHdfsReadTime(*): 23.053ms - TotalReadThroughput: 1.12 MB/sec Instance c34cc30188b6a8c9:37 (host=vd1126.foo.fa.com:22000):(Total: 2s183ms, non-child: 406.292ms, % non-child: 18.60%) Hdfs split stats (:<# splits>/): 4:252/343.78 KB 5:227/309.51 KB 2:265/361.35 KB 6:240/327.38 KB 1:253/344.49 KB 7:225/306.94 KB 0:246/335.30 KB 3:236/321.26 KB MemoryUsage(500.000ms): 32.36 KB, 184.36 KB, 208.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 403.127us - TotalStorageWaitTime: 26s537ms - TotalThreadsInvoluntaryContextSwitches: 25 (25) - TotalThreadsTotalWallClockTime: 2m34s - TotalThreadsSysTime: 69.967ms - TotalThreadsUserTime: 462.896ms - TotalThreadsVoluntaryContextSwitches: 9.26K (9259) Fragment Instance Lifecycle Timings: - ExecTime: 596.266us - ExecTreeExecTime: 5.228us - OpenTime: 1s792ms - ExecTreeOpenTime: 1s746ms - PrepareTime: 390.436ms - ExecTreePrepareTime: 6.695ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 87.602us, non-child: 87.602us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 230.77 KB/sec - OverallThroughput: 178.36 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.743us - TransmitDataRPCTime: 67.709us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.183ms, non-child: 24.183ms, % non-child: 100.00%) - CodegenTime: 606.924us - CompileTime: 4.022ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.558ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.232ms AGGREGATION_NODE (id=1):(Total: 1s753ms, non-child: 6.434ms, % non-child: 0.37%) ExecOption: Codegen Enabled - BuildTime: 1.579ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s746ms, non-child: 1s746ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:252/343.78 KB 5:227/309.51 KB 2:265/361.35 KB 6:240/327.38 KB 1:253/344.49 KB 7:225/306.94 KB 0:246/335.30 KB 3:236/321.26 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 0, 794.48 KB, 1.70 MB, 2.27 MB - FooterProcessingTime: (Avg: 13.726ms ; Min: 5.735ms ; Max: 203.367ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713621) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713621) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713621) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.84 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s699ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.11 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 6 (6) - ScannerThreadsTotalWallClockTime: 2m32s - MaterializeTupleTime(*): 2.509ms - ScannerThreadsSysTime: 38.971ms - ScannerThreadsUserTime: 356.912ms - ScannerThreadsVoluntaryContextSwitches: 7.14K (7144) - TotalRawHdfsReadTime(*): 25.164ms - TotalReadThroughput: 1.14 MB/sec Instance c34cc30188b6a8c9:7b (host=vhs031.foo.fa.com:22000):(Total: 2s176ms, non-child: 336.742ms, % non-child: 15.47%) Hdfs split stats (:<# splits>/): 2:255/347.83 KB 3:232/316.33 KB 0:241/328.12 KB 5:238/324.25 KB 7:247/336.46 KB 4:224/305.02 KB 1:242/329.37 KB 6:266/362.99 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB, 130.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 329.662us - TotalStorageWaitTime: 27s385ms - TotalThreadsInvoluntaryContextSwitches: 30 (30) - TotalThreadsTotalWallClockTime: 2m39s - TotalThreadsSysTime: 66.972ms - TotalThreadsUserTime: 436.898ms - TotalThreadsVoluntaryContextSwitches: 9.34K (9340) Fragment Instance Lifecycle Timings: - ExecTime: 494.982us - ExecTreeExecTime: 4.690us - OpenTime: 1s854ms - ExecTreeOpenTime: 1s808ms - PrepareTime: 320.977ms - ExecTreePrepareTime: 6.585ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 68.992us, non-child: 68.992us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 301.40 KB/sec - OverallThroughput: 226.47 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.598us - TransmitDataRPCTime: 51.842us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.099ms, non-child: 24.099ms, % non-child: 100.00%) - CodegenTime: 543.022us - CompileTime: 3.976ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.500ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.251ms AGGREGATION_NODE (id=1):(Total: 1s815ms, non-child: 5.612ms, % non-child: 0.31%) ExecOption: Codegen Enabled - BuildTime: 1.698ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s809ms, non-child: 1s809ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:255/347.83 KB 3:232/316.33 KB 0:241/328.12 KB 5:238/324.25 KB 7:247/336.46 KB 4:224/305.02 KB 1:242/329.37 KB 6:266/362.99 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 336.66 KB, 972.30 KB, 1.72 MB, 2.50 MB - FooterProcessingTime: (Avg: 14.149ms ; Min: 6.091ms ; Max: 202.418ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713972) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713972) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713972) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 108.56 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s764ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.07 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 6 (6) - ScannerThreadsTotalWallClockTime: 2m37s - MaterializeTupleTime(*): 2.398ms - ScannerThreadsSysTime: 32.977ms - ScannerThreadsUserTime: 335.914ms - ScannerThreadsVoluntaryContextSwitches: 7.25K (7249) - TotalRawHdfsReadTime(*): 23.841ms - TotalReadThroughput: 1.25 MB/sec Instance c34cc30188b6a8c9:6d (host=ve1134.foo.fa.com:22000):(Total: 2s161ms, non-child: 484.543ms, % non-child: 22.41%) Hdfs split stats (:<# splits>/): 6:263/358.08 KB 0:244/332.47 KB 2:231/314.94 KB 1:247/336.21 KB 7:254/345.83 KB 3:230/313.47 KB 4:227/309.60 KB 5:249/339.34 KB MemoryUsage(500.000ms): 0, 194.36 KB, 194.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 226.36 KB (231790) - PerHostPeakMemUsage: 226.36 KB (231790) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 463.632us - TotalStorageWaitTime: 24s467ms - TotalThreadsInvoluntaryContextSwitches: 39 (39) - TotalThreadsTotalWallClockTime: 2m11s - TotalThreadsSysTime: 77.962ms - TotalThreadsUserTime: 423.902ms - TotalThreadsVoluntaryContextSwitches: 9.43K (9429) Fragment Instance Lifecycle Timings: - ExecTime: 653.565us - ExecTreeExecTime: 5.870us - OpenTime: 1s712ms - ExecTreeOpenTime: 1s642ms - PrepareTime: 448.318ms - ExecTreePrepareTime: 8.951ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 88.356us, non-child: 88.356us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 284.30 KB/sec - OverallThroughput: 176.84 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.130us - TransmitDataRPCTime: 54.959us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.898ms, non-child: 25.898ms, % non-child: 100.00%) - CodegenTime: 750.794us - CompileTime: 4.254ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 7.098ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.142ms AGGREGATION_NODE (id=1):(Total: 1s651ms, non-child: 1.656ms, % non-child: 0.10%) ExecOption: Codegen Enabled - BuildTime: 1.174ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s649ms, non-child: 1s649ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:263/358.08 KB 0:244/332.47 KB 2:231/314.94 KB 1:247/336.21 KB 7:254/345.83 KB 3:230/313.47 KB 4:227/309.60 KB 5:249/339.34 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 681.07 KB, 1.57 MB, 2.25 MB - FooterProcessingTime: (Avg: 12.642ms ; Min: 5.239ms ; Max: 32.139ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2713538) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713538) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713538) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 186.00 KB (190464) - PerReadThreadRawHdfsThroughput: 108.83 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s598ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.18 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m9s - MaterializeTupleTime(*): 2.032ms - ScannerThreadsSysTime: 47.967ms - ScannerThreadsUserTime: 298.921ms - ScannerThreadsVoluntaryContextSwitches: 7.35K (7353) - TotalRawHdfsReadTime(*): 23.779ms - TotalReadThroughput: 1.50 MB/sec Instance c34cc30188b6a8c9:4c (host=ve1140.foo.fa.com:22000):(Total: 2s149ms, non-child: 342.711ms, % non-child: 15.95%) Hdfs split stats (:<# splits>/): 2:238/324.46 KB 7:251/342.29 KB 5:232/316.48 KB 6:253/344.76 KB 0:224/305.24 KB 4:252/343.57 KB 1:267/363.95 KB 3:228/310.26 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 399.599us - TotalStorageWaitTime: 26s418ms - TotalThreadsInvoluntaryContextSwitches: 30 (30) - TotalThreadsTotalWallClockTime: 2m22s - TotalThreadsSysTime: 61.969ms - TotalThreadsUserTime: 462.903ms - TotalThreadsVoluntaryContextSwitches: 9.12K (9117) Fragment Instance Lifecycle Timings: - ExecTime: 607.004us - ExecTreeExecTime: 14.720us - OpenTime: 1s821ms - ExecTreeOpenTime: 1s774ms - PrepareTime: 327.109ms - ExecTreePrepareTime: 6.945ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 93.880us, non-child: 93.880us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 267.36 KB/sec - OverallThroughput: 166.44 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 22.772us - TransmitDataRPCTime: 58.442us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.944ms, non-child: 24.944ms, % non-child: 100.00%) - CodegenTime: 567.778us - CompileTime: 4.511ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.754ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.271ms AGGREGATION_NODE (id=1):(Total: 1s781ms, non-child: 5.274ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.639ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s776ms, non-child: 1s776ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:238/324.46 KB 7:251/342.29 KB 5:232/316.48 KB 6:253/344.76 KB 0:224/305.24 KB 4:252/343.57 KB 1:267/363.95 KB 3:228/310.26 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 525.46 KB, 1.47 MB, 2.08 MB - FooterProcessingTime: (Avg: 13.653ms ; Min: 5.666ms ; Max: 31.789ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2714643) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714643) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714643) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 200.00 KB (204800) - PerReadThreadRawHdfsThroughput: 109.64 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s729ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.09 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m20s - MaterializeTupleTime(*): 2.399ms - ScannerThreadsSysTime: 41.972ms - ScannerThreadsUserTime: 348.920ms - ScannerThreadsVoluntaryContextSwitches: 7.03K (7028) - TotalRawHdfsReadTime(*): 23.611ms - TotalReadThroughput: 1.38 MB/sec Instance c34cc30188b6a8c9:72 (host=vc1515.foo.fa.com:22000):(Total: 2s148ms, non-child: 418.337ms, % non-child: 19.47%) Hdfs split stats (:<# splits>/): 5:229/311.66 KB 4:244/332.78 KB 6:244/332.40 KB 2:240/327.28 KB 0:241/328.39 KB 7:244/332.72 KB 1:255/347.96 KB 3:247/336.74 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB, 166.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 480.045us - TotalStorageWaitTime: 26s282ms - TotalThreadsInvoluntaryContextSwitches: 22 (22) - TotalThreadsTotalWallClockTime: 2m30s - TotalThreadsSysTime: 83.964ms - TotalThreadsUserTime: 415.900ms - TotalThreadsVoluntaryContextSwitches: 9.24K (9239) Fragment Instance Lifecycle Timings: - ExecTime: 679.439us - ExecTreeExecTime: 6.241us - OpenTime: 1s746ms - ExecTreeOpenTime: 1s698ms - PrepareTime: 401.706ms - ExecTreePrepareTime: 6.974ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 91.068us, non-child: 91.068us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 330.72 KB/sec - OverallThroughput: 171.57 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.409us - TransmitDataRPCTime: 47.246us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.566ms, non-child: 24.566ms, % non-child: 100.00%) - CodegenTime: 601.184us - CompileTime: 4.010ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.614ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.584ms AGGREGATION_NODE (id=1):(Total: 1s705ms, non-child: 5.322ms, % non-child: 0.31%) ExecOption: Codegen Enabled - BuildTime: 1.729ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s700ms, non-child: 1s700ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:229/311.66 KB 4:244/332.78 KB 6:244/332.40 KB 2:240/327.28 KB 0:241/328.39 KB 7:244/332.72 KB 1:255/347.96 KB 3:247/336.74 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 240.95 KB, 1.02 MB, 1.71 MB, 2.52 MB - FooterProcessingTime: (Avg: 13.586ms ; Min: 5.718ms ; Max: 30.527ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713523) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713523) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713523) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.75 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s655ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.14 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m28s - MaterializeTupleTime(*): 2.262ms - ScannerThreadsSysTime: 50.969ms - ScannerThreadsUserTime: 317.915ms - ScannerThreadsVoluntaryContextSwitches: 7.16K (7158) - TotalRawHdfsReadTime(*): 25.185ms - TotalReadThroughput: 1.26 MB/sec Instance c34cc30188b6a8c9:36 (host=vhs030.foo.fa.com:22000):(Total: 2s144ms, non-child: 348.183ms, % non-child: 16.24%) Hdfs split stats (:<# splits>/): 0:246/335.04 KB 6:274/373.01 KB 3:251/341.98 KB 2:247/336.86 KB 5:215/292.91 KB 4:248/338.22 KB 7:229/312.34 KB 1:236/321.19 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 194.36 KB, 130.36 KB ThreadUsage(500.000ms): 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 433.853us - TotalStorageWaitTime: 25s262ms - TotalThreadsInvoluntaryContextSwitches: 20 (20) - TotalThreadsTotalWallClockTime: 2m36s - TotalThreadsSysTime: 66.972ms - TotalThreadsUserTime: 423.899ms - TotalThreadsVoluntaryContextSwitches: 9.57K (9575) Fragment Instance Lifecycle Timings: - ExecTime: 718.306us - ExecTreeExecTime: 6.368us - OpenTime: 1s812ms - ExecTreeOpenTime: 1s765ms - PrepareTime: 330.919ms - ExecTreePrepareTime: 6.616ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 85.653us, non-child: 85.653us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 277.71 KB/sec - OverallThroughput: 182.42 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.215us - TransmitDataRPCTime: 56.263us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.034ms, non-child: 24.034ms, % non-child: 100.00%) - CodegenTime: 570.727us - CompileTime: 4.171ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.642ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 12.848ms AGGREGATION_NODE (id=1):(Total: 1s772ms, non-child: 6.017ms, % non-child: 0.34%) ExecOption: Codegen Enabled - BuildTime: 1.651ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s766ms, non-child: 1s766ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:246/335.04 KB 6:274/373.01 KB 3:251/341.98 KB 2:247/336.86 KB 5:215/292.91 KB 4:248/338.22 KB 7:229/312.34 KB 1:236/321.19 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1946 out of 1946 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1946 BytesRead(500.000ms): 72.10 KB, 950.01 KB, 1.84 MB, 2.49 MB - FooterProcessingTime: (Avg: 13.049ms ; Min: 5.686ms ; Max: 205.416ms ; Number of samples: 1946) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2715189) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2715189) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2715189) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.41 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s720ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1946) - RowsReturned: 1.95K (1946) - RowsReturnedRate: 1.10 K/sec - ScanRangesComplete: 1.95K (1946) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m34s - MaterializeTupleTime(*): 2.304ms - ScannerThreadsSysTime: 37.977ms - ScannerThreadsUserTime: 321.914ms - ScannerThreadsVoluntaryContextSwitches: 7.50K (7502) - TotalRawHdfsReadTime(*): 25.284ms - TotalReadThroughput: 1.24 MB/sec Instance c34cc30188b6a8c9:5d (host=vd1102.foo.fa.com:22000):(Total: 2s141ms, non-child: 416.204ms, % non-child: 19.44%) Hdfs split stats (:<# splits>/): 4:238/324.38 KB 7:230/313.46 KB 0:221/301.09 KB 6:257/350.09 KB 1:241/328.39 KB 2:249/340.02 KB 3:257/350.43 KB 5:251/342.15 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 300.843us - TotalStorageWaitTime: 26s053ms - TotalThreadsInvoluntaryContextSwitches: 16 (16) - TotalThreadsTotalWallClockTime: 2m29s - TotalThreadsSysTime: 70.967ms - TotalThreadsUserTime: 434.896ms - TotalThreadsVoluntaryContextSwitches: 9.46K (9461) Fragment Instance Lifecycle Timings: - ExecTime: 483.348us - ExecTreeExecTime: 5.953us - OpenTime: 1s739ms - ExecTreeOpenTime: 1s692ms - PrepareTime: 400.848ms - ExecTreePrepareTime: 7.207ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 80.411us, non-child: 80.411us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 317.52 KB/sec - OverallThroughput: 194.31 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.394us - TransmitDataRPCTime: 49.210us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.495ms, non-child: 25.495ms, % non-child: 100.00%) - CodegenTime: 601.733us - CompileTime: 4.191ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.823ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.080ms AGGREGATION_NODE (id=1):(Total: 1s699ms, non-child: 4.491ms, % non-child: 0.26%) ExecOption: Codegen Enabled - BuildTime: 1.636ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s694ms, non-child: 1s694ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:238/324.38 KB 7:230/313.46 KB 0:221/301.09 KB 6:257/350.09 KB 1:241/328.39 KB 2:249/340.02 KB 3:257/350.43 KB 5:251/342.15 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 0, 402.20 KB, 1.26 MB, 2.26 MB - FooterProcessingTime: (Avg: 13.469ms ; Min: 5.605ms ; Max: 231.341ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713599) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713599) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713599) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 103.16 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s646ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.15 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 2m27s - MaterializeTupleTime(*): 2.255ms - ScannerThreadsSysTime: 42.972ms - ScannerThreadsUserTime: 326.913ms - ScannerThreadsVoluntaryContextSwitches: 7.40K (7400) - TotalRawHdfsReadTime(*): 25.085ms - TotalReadThroughput: 1.13 MB/sec Instance c34cc30188b6a8c9:70 (host=vc1538.foo.fa.com:22000):(Total: 2s123ms, non-child: 418.565ms, % non-child: 19.71%) Hdfs split stats (:<# splits>/): 2:240/327.41 KB 1:236/321.25 KB 0:237/323.20 KB 5:239/325.44 KB 6:248/338.01 KB 3:237/323.23 KB 4:256/349.01 KB 7:253/344.94 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 502.113us - TotalStorageWaitTime: 25s727ms - TotalThreadsInvoluntaryContextSwitches: 25 (25) - TotalThreadsTotalWallClockTime: 2m27s - TotalThreadsSysTime: 78.964ms - TotalThreadsUserTime: 455.898ms - TotalThreadsVoluntaryContextSwitches: 9.47K (9473) Fragment Instance Lifecycle Timings: - ExecTime: 683.012us - ExecTreeExecTime: 5.279us - OpenTime: 1s720ms - ExecTreeOpenTime: 1s671ms - PrepareTime: 402.657ms - ExecTreePrepareTime: 7.225ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 129.851us, non-child: 129.851us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 142.45 KB/sec - OverallThroughput: 120.33 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.582us - TransmitDataRPCTime: 109.690us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.813ms, non-child: 25.813ms, % non-child: 100.00%) - CodegenTime: 576.781us - CompileTime: 4.196ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.902ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.318ms AGGREGATION_NODE (id=1):(Total: 1s678ms, non-child: 7.070ms, % non-child: 0.42%) ExecOption: Codegen Enabled - BuildTime: 1.644ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s671ms, non-child: 1s671ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:240/327.41 KB 1:236/321.25 KB 0:237/323.20 KB 5:239/325.44 KB 6:248/338.01 KB 3:237/323.23 KB 4:256/349.01 KB 7:253/344.94 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1946 out of 1946 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1946 BytesRead(500.000ms): 526.62 KB, 1.38 MB, 2.06 MB - FooterProcessingTime: (Avg: 13.292ms ; Min: 5.520ms ; Max: 36.157ms ; Number of samples: 1946) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2716147) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2716147) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2716147) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 97.56 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s623ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1946) - RowsReturned: 1.95K (1946) - RowsReturnedRate: 1.16 K/sec - ScanRangesComplete: 1.95K (1946) - ScannerThreadsInvoluntaryContextSwitches: 7 (7) - ScannerThreadsTotalWallClockTime: 2m25s - MaterializeTupleTime(*): 2.516ms - ScannerThreadsSysTime: 48.969ms - ScannerThreadsUserTime: 345.914ms - ScannerThreadsVoluntaryContextSwitches: 7.32K (7323) - TotalRawHdfsReadTime(*): 26.552ms - TotalReadThroughput: 1.37 MB/sec Instance c34cc30188b6a8c9:2b (host=vc1521.foo.fa.com:22000):(Total: 2s117ms, non-child: 417.439ms, % non-child: 19.72%) Hdfs split stats (:<# splits>/): 2:232/316.73 KB 1:244/332.60 KB 5:241/328.04 KB 4:241/328.48 KB 0:233/317.04 KB 6:260/354.45 KB 3:242/330.04 KB 7:251/342.68 KB MemoryUsage(500.000ms): 0, 194.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 540.926us - TotalStorageWaitTime: 25s369ms - TotalThreadsInvoluntaryContextSwitches: 35 (35) - TotalThreadsTotalWallClockTime: 2m27s - TotalThreadsSysTime: 90.958ms - TotalThreadsUserTime: 443.898ms - TotalThreadsVoluntaryContextSwitches: 9.15K (9151) Fragment Instance Lifecycle Timings: - ExecTime: 711.125us - ExecTreeExecTime: 5.215us - OpenTime: 1s715ms - ExecTreeOpenTime: 1s667ms - PrepareTime: 401.317ms - ExecTreePrepareTime: 7.014ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 102.710us, non-child: 102.710us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 206.66 KB/sec - OverallThroughput: 152.13 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.628us - TransmitDataRPCTime: 75.609us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.838ms, non-child: 24.838ms, % non-child: 100.00%) - CodegenTime: 599.478us - CompileTime: 4.165ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.775ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.506ms AGGREGATION_NODE (id=1):(Total: 1s674ms, non-child: 4.807ms, % non-child: 0.29%) ExecOption: Codegen Enabled - BuildTime: 1.658ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s669ms, non-child: 1s669ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:232/316.73 KB 1:244/332.60 KB 5:241/328.04 KB 4:241/328.48 KB 0:233/317.04 KB 6:260/354.45 KB 3:242/330.04 KB 7:251/342.68 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 320.22 KB, 1.20 MB, 2.14 MB - FooterProcessingTime: (Avg: 13.118ms ; Min: 6.174ms ; Max: 207.768ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713652) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713652) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713652) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.52 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s622ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.16 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 14 (14) - ScannerThreadsTotalWallClockTime: 2m25s - MaterializeTupleTime(*): 2.399ms - ScannerThreadsSysTime: 61.962ms - ScannerThreadsUserTime: 339.914ms - ScannerThreadsVoluntaryContextSwitches: 7.06K (7063) - TotalRawHdfsReadTime(*): 25.242ms - TotalReadThroughput: 1.43 MB/sec Instance c34cc30188b6a8c9:29 (host=vd1130.foo.fa.com:22000):(Total: 2s105ms, non-child: 420.123ms, % non-child: 19.95%) Hdfs split stats (:<# splits>/): 7:250/340.71 KB 6:229/311.99 KB 0:256/349.08 KB 2:234/318.85 KB 3:227/309.28 KB 5:259/352.73 KB 4:249/339.31 KB 1:241/328.71 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 258.36 KB (264558) - PerHostPeakMemUsage: 258.36 KB (264558) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 505.065us - TotalStorageWaitTime: 25s266ms - TotalThreadsInvoluntaryContextSwitches: 14 (14) - TotalThreadsTotalWallClockTime: 2m12s - TotalThreadsSysTime: 81.965ms - TotalThreadsUserTime: 389.907ms - TotalThreadsVoluntaryContextSwitches: 9.68K (9681) Fragment Instance Lifecycle Timings: - ExecTime: 768.311us - ExecTreeExecTime: 4.965us - OpenTime: 1s702ms - ExecTreeOpenTime: 1s654ms - PrepareTime: 403.007ms - ExecTreePrepareTime: 6.938ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 106.453us, non-child: 106.453us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 180.63 KB/sec - OverallThroughput: 146.78 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.750us - TransmitDataRPCTime: 86.503us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.525ms, non-child: 24.525ms, % non-child: 100.00%) - CodegenTime: 564.693us - CompileTime: 3.967ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.670ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.512ms AGGREGATION_NODE (id=1):(Total: 1s661ms, non-child: 4.594ms, % non-child: 0.28%) ExecOption: Codegen Enabled - BuildTime: 1.419ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s656ms, non-child: 1s656ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:250/340.71 KB 6:229/311.99 KB 0:256/349.08 KB 2:234/318.85 KB 3:227/309.28 KB 5:259/352.73 KB 4:249/339.31 KB 1:241/328.71 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 420.91 KB, 1007.05 KB, 1.86 MB - FooterProcessingTime: (Avg: 13.052ms ; Min: 6.574ms ; Max: 207.519ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2714271) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714271) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714271) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 218.00 KB (223232) - PerReadThreadRawHdfsThroughput: 107.53 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s611ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.17 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m11s - MaterializeTupleTime(*): 2.160ms - ScannerThreadsSysTime: 47.970ms - ScannerThreadsUserTime: 293.922ms - ScannerThreadsVoluntaryContextSwitches: 7.58K (7577) - TotalRawHdfsReadTime(*): 24.073ms - TotalReadThroughput: 1.24 MB/sec Instance c34cc30188b6a8c9:6b (host=vc1504.foo.fa.com:22000):(Total: 2s103ms, non-child: 341.420ms, % non-child: 16.23%) Hdfs split stats (:<# splits>/): 5:225/306.88 KB 6:222/302.75 KB 2:257/350.54 KB 3:252/342.97 KB 1:239/326.08 KB 4:258/351.82 KB 7:236/321.34 KB 0:254/346.33 KB MemoryUsage(500.000ms): 254.36 KB, 166.36 KB, 182.36 KB, 176.36 KB ThreadUsage(500.000ms): 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 288.36 KB (295278) - PerHostPeakMemUsage: 288.36 KB (295278) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 455.458us - TotalStorageWaitTime: 27s547ms - TotalThreadsInvoluntaryContextSwitches: 22 (22) - TotalThreadsTotalWallClockTime: 2m32s - TotalThreadsSysTime: 88.958ms - TotalThreadsUserTime: 442.898ms - TotalThreadsVoluntaryContextSwitches: 9.26K (9259) Fragment Instance Lifecycle Timings: - ExecTime: 667.727us - ExecTreeExecTime: 6.067us - OpenTime: 1s776ms - ExecTreeOpenTime: 1s729ms - PrepareTime: 326.364ms - ExecTreePrepareTime: 6.716ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 111.381us, non-child: 111.381us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 230.04 KB/sec - OverallThroughput: 140.28 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.783us - TransmitDataRPCTime: 67.923us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.851ms, non-child: 25.851ms, % non-child: 100.00%) - CodegenTime: 553.121us - CompileTime: 4.113ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.851ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.497ms AGGREGATION_NODE (id=1):(Total: 1s735ms, non-child: 5.807ms, % non-child: 0.33%) ExecOption: Codegen Enabled - BuildTime: 1.691ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s729ms, non-child: 1s729ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:225/306.88 KB 6:222/302.75 KB 2:257/350.54 KB 3:252/342.97 KB 1:239/326.08 KB 4:258/351.82 KB 7:236/321.34 KB 0:254/346.33 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 0, 660.88 KB, 1.16 MB, 2.20 MB - FooterProcessingTime: (Avg: 14.249ms ; Min: 5.104ms ; Max: 209.344ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2712280) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712280) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712280) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 248.00 KB (253952) - PerReadThreadRawHdfsThroughput: 108.12 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s682ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.12 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 12 (12) - ScannerThreadsTotalWallClockTime: 2m30s - MaterializeTupleTime(*): 2.413ms - ScannerThreadsSysTime: 48.964ms - ScannerThreadsUserTime: 350.912ms - ScannerThreadsVoluntaryContextSwitches: 7.17K (7170) - TotalRawHdfsReadTime(*): 23.922ms - TotalReadThroughput: 1.10 MB/sec Instance c34cc30188b6a8c9:73 (host=vc1528.foo.fa.com:22000):(Total: 2s095ms, non-child: 337.986ms, % non-child: 16.13%) Hdfs split stats (:<# splits>/): 0:236/321.43 KB 7:253/344.92 KB 6:240/327.19 KB 1:253/344.80 KB 3:236/322.12 KB 2:238/323.95 KB 4:242/329.69 KB 5:247/336.29 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 416.583us - TotalStorageWaitTime: 26s715ms - TotalThreadsInvoluntaryContextSwitches: 41 (41) - TotalThreadsTotalWallClockTime: 2m32s - TotalThreadsSysTime: 88.965ms - TotalThreadsUserTime: 405.901ms - TotalThreadsVoluntaryContextSwitches: 9.38K (9376) Fragment Instance Lifecycle Timings: - ExecTime: 663.932us - ExecTreeExecTime: 5.403us - OpenTime: 1s772ms - ExecTreeOpenTime: 1s727ms - PrepareTime: 322.189ms - ExecTreePrepareTime: 6.562ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 77.222us, non-child: 77.222us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 287.76 KB/sec - OverallThroughput: 202.34 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.316us - TransmitDataRPCTime: 54.299us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.921ms, non-child: 23.921ms, % non-child: 100.00%) - CodegenTime: 565.461us - CompileTime: 4.101ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.536ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 12.904ms AGGREGATION_NODE (id=1):(Total: 1s733ms, non-child: 4.619ms, % non-child: 0.27%) ExecOption: Codegen Enabled - BuildTime: 1.462ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s729ms, non-child: 1s729ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:236/321.43 KB 7:253/344.92 KB 6:240/327.19 KB 1:253/344.80 KB 3:236/322.12 KB 2:238/323.95 KB 4:242/329.69 KB 5:247/336.29 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:66.67% 1:33.33% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 291.96 KB, 989.29 KB, 1.72 MB - FooterProcessingTime: (Avg: 13.799ms ; Min: 5.749ms ; Max: 35.371ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.33 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714005) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714005) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714005) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 105.98 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s684ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.12 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 6 (6) - ScannerThreadsTotalWallClockTime: 2m30s - MaterializeTupleTime(*): 2.258ms - ScannerThreadsSysTime: 56.970ms - ScannerThreadsUserTime: 299.918ms - ScannerThreadsVoluntaryContextSwitches: 7.33K (7325) - TotalRawHdfsReadTime(*): 24.423ms - TotalReadThroughput: 1.14 MB/sec Instance c34cc30188b6a8c9:79 (host=vc1523.foo.fa.com:22000):(Total: 2s094ms, non-child: 333.948ms, % non-child: 15.94%) Hdfs split stats (:<# splits>/): 2:253/345.49 KB 6:237/322.92 KB 7:227/309.12 KB 0:231/314.74 KB 1:235/320.24 KB 3:261/355.62 KB 5:256/348.93 KB 4:244/332.50 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 194.36 KB, 184.36 KB ThreadUsage(500.000ms): 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 448.264us - TotalStorageWaitTime: 26s424ms - TotalThreadsInvoluntaryContextSwitches: 23 (23) - TotalThreadsTotalWallClockTime: 2m32s - TotalThreadsSysTime: 82.960ms - TotalThreadsUserTime: 397.901ms - TotalThreadsVoluntaryContextSwitches: 9.48K (9480) Fragment Instance Lifecycle Timings: - ExecTime: 637.841us - ExecTreeExecTime: 5.793us - OpenTime: 1s773ms - ExecTreeOpenTime: 1s729ms - PrepareTime: 320.247ms - ExecTreePrepareTime: 6.863ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 83.813us, non-child: 83.813us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 244.84 KB/sec - OverallThroughput: 186.43 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.382us - TransmitDataRPCTime: 63.817us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.653ms, non-child: 24.653ms, % non-child: 100.00%) - CodegenTime: 544.682us - CompileTime: 3.902ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.276ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.128ms AGGREGATION_NODE (id=1):(Total: 1s736ms, non-child: 5.143ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.594ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s730ms, non-child: 1s730ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:253/345.49 KB 6:237/322.92 KB 7:227/309.12 KB 0:231/314.74 KB 1:235/320.24 KB 3:261/355.62 KB 5:256/348.93 KB 4:244/332.50 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 105.09 KB, 772.72 KB, 1.64 MB, 2.37 MB - FooterProcessingTime: (Avg: 13.658ms ; Min: 4.785ms ; Max: 201.931ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713165) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713165) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713165) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 107.40 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s686ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.12 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 2m31s - MaterializeTupleTime(*): 2.326ms - ScannerThreadsSysTime: 60.963ms - ScannerThreadsUserTime: 290.917ms - ScannerThreadsVoluntaryContextSwitches: 7.39K (7391) - TotalRawHdfsReadTime(*): 24.091ms - TotalReadThroughput: 1.19 MB/sec Instance c34cc30188b6a8c9:6f (host=vhs042.foo.fa.com:22000):(Total: 2s089ms, non-child: 340.633ms, % non-child: 16.30%) Hdfs split stats (:<# splits>/): 6:249/339.67 KB 3:259/353.12 KB 4:237/322.98 KB 5:261/355.55 KB 7:236/321.56 KB 1:240/327.77 KB 2:226/308.69 KB 0:233/317.61 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB, 122.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 432.811us - TotalStorageWaitTime: 26s070ms - TotalThreadsInvoluntaryContextSwitches: 25 (25) - TotalThreadsTotalWallClockTime: 2m31s - TotalThreadsSysTime: 86.961ms - TotalThreadsUserTime: 440.897ms - TotalThreadsVoluntaryContextSwitches: 9.30K (9304) Fragment Instance Lifecycle Timings: - ExecTime: 677.740us - ExecTreeExecTime: 6.832us - OpenTime: 1s764ms - ExecTreeOpenTime: 1s717ms - PrepareTime: 324.089ms - ExecTreePrepareTime: 6.585ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 78.565us, non-child: 78.565us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 265.24 KB/sec - OverallThroughput: 198.88 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.362us - TransmitDataRPCTime: 58.909us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.451ms, non-child: 24.451ms, % non-child: 100.00%) - CodegenTime: 577.787us - CompileTime: 4.098ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.631ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.364ms AGGREGATION_NODE (id=1):(Total: 1s724ms, non-child: 6.392ms, % non-child: 0.37%) ExecOption: Codegen Enabled - BuildTime: 1.723ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s718ms, non-child: 1s718ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:249/339.67 KB 3:259/353.12 KB 4:237/322.98 KB 5:261/355.55 KB 7:236/321.56 KB 1:240/327.77 KB 2:226/308.69 KB 0:233/317.61 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1941 out of 1941 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1941 BytesRead(500.000ms): 207.23 KB, 1.05 MB, 1.73 MB, 2.50 MB - FooterProcessingTime: (Avg: 13.501ms ; Min: 5.650ms ; Max: 30.653ms ; Number of samples: 1941) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.58 MB (2710475) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.58 MB (2710475) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.58 MB (2710475) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 200.00 KB (204800) - PerReadThreadRawHdfsThroughput: 106.97 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s671ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1941) - RowsReturned: 1.94K (1941) - RowsReturnedRate: 1.13 K/sec - ScanRangesComplete: 1.94K (1941) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 2m29s - MaterializeTupleTime(*): 2.331ms - ScannerThreadsSysTime: 54.966ms - ScannerThreadsUserTime: 327.914ms - ScannerThreadsVoluntaryContextSwitches: 7.22K (7222) - TotalRawHdfsReadTime(*): 24.164ms - TotalReadThroughput: 1.25 MB/sec Instance c34cc30188b6a8c9:27 (host=vd1112.foo.fa.com:22000):(Total: 2s087ms, non-child: 415.968ms, % non-child: 19.92%) Hdfs split stats (:<# splits>/): 1:262/357.23 KB 4:249/339.46 KB 6:242/329.88 KB 0:258/352.25 KB 2:224/305.19 KB 5:235/320.47 KB 7:234/319.37 KB 3:240/327.21 KB MemoryUsage(500.000ms): 0, 194.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 260.36 KB (266606) - PerHostPeakMemUsage: 260.36 KB (266606) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 625.936us - TotalStorageWaitTime: 24s769ms - TotalThreadsInvoluntaryContextSwitches: 20 (20) - TotalThreadsTotalWallClockTime: 2m24s - TotalThreadsSysTime: 59.966ms - TotalThreadsUserTime: 437.896ms - TotalThreadsVoluntaryContextSwitches: 9.57K (9570) Fragment Instance Lifecycle Timings: - ExecTime: 870.824us - ExecTreeExecTime: 5.296us - OpenTime: 1s686ms - ExecTreeOpenTime: 1s639ms - PrepareTime: 399.990ms - ExecTreePrepareTime: 7.139ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 88.610us, non-child: 88.610us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 251.61 KB/sec - OverallThroughput: 176.33 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 18.128us - TransmitDataRPCTime: 62.100us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.624ms, non-child: 25.624ms, % non-child: 100.00%) - CodegenTime: 617.388us - CompileTime: 4.087ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.863ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.260ms AGGREGATION_NODE (id=1):(Total: 1s646ms, non-child: 5.561ms, % non-child: 0.34%) ExecOption: Codegen Enabled - BuildTime: 1.723ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s640ms, non-child: 1s640ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:262/357.23 KB 4:249/339.46 KB 6:242/329.88 KB 0:258/352.25 KB 2:224/305.19 KB 5:235/320.47 KB 7:234/319.37 KB 3:240/327.21 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 399.58 KB, 1.34 MB, 1.96 MB - FooterProcessingTime: (Avg: 12.811ms ; Min: 6.032ms ; Max: 207.642ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714672) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714672) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714672) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 220.00 KB (225280) - PerReadThreadRawHdfsThroughput: 99.11 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s592ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.18 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 5 (5) - ScannerThreadsTotalWallClockTime: 2m22s - MaterializeTupleTime(*): 2.435ms - ScannerThreadsSysTime: 41.969ms - ScannerThreadsUserTime: 329.913ms - ScannerThreadsVoluntaryContextSwitches: 7.47K (7470) - TotalRawHdfsReadTime(*): 26.122ms - TotalReadThroughput: 1.31 MB/sec Instance c34cc30188b6a8c9:39 (host=vd1110.foo.fa.com:22000):(Total: 2s068ms, non-child: 417.187ms, % non-child: 20.17%) Hdfs split stats (:<# splits>/): 4:248/338.31 KB 0:241/328.70 KB 7:237/323.15 KB 3:243/330.96 KB 2:256/348.96 KB 6:231/314.67 KB 5:265/361.49 KB 1:222/302.28 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 262.36 KB (268654) - PerHostPeakMemUsage: 262.36 KB (268654) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 518.619us - TotalStorageWaitTime: 24s506ms - TotalThreadsInvoluntaryContextSwitches: 21 (21) - TotalThreadsTotalWallClockTime: 2m22s - TotalThreadsSysTime: 80.963ms - TotalThreadsUserTime: 408.901ms - TotalThreadsVoluntaryContextSwitches: 9.60K (9597) Fragment Instance Lifecycle Timings: - ExecTime: 739.976us - ExecTreeExecTime: 4.634us - OpenTime: 1s667ms - ExecTreeOpenTime: 1s618ms - PrepareTime: 399.848ms - ExecTreePrepareTime: 7.012ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 84.944us, non-child: 84.944us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 235.14 KB/sec - OverallThroughput: 183.94 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.687us - TransmitDataRPCTime: 66.450us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.288ms, non-child: 25.288ms, % non-child: 100.00%) - CodegenTime: 601.117us - CompileTime: 4.131ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.887ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.839ms AGGREGATION_NODE (id=1):(Total: 1s625ms, non-child: 4.560ms, % non-child: 0.28%) ExecOption: Codegen Enabled - BuildTime: 1.639ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s621ms, non-child: 1s621ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:248/338.31 KB 0:241/328.70 KB 7:237/323.15 KB 3:243/330.96 KB 2:256/348.96 KB 6:231/314.67 KB 5:265/361.49 KB 1:222/302.28 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 517.84 KB, 1.16 MB, 2.18 MB - FooterProcessingTime: (Avg: 12.680ms ; Min: 5.318ms ; Max: 202.700ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2712076) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712076) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712076) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 222.00 KB (227328) - PerReadThreadRawHdfsThroughput: 99.85 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s575ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.20 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 5 (5) - ScannerThreadsTotalWallClockTime: 2m21s - MaterializeTupleTime(*): 2.310ms - ScannerThreadsSysTime: 58.966ms - ScannerThreadsUserTime: 310.916ms - ScannerThreadsVoluntaryContextSwitches: 7.46K (7464) - TotalRawHdfsReadTime(*): 25.902ms - TotalReadThroughput: 1.46 MB/sec Instance c34cc30188b6a8c9:28 (host=vc1529.foo.fa.com:22000):(Total: 2s065ms, non-child: 409.911ms, % non-child: 19.84%) Hdfs split stats (:<# splits>/): 0:257/350.49 KB 1:243/331.23 KB 2:235/320.60 KB 4:246/335.13 KB 3:246/335.18 KB 5:245/334.40 KB 7:225/306.51 KB 6:247/336.75 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 430.217us - TotalStorageWaitTime: 24s815ms - TotalThreadsInvoluntaryContextSwitches: 11 (11) - TotalThreadsTotalWallClockTime: 2m23s - TotalThreadsSysTime: 88.956ms - TotalThreadsUserTime: 387.899ms - TotalThreadsVoluntaryContextSwitches: 9.41K (9413) Fragment Instance Lifecycle Timings: - ExecTime: 679.374us - ExecTreeExecTime: 4.992us - OpenTime: 1s670ms - ExecTreeOpenTime: 1s624ms - PrepareTime: 394.174ms - ExecTreePrepareTime: 6.909ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 73.989us, non-child: 73.989us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 285.04 KB/sec - OverallThroughput: 211.18 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.214us - TransmitDataRPCTime: 54.816us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.536ms, non-child: 24.536ms, % non-child: 100.00%) - CodegenTime: 603.895us - CompileTime: 4.110ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.690ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.362ms AGGREGATION_NODE (id=1):(Total: 1s631ms, non-child: 4.700ms, % non-child: 0.29%) ExecOption: Codegen Enabled - BuildTime: 1.579ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s626ms, non-child: 1s626ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:257/350.49 KB 1:243/331.23 KB 2:235/320.60 KB 4:246/335.13 KB 3:246/335.18 KB 5:245/334.40 KB 7:225/306.51 KB 6:247/336.75 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 568.74 KB, 1.43 MB, 2.12 MB - FooterProcessingTime: (Avg: 12.826ms ; Min: 5.925ms ; Max: 207.380ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713905) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713905) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713905) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 111.92 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s582ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.20 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m21s - MaterializeTupleTime(*): 2.273ms - ScannerThreadsSysTime: 58.960ms - ScannerThreadsUserTime: 289.914ms - ScannerThreadsVoluntaryContextSwitches: 7.33K (7328) - TotalRawHdfsReadTime(*): 23.124ms - TotalReadThroughput: 1.41 MB/sec Instance c34cc30188b6a8c9:7f (host=vd1118.foo.fa.com:22000):(Total: 2s053ms, non-child: 418.277ms, % non-child: 20.36%) Hdfs split stats (:<# splits>/): 5:248/337.98 KB 4:225/306.69 KB 2:237/322.90 KB 1:252/343.52 KB 3:245/333.58 KB 7:240/327.29 KB 0:229/312.14 KB 6:268/365.70 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 232.36 KB (237934) - PerHostPeakMemUsage: 232.36 KB (237934) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 415.358us - TotalStorageWaitTime: 24s039ms - TotalThreadsInvoluntaryContextSwitches: 25 (25) - TotalThreadsTotalWallClockTime: 2m21s - TotalThreadsSysTime: 83.963ms - TotalThreadsUserTime: 432.902ms - TotalThreadsVoluntaryContextSwitches: 9.32K (9320) Fragment Instance Lifecycle Timings: - ExecTime: 600.264us - ExecTreeExecTime: 12.880us - OpenTime: 1s650ms - ExecTreeOpenTime: 1s602ms - PrepareTime: 403.335ms - ExecTreePrepareTime: 7.420ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 107.889us, non-child: 107.889us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 227.85 KB/sec - OverallThroughput: 144.82 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 22.502us - TransmitDataRPCTime: 68.577us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.384ms, non-child: 25.384ms, % non-child: 100.00%) - CodegenTime: 567.191us - CompileTime: 4.112ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.751ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.060ms AGGREGATION_NODE (id=1):(Total: 1s610ms, non-child: 4.349ms, % non-child: 0.27%) ExecOption: Codegen Enabled - BuildTime: 1.583ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s605ms, non-child: 1s605ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:248/337.98 KB 4:225/306.69 KB 2:237/322.90 KB 1:252/343.52 KB 3:245/333.58 KB 7:240/327.29 KB 0:229/312.14 KB 6:268/365.70 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 179.83 KB, 1004.03 KB, 1.74 MB - FooterProcessingTime: (Avg: 12.436ms ; Min: 4.923ms ; Max: 39.598ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713378) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713378) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713378) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 192.00 KB (196608) - PerReadThreadRawHdfsThroughput: 102.09 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s558ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.21 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m19s - MaterializeTupleTime(*): 2.404ms - ScannerThreadsSysTime: 46.969ms - ScannerThreadsUserTime: 339.916ms - ScannerThreadsVoluntaryContextSwitches: 7.22K (7225) - TotalRawHdfsReadTime(*): 25.346ms - TotalReadThroughput: 1.16 MB/sec Instance c34cc30188b6a8c9:2 (host=vd1328.foo.fa.com:22000):(Total: 2s048ms, non-child: 347.943ms, % non-child: 16.98%) Hdfs split stats (:<# splits>/): 1:163/221.83 KB 5:174/237.16 KB 7:160/218.41 KB 4:180/245.23 KB 0:176/239.90 KB 6:168/229.06 KB 2:167/227.89 KB 3:174/236.99 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 194.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 258.36 KB (264558) - PerHostPeakMemUsage: 258.36 KB (264558) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 476.309us - TotalStorageWaitTime: 26s041ms - TotalThreadsInvoluntaryContextSwitches: 16 (16) - TotalThreadsTotalWallClockTime: 2m14s - TotalThreadsSysTime: 75.968ms - TotalThreadsUserTime: 295.916ms - TotalThreadsVoluntaryContextSwitches: 6.58K (6576) Fragment Instance Lifecycle Timings: - ExecTime: 689.297us - ExecTreeExecTime: 5.456us - OpenTime: 1s719ms - ExecTreeOpenTime: 1s671ms - PrepareTime: 328.513ms - ExecTreePrepareTime: 4.788ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 78.819us, non-child: 78.819us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 293.17 KB/sec - OverallThroughput: 198.24 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.024us - TransmitDataRPCTime: 53.296us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.292ms, non-child: 24.292ms, % non-child: 100.00%) - CodegenTime: 622.067us - CompileTime: 3.954ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.628ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.345ms AGGREGATION_NODE (id=1):(Total: 1s676ms, non-child: 4.408ms, % non-child: 0.26%) ExecOption: Codegen Enabled - BuildTime: 1.218ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s672ms, non-child: 1s672ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:163/221.83 KB 5:174/237.16 KB 7:160/218.41 KB 4:180/245.23 KB 0:176/239.90 KB 6:168/229.06 KB 2:167/227.89 KB 3:174/236.99 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1362 out of 1362 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1362 BytesRead(500.000ms): 205.56 KB, 664.36 KB, 992.83 KB - FooterProcessingTime: (Avg: 19.188ms ; Min: 5.545ms ; Max: 437.146ms ; Number of samples: 1362) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1901023) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1901023) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1901023) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 218.00 KB (223232) - PerReadThreadRawHdfsThroughput: 91.57 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s639ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1362) - RowsReturned: 1.36K (1362) - RowsReturnedRate: 814.00 /sec - ScanRangesComplete: 1.36K (1362) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 2m12s - MaterializeTupleTime(*): 1.704ms - ScannerThreadsSysTime: 52.972ms - ScannerThreadsUserTime: 209.929ms - ScannerThreadsVoluntaryContextSwitches: 5.12K (5120) - TotalRawHdfsReadTime(*): 19.798ms - TotalReadThroughput: 661.88 KB/sec Instance c34cc30188b6a8c9:71 (host=vc1537.foo.fa.com:22000):(Total: 2s048ms, non-child: 332.317ms, % non-child: 16.22%) Hdfs split stats (:<# splits>/): 7:231/314.25 KB 0:254/346.13 KB 1:237/322.79 KB 2:251/341.77 KB 3:247/337.12 KB 6:235/320.16 KB 4:245/333.79 KB 5:245/334.29 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 226.36 KB (231790) - PerHostPeakMemUsage: 226.36 KB (231790) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 387.757us - TotalStorageWaitTime: 26s134ms - TotalThreadsInvoluntaryContextSwitches: 33 (33) - TotalThreadsTotalWallClockTime: 2m28s - TotalThreadsSysTime: 80.967ms - TotalThreadsUserTime: 414.903ms - TotalThreadsVoluntaryContextSwitches: 9.45K (9446) Fragment Instance Lifecycle Timings: - ExecTime: 616.267us - ExecTreeExecTime: 5.864us - OpenTime: 1s730ms - ExecTreeOpenTime: 1s685ms - PrepareTime: 317.048ms - ExecTreePrepareTime: 6.431ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 87.079us, non-child: 87.079us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 276.84 KB/sec - OverallThroughput: 179.43 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.207us - TransmitDataRPCTime: 56.441us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.255ms, non-child: 24.255ms, % non-child: 100.00%) - CodegenTime: 547.932us - CompileTime: 3.945ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.738ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.213ms AGGREGATION_NODE (id=1):(Total: 1s691ms, non-child: 4.723ms, % non-child: 0.28%) ExecOption: Codegen Enabled - BuildTime: 1.393ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s687ms, non-child: 1s687ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:231/314.25 KB 0:254/346.13 KB 1:237/322.79 KB 2:251/341.77 KB 3:247/337.12 KB 6:235/320.16 KB 4:245/333.79 KB 5:245/334.29 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 446.57 KB, 1.11 MB, 1.82 MB - FooterProcessingTime: (Avg: 13.505ms ; Min: 5.331ms ; Max: 26.706ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713902) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713902) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713902) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 186.00 KB (190464) - PerReadThreadRawHdfsThroughput: 108.57 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s641ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.15 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 5 (5) - ScannerThreadsTotalWallClockTime: 2m27s - MaterializeTupleTime(*): 2.307ms - ScannerThreadsSysTime: 54.971ms - ScannerThreadsUserTime: 313.918ms - ScannerThreadsVoluntaryContextSwitches: 7.35K (7351) - TotalRawHdfsReadTime(*): 23.839ms - TotalReadThroughput: 1.21 MB/sec Instance c34cc30188b6a8c9:75 (host=vc1530.foo.fa.com:22000):(Total: 2s045ms, non-child: 331.978ms, % non-child: 16.23%) Hdfs split stats (:<# splits>/): 6:251/342.19 KB 5:227/309.33 KB 4:232/316.55 KB 7:239/325.68 KB 0:247/337.13 KB 1:247/337.08 KB 3:249/338.91 KB 2:251/342.78 KB MemoryUsage(500.000ms): 184.36 KB, 194.36 KB, 184.36 KB, 166.36 KB ThreadUsage(500.000ms): 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 414.450us - TotalStorageWaitTime: 26s111ms - TotalThreadsInvoluntaryContextSwitches: 32 (32) - TotalThreadsTotalWallClockTime: 2m28s - TotalThreadsSysTime: 73.968ms - TotalThreadsUserTime: 447.899ms - TotalThreadsVoluntaryContextSwitches: 9.34K (9342) Fragment Instance Lifecycle Timings: - ExecTime: 609.186us - ExecTreeExecTime: 5.360us - OpenTime: 1s727ms - ExecTreeOpenTime: 1s681ms - PrepareTime: 317.784ms - ExecTreePrepareTime: 6.657ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 85.139us, non-child: 85.139us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 257.51 KB/sec - OverallThroughput: 183.52 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.981us - TransmitDataRPCTime: 60.676us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.271ms, non-child: 25.271ms, % non-child: 100.00%) - CodegenTime: 603.367us - CompileTime: 4.141ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.841ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.901ms AGGREGATION_NODE (id=1):(Total: 1s688ms, non-child: 5.510ms, % non-child: 0.33%) ExecOption: Codegen Enabled - BuildTime: 1.551ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s682ms, non-child: 1s682ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:251/342.19 KB 5:227/309.33 KB 4:232/316.55 KB 7:239/325.68 KB 0:247/337.13 KB 1:247/337.08 KB 3:249/338.91 KB 2:251/342.78 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 60.13 KB, 737.66 KB, 1.42 MB, 2.40 MB - FooterProcessingTime: (Avg: 13.509ms ; Min: 5.682ms ; Max: 37.768ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713235) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713235) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713235) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.39 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s636ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.15 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 2m26s - MaterializeTupleTime(*): 2.416ms - ScannerThreadsSysTime: 45.972ms - ScannerThreadsUserTime: 342.915ms - ScannerThreadsVoluntaryContextSwitches: 7.26K (7265) - TotalRawHdfsReadTime(*): 25.270ms - TotalReadThroughput: 1.20 MB/sec Instance c34cc30188b6a8c9:7e (host=vc1533.foo.fa.com:22000):(Total: 2s039ms, non-child: 414.248ms, % non-child: 20.32%) Hdfs split stats (:<# splits>/): 2:232/316.45 KB 1:254/346.35 KB 6:223/304.32 KB 4:253/344.76 KB 7:236/321.69 KB 0:235/320.71 KB 5:259/353.13 KB 3:252/343.74 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB, 112.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89, 89 - AverageThreadTokens: 71.40 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 456.581us - TotalStorageWaitTime: 24s348ms - TotalThreadsInvoluntaryContextSwitches: 26 (26) - TotalThreadsTotalWallClockTime: 2m20s - TotalThreadsSysTime: 75.965ms - TotalThreadsUserTime: 414.899ms - TotalThreadsVoluntaryContextSwitches: 9.64K (9642) Fragment Instance Lifecycle Timings: - ExecTime: 619.942us - ExecTreeExecTime: 4.815us - OpenTime: 1s639ms - ExecTreeOpenTime: 1s591ms - PrepareTime: 399.450ms - ExecTreePrepareTime: 7.043ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 78.197us, non-child: 78.197us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 266.60 KB/sec - OverallThroughput: 199.82 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.481us - TransmitDataRPCTime: 58.608us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 26.129ms, non-child: 26.129ms, % non-child: 100.00%) - CodegenTime: 603.437us - CompileTime: 4.243ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 7.119ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.294ms AGGREGATION_NODE (id=1):(Total: 1s598ms, non-child: 5.692ms, % non-child: 0.36%) ExecOption: Codegen Enabled - BuildTime: 1.678ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s592ms, non-child: 1s592ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:232/316.45 KB 1:254/346.35 KB 6:223/304.32 KB 4:253/344.76 KB 7:236/321.69 KB 0:235/320.71 KB 5:259/353.13 KB 3:252/343.74 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 31.47 KB, 954.73 KB, 1.67 MB, 2.55 MB - FooterProcessingTime: (Avg: 12.590ms ; Min: 5.626ms ; Max: 42.420ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714790) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714790) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714790) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 97.67 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s547ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.22 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 2m18s - MaterializeTupleTime(*): 2.199ms - ScannerThreadsSysTime: 45.969ms - ScannerThreadsUserTime: 309.915ms - ScannerThreadsVoluntaryContextSwitches: 7.52K (7524) - TotalRawHdfsReadTime(*): 26.507ms - TotalReadThroughput: 1.28 MB/sec Instance c34cc30188b6a8c9:38 (host=vd1131.foo.fa.com:22000):(Total: 2s033ms, non-child: 418.519ms, % non-child: 20.58%) Hdfs split stats (:<# splits>/): 7:233/317.45 KB 4:254/346.24 KB 0:241/328.42 KB 3:243/331.38 KB 1:242/329.79 KB 6:249/339.60 KB 5:233/317.98 KB 2:249/339.98 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 321.896us - TotalStorageWaitTime: 24s553ms - TotalThreadsInvoluntaryContextSwitches: 17 (17) - TotalThreadsTotalWallClockTime: 2m7s - TotalThreadsSysTime: 69.969ms - TotalThreadsUserTime: 407.908ms - TotalThreadsVoluntaryContextSwitches: 9.52K (9517) Fragment Instance Lifecycle Timings: - ExecTime: 535.451us - ExecTreeExecTime: 4.350us - OpenTime: 1s631ms - ExecTreeOpenTime: 1s583ms - PrepareTime: 401.086ms - ExecTreePrepareTime: 6.886ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 88.675us, non-child: 88.675us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 282.58 KB/sec - OverallThroughput: 176.21 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 22.028us - TransmitDataRPCTime: 55.294us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.337ms, non-child: 24.337ms, % non-child: 100.00%) - CodegenTime: 553.705us - CompileTime: 4.055ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.693ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.229ms AGGREGATION_NODE (id=1):(Total: 1s590ms, non-child: 3.116ms, % non-child: 0.20%) ExecOption: Codegen Enabled - BuildTime: 1.306ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s587ms, non-child: 1s587ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:233/317.45 KB 4:254/346.24 KB 0:241/328.42 KB 3:243/331.38 KB 1:242/329.79 KB 6:249/339.60 KB 5:233/317.98 KB 2:249/339.98 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 529.01 KB, 1.00 MB, 2.02 MB - FooterProcessingTime: (Avg: 12.696ms ; Min: 5.657ms ; Max: 203.364ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2714465) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714465) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714465) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 106.60 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s543ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.22 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 2m5s - MaterializeTupleTime(*): 2.346ms - ScannerThreadsSysTime: 43.973ms - ScannerThreadsUserTime: 313.922ms - ScannerThreadsVoluntaryContextSwitches: 7.46K (7458) - TotalRawHdfsReadTime(*): 24.283ms - TotalReadThroughput: 1.34 MB/sec Instance c34cc30188b6a8c9:76 (host=vc1505.foo.fa.com:22000):(Total: 2s029ms, non-child: 423.398ms, % non-child: 20.86%) Hdfs split stats (:<# splits>/): 5:237/322.90 KB 6:245/334.38 KB 7:250/340.67 KB 1:249/339.13 KB 0:223/303.88 KB 2:251/342.27 KB 4:246/335.60 KB 3:244/332.46 KB MemoryUsage(500.000ms): 194.36 KB, 194.36 KB, 184.36 KB, 148.36 KB ThreadUsage(500.000ms): 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 410.939us - TotalStorageWaitTime: 24s398ms - TotalThreadsInvoluntaryContextSwitches: 23 (23) - TotalThreadsTotalWallClockTime: 2m18s - TotalThreadsSysTime: 69.970ms - TotalThreadsUserTime: 423.902ms - TotalThreadsVoluntaryContextSwitches: 9.45K (9450) Fragment Instance Lifecycle Timings: - ExecTime: 610.395us - ExecTreeExecTime: 5.974us - OpenTime: 1s622ms - ExecTreeOpenTime: 1s574ms - PrepareTime: 405.857ms - ExecTreePrepareTime: 6.961ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 73.106us, non-child: 73.106us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 321.64 KB/sec - OverallThroughput: 213.73 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.812us - TransmitDataRPCTime: 48.579us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.718ms, non-child: 24.718ms, % non-child: 100.00%) - CodegenTime: 572.992us - CompileTime: 4.050ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.666ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.634ms AGGREGATION_NODE (id=1):(Total: 1s581ms, non-child: 5.323ms, % non-child: 0.34%) ExecOption: Codegen Enabled - BuildTime: 1.674ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s575ms, non-child: 1s575ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:237/322.90 KB 6:245/334.38 KB 7:250/340.67 KB 1:249/339.13 KB 0:223/303.88 KB 2:251/342.27 KB 4:246/335.60 KB 3:244/332.46 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 14.89 KB, 722.52 KB, 1.50 MB, 2.55 MB - FooterProcessingTime: (Avg: 12.613ms ; Min: 4.915ms ; Max: 38.403ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714934) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714934) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714934) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 200.00 KB (204800) - PerReadThreadRawHdfsThroughput: 105.50 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s530ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.23 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 7 (7) - ScannerThreadsTotalWallClockTime: 2m17s - MaterializeTupleTime(*): 2.375ms - ScannerThreadsSysTime: 47.973ms - ScannerThreadsUserTime: 322.917ms - ScannerThreadsVoluntaryContextSwitches: 7.33K (7331) - TotalRawHdfsReadTime(*): 24.540ms - TotalReadThroughput: 1.27 MB/sec Instance c34cc30188b6a8c9:53 (host=vd1139.foo.fa.com:22000):(Total: 2s018ms, non-child: 413.935ms, % non-child: 20.50%) Hdfs split stats (:<# splits>/): 2:255/347.40 KB 6:245/334.10 KB 7:243/332.02 KB 1:246/335.13 KB 0:261/355.53 KB 5:228/310.70 KB 3:228/310.83 KB 4:238/324.67 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 200.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 250.36 KB (256366) - PerHostPeakMemUsage: 250.36 KB (256366) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 396.961us - TotalStorageWaitTime: 24s130ms - TotalThreadsInvoluntaryContextSwitches: 26 (26) - TotalThreadsTotalWallClockTime: 2m6s - TotalThreadsSysTime: 83.965ms - TotalThreadsUserTime: 438.909ms - TotalThreadsVoluntaryContextSwitches: 9.34K (9339) Fragment Instance Lifecycle Timings: - ExecTime: 649.594us - ExecTreeExecTime: 5.783us - OpenTime: 1s621ms - ExecTreeOpenTime: 1s573ms - PrepareTime: 396.667ms - ExecTreePrepareTime: 6.828ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 67.566us, non-child: 67.566us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 315.80 KB/sec - OverallThroughput: 231.25 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.861us - TransmitDataRPCTime: 49.477us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.714ms, non-child: 24.714ms, % non-child: 100.00%) - CodegenTime: 593.847us - CompileTime: 4.130ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.772ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.394ms AGGREGATION_NODE (id=1):(Total: 1s580ms, non-child: 5.412ms, % non-child: 0.34%) ExecOption: Codegen Enabled - BuildTime: 1.663ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s574ms, non-child: 1s574ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:255/347.40 KB 6:245/334.10 KB 7:243/332.02 KB 1:246/335.13 KB 0:261/355.53 KB 5:228/310.70 KB 3:228/310.83 KB 4:238/324.67 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 254.70 KB, 1.01 MB, 2.05 MB - FooterProcessingTime: (Avg: 12.485ms ; Min: 5.118ms ; Max: 41.800ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2713996) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713996) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713996) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 210.00 KB (215040) - PerReadThreadRawHdfsThroughput: 101.76 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s528ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.23 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 11 (11) - ScannerThreadsTotalWallClockTime: 2m4s - MaterializeTupleTime(*): 2.487ms - ScannerThreadsSysTime: 56.969ms - ScannerThreadsUserTime: 335.925ms - ScannerThreadsVoluntaryContextSwitches: 7.25K (7253) - TotalRawHdfsReadTime(*): 25.435ms - TotalReadThroughput: 1.37 MB/sec Instance c34cc30188b6a8c9:3a (host=vd1111.foo.fa.com:22000):(Total: 2s012ms, non-child: 417.903ms, % non-child: 20.76%) Hdfs split stats (:<# splits>/): 6:227/309.76 KB 0:238/325.00 KB 3:255/347.79 KB 2:241/328.77 KB 7:234/319.02 KB 4:256/349.08 KB 1:240/327.29 KB 5:252/343.36 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 456.802us - TotalStorageWaitTime: 23s889ms - TotalThreadsInvoluntaryContextSwitches: 14 (14) - TotalThreadsTotalWallClockTime: 2m17s - TotalThreadsSysTime: 69.965ms - TotalThreadsUserTime: 418.899ms - TotalThreadsVoluntaryContextSwitches: 9.64K (9643) Fragment Instance Lifecycle Timings: - ExecTime: 644.744us - ExecTreeExecTime: 6.078us - OpenTime: 1s609ms - ExecTreeOpenTime: 1s561ms - PrepareTime: 402.418ms - ExecTreePrepareTime: 7.127ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 85.375us, non-child: 85.375us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 242.52 KB/sec - OverallThroughput: 183.02 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.515us - TransmitDataRPCTime: 64.428us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.841ms, non-child: 25.841ms, % non-child: 100.00%) - CodegenTime: 631.810us - CompileTime: 4.167ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.883ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.397ms AGGREGATION_NODE (id=1):(Total: 1s568ms, non-child: 4.045ms, % non-child: 0.26%) ExecOption: Codegen Enabled - BuildTime: 1.523ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s564ms, non-child: 1s564ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:227/309.76 KB 0:238/325.00 KB 3:255/347.79 KB 2:241/328.77 KB 7:234/319.02 KB 4:256/349.08 KB 1:240/327.29 KB 5:252/343.36 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 283.57 KB, 1.25 MB, 2.31 MB - FooterProcessingTime: (Avg: 12.363ms ; Min: 5.445ms ; Max: 205.252ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713688) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713688) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713688) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 105.10 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s519ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.24 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m16s - MaterializeTupleTime(*): 2.334ms - ScannerThreadsSysTime: 45.969ms - ScannerThreadsUserTime: 319.914ms - ScannerThreadsVoluntaryContextSwitches: 7.53K (7530) - TotalRawHdfsReadTime(*): 24.624ms - TotalReadThroughput: 1.54 MB/sec Instance c34cc30188b6a8c9:7d (host=vc1513.foo.fa.com:22000):(Total: 2s008ms, non-child: 329.546ms, % non-child: 16.41%) Hdfs split stats (:<# splits>/): 4:265/361.35 KB 6:236/321.77 KB 7:251/342.32 KB 2:237/322.77 KB 0:245/333.97 KB 1:234/318.97 KB 5:256/349.01 KB 3:220/300.19 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 194.36 KB, 166.36 KB ThreadUsage(500.000ms): 89, 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 456.304us - TotalStorageWaitTime: 25s006ms - TotalThreadsInvoluntaryContextSwitches: 19 (19) - TotalThreadsTotalWallClockTime: 2m25s - TotalThreadsSysTime: 78.962ms - TotalThreadsUserTime: 421.899ms - TotalThreadsVoluntaryContextSwitches: 9.26K (9264) Fragment Instance Lifecycle Timings: - ExecTime: 642.210us - ExecTreeExecTime: 5.586us - OpenTime: 1s693ms - ExecTreeOpenTime: 1s648ms - PrepareTime: 314.426ms - ExecTreePrepareTime: 6.284ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 95.911us, non-child: 95.911us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 205.05 KB/sec - OverallThroughput: 162.91 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.749us - TransmitDataRPCTime: 76.202us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.811ms, non-child: 23.811ms, % non-child: 100.00%) - CodegenTime: 624.201us - CompileTime: 4.046ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.441ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 12.969ms AGGREGATION_NODE (id=1):(Total: 1s654ms, non-child: 5.314ms, % non-child: 0.32%) ExecOption: Codegen Enabled - BuildTime: 1.660ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s649ms, non-child: 1s649ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:265/361.35 KB 6:236/321.77 KB 7:251/342.32 KB 2:237/322.77 KB 0:245/333.97 KB 1:234/318.97 KB 5:256/349.01 KB 3:220/300.19 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 0, 803.05 KB, 1.54 MB, 2.41 MB - FooterProcessingTime: (Avg: 12.931ms ; Min: 5.157ms ; Max: 99.563ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713951) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713951) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713951) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 107.65 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s605ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.18 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 5 (5) - ScannerThreadsTotalWallClockTime: 2m23s - MaterializeTupleTime(*): 2.396ms - ScannerThreadsSysTime: 43.968ms - ScannerThreadsUserTime: 327.914ms - ScannerThreadsVoluntaryContextSwitches: 7.18K (7183) - TotalRawHdfsReadTime(*): 24.043ms - TotalReadThroughput: 1.20 MB/sec Instance c34cc30188b6a8c9:16 (host=vc1524.foo.fa.com:22000):(Total: 2s003ms, non-child: 417.053ms, % non-child: 20.82%) Hdfs split stats (:<# splits>/): 7:244/332.78 KB 3:210/286.58 KB 5:250/341.07 KB 2:264/359.88 KB 0:241/328.42 KB 1:238/324.18 KB 4:241/328.92 KB 6:256/349.01 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 166.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 268.36 KB (274798) - PerHostPeakMemUsage: 268.36 KB (274798) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 425.070us - TotalStorageWaitTime: 23s202ms - TotalThreadsInvoluntaryContextSwitches: 19 (19) - TotalThreadsTotalWallClockTime: 2m17s - TotalThreadsSysTime: 97.957ms - TotalThreadsUserTime: 401.902ms - TotalThreadsVoluntaryContextSwitches: 9.56K (9558) Fragment Instance Lifecycle Timings: - ExecTime: 652.514us - ExecTreeExecTime: 5.329us - OpenTime: 1s601ms - ExecTreeOpenTime: 1s553ms - PrepareTime: 401.494ms - ExecTreePrepareTime: 7.370ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 100.133us, non-child: 100.133us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 229.00 KB/sec - OverallThroughput: 156.04 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.276us - TransmitDataRPCTime: 68.231us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.240ms, non-child: 25.240ms, % non-child: 100.00%) - CodegenTime: 600.029us - CompileTime: 4.016ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.918ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.900ms AGGREGATION_NODE (id=1):(Total: 1s560ms, non-child: 4.725ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.606ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s556ms, non-child: 1s556ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:244/332.78 KB 3:210/286.58 KB 5:250/341.07 KB 2:264/359.88 KB 0:241/328.42 KB 1:238/324.18 KB 4:241/328.92 KB 6:256/349.01 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 372.96 KB, 1.29 MB, 2.29 MB - FooterProcessingTime: (Avg: 12.003ms ; Min: 4.902ms ; Max: 197.006ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714472) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714472) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714472) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 228.00 KB (233472) - PerReadThreadRawHdfsThroughput: 104.56 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s508ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.25 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 2m15s - MaterializeTupleTime(*): 2.215ms - ScannerThreadsSysTime: 64.962ms - ScannerThreadsUserTime: 304.916ms - ScannerThreadsVoluntaryContextSwitches: 7.48K (7476) - TotalRawHdfsReadTime(*): 24.758ms - TotalReadThroughput: 1.53 MB/sec Instance c34cc30188b6a8c9:25 (host=vd1123.foo.fa.com:22000):(Total: 1s999ms, non-child: 427.272ms, % non-child: 21.37%) Hdfs split stats (:<# splits>/): 0:249/339.55 KB 6:245/334.16 KB 4:239/325.84 KB 7:232/316.34 KB 1:237/323.33 KB 3:246/335.64 KB 2:257/350.66 KB 5:239/325.84 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 521.076us - TotalStorageWaitTime: 23s503ms - TotalThreadsInvoluntaryContextSwitches: 26 (26) - TotalThreadsTotalWallClockTime: 2m15s - TotalThreadsSysTime: 68.968ms - TotalThreadsUserTime: 431.898ms - TotalThreadsVoluntaryContextSwitches: 9.47K (9472) Fragment Instance Lifecycle Timings: - ExecTime: 714.723us - ExecTreeExecTime: 6.104us - OpenTime: 1s590ms - ExecTreeOpenTime: 1s541ms - PrepareTime: 408.292ms - ExecTreePrepareTime: 6.885ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 81.258us, non-child: 81.258us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 266.07 KB/sec - OverallThroughput: 192.29 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.267us - TransmitDataRPCTime: 58.725us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.167ms, non-child: 24.167ms, % non-child: 100.00%) - CodegenTime: 590.584us - CompileTime: 3.931ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.846ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.045ms AGGREGATION_NODE (id=1):(Total: 1s548ms, non-child: 4.355ms, % non-child: 0.28%) ExecOption: Codegen Enabled - BuildTime: 1.558ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s543ms, non-child: 1s543ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:249/339.55 KB 6:245/334.16 KB 4:239/325.84 KB 7:232/316.34 KB 1:237/323.33 KB 3:246/335.64 KB 2:257/350.66 KB 5:239/325.84 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 813.67 KB, 1.84 MB, 2.39 MB - FooterProcessingTime: (Avg: 12.159ms ; Min: 5.366ms ; Max: 207.415ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714983) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714983) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714983) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 200.00 KB (204800) - PerReadThreadRawHdfsThroughput: 98.91 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s499ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.26 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 5 (5) - ScannerThreadsTotalWallClockTime: 2m14s - MaterializeTupleTime(*): 2.352ms - ScannerThreadsSysTime: 43.972ms - ScannerThreadsUserTime: 329.913ms - ScannerThreadsVoluntaryContextSwitches: 7.38K (7378) - TotalRawHdfsReadTime(*): 26.177ms - TotalReadThroughput: 1.59 MB/sec Instance c34cc30188b6a8c9:64 (host=ve1125.foo.fa.com:22000):(Total: 1s999ms, non-child: 333.609ms, % non-child: 16.69%) Hdfs split stats (:<# splits>/): 0:239/325.62 KB 3:261/356.04 KB 4:252/343.37 KB 2:243/330.81 KB 6:246/335.17 KB 5:230/313.81 KB 7:233/317.84 KB 1:240/326.66 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 184.36 KB, 50.36 KB ThreadUsage(500.000ms): 81, 81, 81, 2 - AverageThreadTokens: 61.25 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 384.318us - TotalStorageWaitTime: 24s848ms - TotalThreadsInvoluntaryContextSwitches: 21 (21) - TotalThreadsTotalWallClockTime: 2m11s - TotalThreadsSysTime: 70.969ms - TotalThreadsUserTime: 420.906ms - TotalThreadsVoluntaryContextSwitches: 9.23K (9232) Fragment Instance Lifecycle Timings: - ExecTime: 571.376us - ExecTreeExecTime: 6.030us - OpenTime: 1s680ms - ExecTreeOpenTime: 1s634ms - PrepareTime: 318.143ms - ExecTreePrepareTime: 6.686ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 85.336us, non-child: 85.336us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 273.84 KB/sec - OverallThroughput: 183.10 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.559us - TransmitDataRPCTime: 57.058us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.230ms, non-child: 24.230ms, % non-child: 100.00%) - CodegenTime: 581.220us - CompileTime: 3.971ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.464ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.419ms AGGREGATION_NODE (id=1):(Total: 1s641ms, non-child: 5.278ms, % non-child: 0.32%) ExecOption: Codegen Enabled - BuildTime: 1.572ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s636ms, non-child: 1s636ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:239/325.62 KB 3:261/356.04 KB 4:252/343.37 KB 2:243/330.81 KB 6:246/335.17 KB 5:230/313.81 KB 7:233/317.84 KB 1:240/326.66 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 214.02 KB, 1.22 MB, 1.98 MB, 2.59 MB - FooterProcessingTime: (Avg: 12.844ms ; Min: 5.878ms ; Max: 32.586ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 60.25 - BytesRead: 2.59 MB (2712903) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712903) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712903) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 108.15 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s593ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.19 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 2m9s - MaterializeTupleTime(*): 2.264ms - ScannerThreadsSysTime: 46.972ms - ScannerThreadsUserTime: 311.922ms - ScannerThreadsVoluntaryContextSwitches: 7.16K (7161) - TotalRawHdfsReadTime(*): 23.921ms - TotalReadThroughput: 1.29 MB/sec Instance c34cc30188b6a8c9:3b (host=ve1141.foo.fa.com:22000):(Total: 1s999ms, non-child: 417.635ms, % non-child: 20.89%) Hdfs split stats (:<# splits>/): 6:218/297.59 KB 4:234/318.62 KB 5:254/345.48 KB 7:243/331.32 KB 1:249/339.44 KB 0:243/331.22 KB 3:251/341.85 KB 2:253/344.40 KB MemoryUsage(500.000ms): 0, 192.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 487.448us - TotalStorageWaitTime: 23s879ms - TotalThreadsInvoluntaryContextSwitches: 29 (29) - TotalThreadsTotalWallClockTime: 2m4s - TotalThreadsSysTime: 69.963ms - TotalThreadsUserTime: 429.906ms - TotalThreadsVoluntaryContextSwitches: 9.29K (9291) Fragment Instance Lifecycle Timings: - ExecTime: 652.095us - ExecTreeExecTime: 5.830us - OpenTime: 1s596ms - ExecTreeOpenTime: 1s548ms - PrepareTime: 401.871ms - ExecTreePrepareTime: 7.462ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 83.126us, non-child: 83.126us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 250.20 KB/sec - OverallThroughput: 187.97 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.686us - TransmitDataRPCTime: 62.450us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.173ms, non-child: 25.173ms, % non-child: 100.00%) - CodegenTime: 601.944us - CompileTime: 4.107ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.866ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.794ms AGGREGATION_NODE (id=1):(Total: 1s556ms, non-child: 2.965ms, % non-child: 0.19%) ExecOption: Codegen Enabled - BuildTime: 1.427ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s553ms, non-child: 1s553ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:218/297.59 KB 4:234/318.62 KB 5:254/345.48 KB 7:243/331.32 KB 1:249/339.44 KB 0:243/331.22 KB 3:251/341.85 KB 2:253/344.40 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:66.67% 1:33.33% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 48.78 KB, 712.01 KB, 1.73 MB - FooterProcessingTime: (Avg: 12.346ms ; Min: 5.064ms ; Max: 202.910ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.33 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2713517) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713517) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713517) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 99.44 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s509ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.25 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 8 (8) - ScannerThreadsTotalWallClockTime: 2m2s - MaterializeTupleTime(*): 2.452ms - ScannerThreadsSysTime: 47.966ms - ScannerThreadsUserTime: 329.921ms - ScannerThreadsVoluntaryContextSwitches: 7.20K (7201) - TotalRawHdfsReadTime(*): 26.024ms - TotalReadThroughput: 1.15 MB/sec Instance c34cc30188b6a8c9:2a (host=vc1526.foo.fa.com:22000):(Total: 1s998ms, non-child: 341.071ms, % non-child: 17.06%) Hdfs split stats (:<# splits>/): 2:252/343.51 KB 7:243/331.58 KB 6:247/336.73 KB 0:224/305.21 KB 1:259/353.10 KB 4:237/323.20 KB 3:240/326.97 KB 5:243/331.54 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 451.117us - TotalStorageWaitTime: 24s779ms - TotalThreadsInvoluntaryContextSwitches: 23 (23) - TotalThreadsTotalWallClockTime: 2m23s - TotalThreadsSysTime: 88.957ms - TotalThreadsUserTime: 445.900ms - TotalThreadsVoluntaryContextSwitches: 9.36K (9357) Fragment Instance Lifecycle Timings: - ExecTime: 649.973us - ExecTreeExecTime: 6.555us - OpenTime: 1s671ms - ExecTreeOpenTime: 1s623ms - PrepareTime: 326.624ms - ExecTreePrepareTime: 7.115ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 84.101us, non-child: 84.101us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 246.57 KB/sec - OverallThroughput: 185.79 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.070us - TransmitDataRPCTime: 63.369us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 26.597ms, non-child: 26.597ms, % non-child: 100.00%) - CodegenTime: 725.144us - CompileTime: 4.801ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 7.052ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.349ms AGGREGATION_NODE (id=1):(Total: 1s630ms, non-child: 4.798ms, % non-child: 0.29%) ExecOption: Codegen Enabled - BuildTime: 1.660ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s626ms, non-child: 1s626ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:252/343.51 KB 7:243/331.58 KB 6:247/336.73 KB 0:224/305.21 KB 1:259/353.10 KB 4:237/323.20 KB 3:240/326.97 KB 5:243/331.54 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 211.30 KB, 1.23 MB, 1.79 MB - FooterProcessingTime: (Avg: 12.811ms ; Min: 5.972ms ; Max: 207.867ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2715480) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2715480) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2715480) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 103.14 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s577ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.20 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 10 (10) - ScannerThreadsTotalWallClockTime: 2m21s - MaterializeTupleTime(*): 2.517ms - ScannerThreadsSysTime: 59.961ms - ScannerThreadsUserTime: 342.916ms - ScannerThreadsVoluntaryContextSwitches: 7.22K (7223) - TotalRawHdfsReadTime(*): 25.108ms - TotalReadThroughput: 1.19 MB/sec Instance c34cc30188b6a8c9:5 (host=vd1330.foo.fa.com:22000):(Total: 1s967ms, non-child: 375.532ms, % non-child: 19.09%) Hdfs split stats (:<# splits>/): 7:169/230.80 KB 3:157/213.93 KB 6:167/227.79 KB 1:168/229.70 KB 2:171/233.09 KB 5:185/252.25 KB 0:173/235.76 KB 4:171/232.63 KB MemoryUsage(500.000ms): 0, 184.36 KB, 194.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 402.881us - TotalStorageWaitTime: 23s761ms - TotalThreadsInvoluntaryContextSwitches: 14 (14) - TotalThreadsTotalWallClockTime: 2m5s - TotalThreadsSysTime: 62.974ms - TotalThreadsUserTime: 303.912ms - TotalThreadsVoluntaryContextSwitches: 6.68K (6685) Fragment Instance Lifecycle Timings: - ExecTime: 573.108us - ExecTreeExecTime: 5.125us - OpenTime: 1s613ms - ExecTreeOpenTime: 1s562ms - PrepareTime: 353.200ms - ExecTreePrepareTime: 4.943ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 94.701us, non-child: 94.701us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 303.49 KB/sec - OverallThroughput: 164.99 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.869us - TransmitDataRPCTime: 51.485us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.323ms, non-child: 24.323ms, % non-child: 100.00%) - CodegenTime: 602.159us - CompileTime: 4.015ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.709ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.230ms AGGREGATION_NODE (id=1):(Total: 1s567ms, non-child: 4.562ms, % non-child: 0.29%) ExecOption: Codegen Enabled - BuildTime: 1.255ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s562ms, non-child: 1s562ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:169/230.80 KB 3:157/213.93 KB 6:167/227.79 KB 1:168/229.70 KB 2:171/233.09 KB 5:185/252.25 KB 0:173/235.76 KB 4:171/232.63 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1361 out of 1361 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1361 BytesRead(500.000ms): 189.37 KB, 806.21 KB, 1.23 MB - FooterProcessingTime: (Avg: 17.529ms ; Min: 6.627ms ; Max: 250.278ms ; Number of samples: 1361) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1900485) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1900485) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1900485) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 200.00 KB (204800) - PerReadThreadRawHdfsThroughput: 95.69 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s527ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1361) - RowsReturned: 1.36K (1361) - RowsReturnedRate: 870.00 /sec - ScanRangesComplete: 1.36K (1361) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m3s - MaterializeTupleTime(*): 1.705ms - ScannerThreadsSysTime: 37.977ms - ScannerThreadsUserTime: 218.925ms - ScannerThreadsVoluntaryContextSwitches: 5.19K (5193) - TotalRawHdfsReadTime(*): 18.941ms - TotalReadThroughput: 840.89 KB/sec Instance c34cc30188b6a8c9:80 (host=vhs035.foo.fa.com:22000):(Total: 1s965ms, non-child: 327.942ms, % non-child: 16.68%) Hdfs split stats (:<# splits>/): 3:230/313.74 KB 4:243/331.53 KB 1:247/336.90 KB 5:238/324.44 KB 6:252/343.46 KB 7:234/319.40 KB 0:263/358.63 KB 2:237/323.01 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 331.402us - TotalStorageWaitTime: 24s433ms - TotalThreadsInvoluntaryContextSwitches: 22 (22) - TotalThreadsTotalWallClockTime: 2m21s - TotalThreadsSysTime: 62.969ms - TotalThreadsUserTime: 425.897ms - TotalThreadsVoluntaryContextSwitches: 9.55K (9552) Fragment Instance Lifecycle Timings: - ExecTime: 532.570us - ExecTreeExecTime: 5.035us - OpenTime: 1s656ms - ExecTreeOpenTime: 1s607ms - PrepareTime: 308.855ms - ExecTreePrepareTime: 6.191ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 72.583us, non-child: 72.583us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 284.92 KB/sec - OverallThroughput: 215.27 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.290us - TransmitDataRPCTime: 54.839us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.780ms, non-child: 23.780ms, % non-child: 100.00%) - CodegenTime: 542.024us - CompileTime: 3.908ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.423ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.107ms AGGREGATION_NODE (id=1):(Total: 1s614ms, non-child: 5.264ms, % non-child: 0.33%) ExecOption: Codegen Enabled - BuildTime: 1.582ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s608ms, non-child: 1s608ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:230/313.74 KB 4:243/331.53 KB 1:247/336.90 KB 5:238/324.44 KB 6:252/343.46 KB 7:234/319.40 KB 0:263/358.63 KB 2:237/323.01 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 154.21 KB, 751.42 KB, 1.71 MB - FooterProcessingTime: (Avg: 12.635ms ; Min: 5.223ms ; Max: 122.910ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.33 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714743) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714743) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714743) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 105.76 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s564ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.21 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 2m20s - MaterializeTupleTime(*): 2.395ms - ScannerThreadsSysTime: 33.974ms - ScannerThreadsUserTime: 326.912ms - ScannerThreadsVoluntaryContextSwitches: 7.46K (7456) - TotalRawHdfsReadTime(*): 24.479ms - TotalReadThroughput: 1.14 MB/sec Instance c34cc30188b6a8c9:83 (host=vd1120.foo.fa.com:22000):(Total: 1s963ms, non-child: 615.080ms, % non-child: 31.32%) Hdfs split stats (:<# splits>/): 6:246/335.67 KB 7:225/307.05 KB 2:251/341.99 KB 0:249/339.12 KB 1:245/333.58 KB 3:244/332.70 KB 4:233/317.55 KB 5:251/342.08 KB MemoryUsage(500.000ms): 0, 200.36 KB, 186.36 KB, 40.36 KB ThreadUsage(500.000ms): 1, 89, 89, 1 - AverageThreadTokens: 45.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 430.411us - TotalStorageWaitTime: 20s200ms - TotalThreadsInvoluntaryContextSwitches: 15 (15) - TotalThreadsTotalWallClockTime: 1m55s - TotalThreadsSysTime: 71.967ms - TotalThreadsUserTime: 439.897ms - TotalThreadsVoluntaryContextSwitches: 9.53K (9525) Fragment Instance Lifecycle Timings: - ExecTime: 614.304us - ExecTreeExecTime: 5.343us - OpenTime: 1s368ms - ExecTreeOpenTime: 1s315ms - PrepareTime: 594.346ms - ExecTreePrepareTime: 7.754ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 86.725us, non-child: 86.725us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 237.46 KB/sec - OverallThroughput: 180.17 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.834us - TransmitDataRPCTime: 65.801us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.676ms, non-child: 25.676ms, % non-child: 100.00%) - CodegenTime: 629.112us - CompileTime: 4.268ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.861ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.133ms AGGREGATION_NODE (id=1):(Total: 1s323ms, non-child: 3.465ms, % non-child: 0.26%) ExecOption: Codegen Enabled - BuildTime: 1.671ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s319ms, non-child: 1s319ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:246/335.67 KB 7:225/307.05 KB 2:251/341.99 KB 0:249/339.12 KB 1:245/333.58 KB 3:244/332.70 KB 4:233/317.55 KB 5:251/342.08 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 607.97 KB, 1.53 MB - FooterProcessingTime: (Avg: 10.462ms ; Min: 4.792ms ; Max: 35.153ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713320) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713320) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713320) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 100.66 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s271ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.47 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 7 (7) - ScannerThreadsTotalWallClockTime: 1m54s - MaterializeTupleTime(*): 2.479ms - ScannerThreadsSysTime: 46.970ms - ScannerThreadsUserTime: 337.913ms - ScannerThreadsVoluntaryContextSwitches: 7.42K (7424) - TotalRawHdfsReadTime(*): 25.705ms - TotalReadThroughput: 1.53 MB/sec Instance c34cc30188b6a8c9:77 (host=vd1136.foo.fa.com:22000):(Total: 1s957ms, non-child: 417.999ms, % non-child: 21.36%) Hdfs split stats (:<# splits>/): 4:229/312.38 KB 3:249/339.44 KB 0:241/328.85 KB 6:243/330.88 KB 5:240/327.88 KB 1:224/305.19 KB 7:272/370.87 KB 2:246/335.47 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 280.36 KB (287086) - PerHostPeakMemUsage: 280.36 KB (287086) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 574.596us - TotalStorageWaitTime: 22s392ms - TotalThreadsInvoluntaryContextSwitches: 23 (23) - TotalThreadsTotalWallClockTime: 2m1s - TotalThreadsSysTime: 68.967ms - TotalThreadsUserTime: 434.906ms - TotalThreadsVoluntaryContextSwitches: 9.47K (9470) Fragment Instance Lifecycle Timings: - ExecTime: 932.798us - ExecTreeExecTime: 4.864us - OpenTime: 1s554ms - ExecTreeOpenTime: 1s507ms - PrepareTime: 401.564ms - ExecTreePrepareTime: 7.017ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 73.980us, non-child: 73.980us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 293.89 KB/sec - OverallThroughput: 211.21 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.352us - TransmitDataRPCTime: 53.166us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.204ms, non-child: 25.204ms, % non-child: 100.00%) - CodegenTime: 637.361us - CompileTime: 4.171ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.805ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.848ms AGGREGATION_NODE (id=1):(Total: 1s514ms, non-child: 5.904ms, % non-child: 0.39%) ExecOption: Codegen Enabled - BuildTime: 1.637ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s508ms, non-child: 1s508ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:229/312.38 KB 3:249/339.44 KB 0:241/328.85 KB 6:243/330.88 KB 5:240/327.88 KB 1:224/305.19 KB 7:272/370.87 KB 2:246/335.47 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 0, 733.57 KB, 1.66 MB - FooterProcessingTime: (Avg: 11.586ms ; Min: 5.297ms ; Max: 40.422ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2714584) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714584) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714584) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 240.00 KB (245760) - PerReadThreadRawHdfsThroughput: 106.93 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s460ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.29 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 1m59s - MaterializeTupleTime(*): 2.328ms - ScannerThreadsSysTime: 40.971ms - ScannerThreadsUserTime: 326.922ms - ScannerThreadsVoluntaryContextSwitches: 7.40K (7403) - TotalRawHdfsReadTime(*): 24.210ms - TotalReadThroughput: 1.11 MB/sec Instance c34cc30188b6a8c9:6e (host=vc1540.foo.fa.com:22000):(Total: 1s957ms, non-child: 414.510ms, % non-child: 21.18%) Hdfs split stats (:<# splits>/): 4:242/330.14 KB 6:251/341.45 KB 1:234/318.98 KB 5:245/333.73 KB 7:227/309.31 KB 3:242/330.20 KB 0:241/328.52 KB 2:263/358.64 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 337.664us - TotalStorageWaitTime: 22s603ms - TotalThreadsInvoluntaryContextSwitches: 21 (21) - TotalThreadsTotalWallClockTime: 2m13s - TotalThreadsSysTime: 74.964ms - TotalThreadsUserTime: 430.898ms - TotalThreadsVoluntaryContextSwitches: 9.43K (9432) Fragment Instance Lifecycle Timings: - ExecTime: 530.993us - ExecTreeExecTime: 5.292us - OpenTime: 1s558ms - ExecTreeOpenTime: 1s511ms - PrepareTime: 397.575ms - ExecTreePrepareTime: 6.831ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 91.426us, non-child: 91.426us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 247.16 KB/sec - OverallThroughput: 170.90 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 17.033us - TransmitDataRPCTime: 63.219us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.126ms, non-child: 24.126ms, % non-child: 100.00%) - CodegenTime: 688.247us - CompileTime: 3.973ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.620ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.168ms AGGREGATION_NODE (id=1):(Total: 1s518ms, non-child: 4.948ms, % non-child: 0.33%) ExecOption: Codegen Enabled - BuildTime: 1.513ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s513ms, non-child: 1s513ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:242/330.14 KB 6:251/341.45 KB 1:234/318.98 KB 5:245/333.73 KB 7:227/309.31 KB 3:242/330.20 KB 0:241/328.52 KB 2:263/358.64 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 539.91 KB, 1.58 MB, 2.33 MB - FooterProcessingTime: (Avg: 11.692ms ; Min: 5.360ms ; Max: 26.977ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714594) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714594) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714594) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 106.29 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s467ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.28 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 2m11s - MaterializeTupleTime(*): 2.347ms - ScannerThreadsSysTime: 52.968ms - ScannerThreadsUserTime: 323.914ms - ScannerThreadsVoluntaryContextSwitches: 7.34K (7338) - TotalRawHdfsReadTime(*): 24.356ms - TotalReadThroughput: 1.56 MB/sec Instance c34cc30188b6a8c9:43 (host=ve1130.foo.fa.com:22000):(Total: 1s945ms, non-child: 407.399ms, % non-child: 20.94%) Hdfs split stats (:<# splits>/): 1:241/328.29 KB 5:237/323.55 KB 6:234/319.49 KB 3:241/328.54 KB 4:236/321.86 KB 7:221/300.98 KB 2:270/367.72 KB 0:264/359.54 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 76.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 320.824us - TotalStorageWaitTime: 22s299ms - TotalThreadsInvoluntaryContextSwitches: 20 (20) - TotalThreadsTotalWallClockTime: 2m - TotalThreadsSysTime: 85.960ms - TotalThreadsUserTime: 418.909ms - TotalThreadsVoluntaryContextSwitches: 9.45K (9451) Fragment Instance Lifecycle Timings: - ExecTime: 505.081us - ExecTreeExecTime: 6.425us - OpenTime: 1s552ms - ExecTreeOpenTime: 1s505ms - PrepareTime: 392.302ms - ExecTreePrepareTime: 7.234ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 84.077us, non-child: 84.077us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 251.04 KB/sec - OverallThroughput: 185.84 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.050us - TransmitDataRPCTime: 62.240us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.133ms, non-child: 25.133ms, % non-child: 100.00%) - CodegenTime: 581.851us - CompileTime: 4.136ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.831ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.762ms AGGREGATION_NODE (id=1):(Total: 1s512ms, non-child: 4.006ms, % non-child: 0.26%) ExecOption: Codegen Enabled - BuildTime: 1.563ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s508ms, non-child: 1s508ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:241/328.29 KB 5:237/323.55 KB 6:234/319.49 KB 3:241/328.54 KB 4:236/321.86 KB 7:221/300.98 KB 2:270/367.72 KB 0:264/359.54 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 700.53 KB, 1.65 MB, 2.53 MB - FooterProcessingTime: (Avg: 11.541ms ; Min: 5.599ms ; Max: 33.216ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2713572) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713572) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713572) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 98.50 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s462ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.29 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 1m59s - MaterializeTupleTime(*): 2.345ms - ScannerThreadsSysTime: 61.963ms - ScannerThreadsUserTime: 314.925ms - ScannerThreadsVoluntaryContextSwitches: 7.38K (7375) - TotalRawHdfsReadTime(*): 26.273ms - TotalReadThroughput: 1.69 MB/sec Instance c34cc30188b6a8c9:5e (host=ve1139.foo.fa.com:22000):(Total: 1s943ms, non-child: 336.388ms, % non-child: 17.30%) Hdfs split stats (:<# splits>/): 4:245/333.66 KB 6:245/333.60 KB 7:253/344.86 KB 5:239/325.90 KB 1:232/316.47 KB 2:235/320.34 KB 0:242/329.57 KB 3:254/346.09 KB MemoryUsage(500.000ms): 0, 186.36 KB, 184.36 KB, 194.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 218.36 KB (223598) - PerHostPeakMemUsage: 218.36 KB (223598) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 380.439us - TotalStorageWaitTime: 24s321ms - TotalThreadsInvoluntaryContextSwitches: 23 (23) - TotalThreadsTotalWallClockTime: 2m6s - TotalThreadsSysTime: 82.964ms - TotalThreadsUserTime: 424.909ms - TotalThreadsVoluntaryContextSwitches: 9.24K (9237) Fragment Instance Lifecycle Timings: - ExecTime: 666.966us - ExecTreeExecTime: 6.469us - OpenTime: 1s622ms - ExecTreeOpenTime: 1s577ms - PrepareTime: 320.568ms - ExecTreePrepareTime: 6.640ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 66.985us, non-child: 66.985us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 340.39 KB/sec - OverallThroughput: 233.26 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 14.500us - TransmitDataRPCTime: 45.903us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.707ms, non-child: 23.707ms, % non-child: 100.00%) - CodegenTime: 536.099us - CompileTime: 3.987ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.477ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 12.868ms AGGREGATION_NODE (id=1):(Total: 1s583ms, non-child: 5.060ms, % non-child: 0.32%) ExecOption: Codegen Enabled - BuildTime: 1.568ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s578ms, non-child: 1s578ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:245/333.66 KB 6:245/333.60 KB 7:253/344.86 KB 5:239/325.90 KB 1:232/316.47 KB 2:235/320.34 KB 0:242/329.57 KB 3:254/346.09 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 148.37 KB, 870.87 KB, 1.88 MB - FooterProcessingTime: (Avg: 12.573ms ; Min: 4.971ms ; Max: 126.280ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2714107) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714107) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714107) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 178.00 KB (182272) - PerReadThreadRawHdfsThroughput: 103.96 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s534ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.23 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m5s - MaterializeTupleTime(*): 2.292ms - ScannerThreadsSysTime: 55.968ms - ScannerThreadsUserTime: 327.924ms - ScannerThreadsVoluntaryContextSwitches: 7.18K (7182) - TotalRawHdfsReadTime(*): 24.897ms - TotalReadThroughput: 1.26 MB/sec Instance c34cc30188b6a8c9:52 (host=vc1534.foo.fa.com:22000):(Total: 1s942ms, non-child: 332.436ms, % non-child: 17.11%) Hdfs split stats (:<# splits>/): 2:238/324.59 KB 6:243/330.94 KB 3:253/344.83 KB 5:250/341.07 KB 4:251/341.65 KB 0:236/321.49 KB 1:250/340.88 KB 7:224/305.54 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 200.36 KB ThreadUsage(500.000ms): 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 462.162us - TotalStorageWaitTime: 24s346ms - TotalThreadsInvoluntaryContextSwitches: 15 (15) - TotalThreadsTotalWallClockTime: 2m19s - TotalThreadsSysTime: 74.968ms - TotalThreadsUserTime: 406.900ms - TotalThreadsVoluntaryContextSwitches: 9.53K (9533) Fragment Instance Lifecycle Timings: - ExecTime: 666.450us - ExecTreeExecTime: 5.292us - OpenTime: 1s625ms - ExecTreeOpenTime: 1s579ms - PrepareTime: 316.697ms - ExecTreePrepareTime: 6.461ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 73.679us, non-child: 73.679us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 335.36 KB/sec - OverallThroughput: 212.07 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.721us - TransmitDataRPCTime: 46.591us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.041ms, non-child: 24.041ms, % non-child: 100.00%) - CodegenTime: 524.663us - CompileTime: 3.928ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.580ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.189ms AGGREGATION_NODE (id=1):(Total: 1s586ms, non-child: 5.429ms, % non-child: 0.34%) ExecOption: Codegen Enabled - BuildTime: 1.658ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s580ms, non-child: 1s580ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:238/324.59 KB 6:243/330.94 KB 3:253/344.83 KB 5:250/341.07 KB 4:251/341.65 KB 0:236/321.49 KB 1:250/340.88 KB 7:224/305.54 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 49.07 KB, 622.79 KB, 1.61 MB - FooterProcessingTime: (Avg: 12.582ms ; Min: 5.720ms ; Max: 43.913ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714610) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714610) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714610) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 103.45 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s537ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.23 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 2m17s - MaterializeTupleTime(*): 2.258ms - ScannerThreadsSysTime: 38.973ms - ScannerThreadsUserTime: 317.914ms - ScannerThreadsVoluntaryContextSwitches: 7.47K (7467) - TotalRawHdfsReadTime(*): 25.025ms - TotalReadThroughput: 1.07 MB/sec Instance c34cc30188b6a8c9:32 (host=vd1129.foo.fa.com:22000):(Total: 1s939ms, non-child: 414.253ms, % non-child: 21.36%) Hdfs split stats (:<# splits>/): 2:256/348.71 KB 0:265/360.71 KB 1:237/323.28 KB 3:264/360.41 KB 6:231/314.78 KB 4:244/332.83 KB 5:211/287.77 KB 7:236/321.59 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 166.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 473.113us - TotalStorageWaitTime: 21s897ms - TotalThreadsInvoluntaryContextSwitches: 16 (16) - TotalThreadsTotalWallClockTime: 1m59s - TotalThreadsSysTime: 59.967ms - TotalThreadsUserTime: 421.906ms - TotalThreadsVoluntaryContextSwitches: 9.48K (9484) Fragment Instance Lifecycle Timings: - ExecTime: 689.718us - ExecTreeExecTime: 6.118us - OpenTime: 1s542ms - ExecTreeOpenTime: 1s494ms - PrepareTime: 396.847ms - ExecTreePrepareTime: 6.984ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 76.770us, non-child: 76.770us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 269.28 KB/sec - OverallThroughput: 203.53 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.161us - TransmitDataRPCTime: 58.026us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.189ms, non-child: 24.189ms, % non-child: 100.00%) - CodegenTime: 579.388us - CompileTime: 3.996ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.543ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.303ms AGGREGATION_NODE (id=1):(Total: 1s501ms, non-child: 4.462ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.631ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s496ms, non-child: 1s496ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:256/348.71 KB 0:265/360.71 KB 1:237/323.28 KB 3:264/360.41 KB 6:231/314.78 KB 4:244/332.83 KB 5:211/287.77 KB 7:236/321.59 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 649.81 KB, 1.61 MB, 2.42 MB - FooterProcessingTime: (Avg: 11.330ms ; Min: 5.352ms ; Max: 28.748ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.33 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2713681) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713681) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713681) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 98.13 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s451ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.30 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 1m58s - MaterializeTupleTime(*): 2.234ms - ScannerThreadsSysTime: 37.971ms - ScannerThreadsUserTime: 325.921ms - ScannerThreadsVoluntaryContextSwitches: 7.40K (7404) - TotalRawHdfsReadTime(*): 26.372ms - TotalReadThroughput: 1.61 MB/sec Instance c34cc30188b6a8c9:33 (host=vc1503.foo.fa.com:22000):(Total: 1s933ms, non-child: 412.941ms, % non-child: 21.35%) Hdfs split stats (:<# splits>/): 0:245/333.64 KB 1:247/336.26 KB 4:255/347.40 KB 7:250/340.88 KB 3:238/325.19 KB 6:242/329.70 KB 2:238/324.61 KB 5:229/311.83 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 94.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 428.512us - TotalStorageWaitTime: 22s761ms - TotalThreadsInvoluntaryContextSwitches: 24 (24) - TotalThreadsTotalWallClockTime: 2m11s - TotalThreadsSysTime: 86.959ms - TotalThreadsUserTime: 388.902ms - TotalThreadsVoluntaryContextSwitches: 9.48K (9476) Fragment Instance Lifecycle Timings: - ExecTime: 617.008us - ExecTreeExecTime: 5.686us - OpenTime: 1s537ms - ExecTreeOpenTime: 1s489ms - PrepareTime: 396.078ms - ExecTreePrepareTime: 7.030ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 82.713us, non-child: 82.713us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 252.00 KB/sec - OverallThroughput: 188.91 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.428us - TransmitDataRPCTime: 62.004us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.223ms, non-child: 24.223ms, % non-child: 100.00%) - CodegenTime: 595.860us - CompileTime: 3.985ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.615ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.262ms AGGREGATION_NODE (id=1):(Total: 1s496ms, non-child: 4.178ms, % non-child: 0.28%) ExecOption: Codegen Enabled - BuildTime: 1.533ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s492ms, non-child: 1s492ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:245/333.64 KB 1:247/336.26 KB 4:255/347.40 KB 7:250/340.88 KB 3:238/325.19 KB 6:242/329.70 KB 2:238/324.61 KB 5:229/311.83 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 0, 802.65 KB, 1.60 MB, 2.54 MB - FooterProcessingTime: (Avg: 11.770ms ; Min: 6.201ms ; Max: 33.146ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713100) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713100) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713100) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 103.21 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s447ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.30 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m9s - MaterializeTupleTime(*): 2.187ms - ScannerThreadsSysTime: 62.963ms - ScannerThreadsUserTime: 283.918ms - ScannerThreadsVoluntaryContextSwitches: 7.39K (7391) - TotalRawHdfsReadTime(*): 25.068ms - TotalReadThroughput: 1.27 MB/sec Instance c34cc30188b6a8c9:15 (host=vd1103.foo.fa.com:22000):(Total: 1s927ms, non-child: 340.692ms, % non-child: 17.67%) Hdfs split stats (:<# splits>/): 2:229/312.35 KB 4:238/324.40 KB 6:229/312.71 KB 7:247/336.77 KB 1:252/343.52 KB 0:246/335.33 KB 5:257/350.59 KB 3:245/334.15 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 130.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 268.36 KB (274798) - PerHostPeakMemUsage: 268.36 KB (274798) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 384.772us - TotalStorageWaitTime: 22s815ms - TotalThreadsInvoluntaryContextSwitches: 21 (21) - TotalThreadsTotalWallClockTime: 2m17s - TotalThreadsSysTime: 78.963ms - TotalThreadsUserTime: 418.900ms - TotalThreadsVoluntaryContextSwitches: 9.55K (9551) Fragment Instance Lifecycle Timings: - ExecTime: 595.270us - ExecTreeExecTime: 4.618us - OpenTime: 1s601ms - ExecTreeOpenTime: 1s555ms - PrepareTime: 326.182ms - ExecTreePrepareTime: 6.763ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 83.846us, non-child: 83.846us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 242.43 KB/sec - OverallThroughput: 186.35 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.687us - TransmitDataRPCTime: 64.450us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.035ms, non-child: 25.035ms, % non-child: 100.00%) - CodegenTime: 595.817us - CompileTime: 4.133ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.813ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.685ms AGGREGATION_NODE (id=1):(Total: 1s562ms, non-child: 4.762ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.580ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s557ms, non-child: 1s557ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:229/312.35 KB 4:238/324.40 KB 6:229/312.71 KB 7:247/336.77 KB 1:252/343.52 KB 0:246/335.33 KB 5:257/350.59 KB 3:245/334.15 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 751.86 KB, 1.68 MB, 2.49 MB - FooterProcessingTime: (Avg: 11.813ms ; Min: 5.174ms ; Max: 196.159ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713420) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713420) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713420) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 228.00 KB (233472) - PerReadThreadRawHdfsThroughput: 106.11 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s511ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.25 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m15s - MaterializeTupleTime(*): 2.293ms - ScannerThreadsSysTime: 49.967ms - ScannerThreadsUserTime: 326.914ms - ScannerThreadsVoluntaryContextSwitches: 7.44K (7444) - TotalRawHdfsReadTime(*): 24.386ms - TotalReadThroughput: 1.66 MB/sec Instance c34cc30188b6a8c9:18 (host=vhs028.foo.fa.com:22000):(Total: 1s921ms, non-child: 329.795ms, % non-child: 17.16%) Hdfs split stats (:<# splits>/): 2:258/351.94 KB 4:237/322.83 KB 7:244/332.69 KB 3:255/347.50 KB 6:241/328.21 KB 5:224/305.44 KB 0:245/334.43 KB 1:241/328.40 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 554.531us - TotalStorageWaitTime: 23s028ms - TotalThreadsInvoluntaryContextSwitches: 34 (34) - TotalThreadsTotalWallClockTime: 2m17s - TotalThreadsSysTime: 77.966ms - TotalThreadsUserTime: 407.901ms - TotalThreadsVoluntaryContextSwitches: 9.26K (9259) Fragment Instance Lifecycle Timings: - ExecTime: 748.174us - ExecTreeExecTime: 6.281us - OpenTime: 1s605ms - ExecTreeOpenTime: 1s560ms - PrepareTime: 315.555ms - ExecTreePrepareTime: 6.788ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 90.975us, non-child: 90.975us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 238.58 KB/sec - OverallThroughput: 171.75 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.246us - TransmitDataRPCTime: 65.491us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.751ms, non-child: 24.751ms, % non-child: 100.00%) - CodegenTime: 571.018us - CompileTime: 4.092ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.715ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.556ms AGGREGATION_NODE (id=1):(Total: 1s567ms, non-child: 3.868ms, % non-child: 0.25%) ExecOption: Codegen Enabled - BuildTime: 1.438ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s563ms, non-child: 1s563ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:258/351.94 KB 4:237/322.83 KB 7:244/332.69 KB 3:255/347.50 KB 6:241/328.21 KB 5:224/305.44 KB 0:245/334.43 KB 1:241/328.40 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 66.95 KB, 1.00 MB, 2.01 MB - FooterProcessingTime: (Avg: 11.903ms ; Min: 5.370ms ; Max: 194.551ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2715073) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2715073) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2715073) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 107.15 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s519ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.24 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 9 (9) - ScannerThreadsTotalWallClockTime: 2m16s - MaterializeTupleTime(*): 2.244ms - ScannerThreadsSysTime: 52.970ms - ScannerThreadsUserTime: 307.916ms - ScannerThreadsVoluntaryContextSwitches: 7.18K (7180) - TotalRawHdfsReadTime(*): 24.164ms - TotalReadThroughput: 1.34 MB/sec Instance c34cc30188b6a8c9:34 (host=vc1536.foo.fa.com:22000):(Total: 1s919ms, non-child: 405.414ms, % non-child: 21.13%) Hdfs split stats (:<# splits>/): 4:267/363.93 KB 1:271/369.20 KB 6:219/298.84 KB 3:241/328.47 KB 5:256/349.58 KB 2:199/271.12 KB 7:238/324.40 KB 0:253/345.15 KB MemoryUsage(500.000ms): 0, 194.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 398.485us - TotalStorageWaitTime: 21s868ms - TotalThreadsInvoluntaryContextSwitches: 22 (22) - TotalThreadsTotalWallClockTime: 2m10s - TotalThreadsSysTime: 78.965ms - TotalThreadsUserTime: 388.904ms - TotalThreadsVoluntaryContextSwitches: 9.55K (9554) Fragment Instance Lifecycle Timings: - ExecTime: 589.656us - ExecTreeExecTime: 4.674us - OpenTime: 1s527ms - ExecTreeOpenTime: 1s481ms - PrepareTime: 390.601ms - ExecTreePrepareTime: 7.087ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 83.322us, non-child: 83.322us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 285.11 KB/sec - OverallThroughput: 187.53 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.338us - TransmitDataRPCTime: 54.803us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.348ms, non-child: 25.348ms, % non-child: 100.00%) - CodegenTime: 579.379us - CompileTime: 4.101ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.882ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.984ms AGGREGATION_NODE (id=1):(Total: 1s488ms, non-child: 3.602ms, % non-child: 0.24%) ExecOption: Codegen Enabled - BuildTime: 1.450ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s484ms, non-child: 1s484ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:267/363.93 KB 1:271/369.20 KB 6:219/298.84 KB 3:241/328.47 KB 5:256/349.58 KB 2:199/271.12 KB 7:238/324.40 KB 0:253/345.15 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:66.67% 1:33.33% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 239.69 KB, 1.16 MB, 2.01 MB - FooterProcessingTime: (Avg: 11.313ms ; Min: 5.088ms ; Max: 35.781ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714298) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714298) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714298) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 112.08 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s438ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.31 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m9s - MaterializeTupleTime(*): 2.199ms - ScannerThreadsSysTime: 59.968ms - ScannerThreadsUserTime: 287.920ms - ScannerThreadsVoluntaryContextSwitches: 7.41K (7414) - TotalRawHdfsReadTime(*): 23.095ms - TotalReadThroughput: 1.34 MB/sec Instance c34cc30188b6a8c9:d (host=vd1105.foo.fa.com:22000):(Total: 1s906ms, non-child: 412.847ms, % non-child: 21.65%) Hdfs split stats (:<# splits>/): 7:218/296.92 KB 3:255/347.19 KB 6:235/320.27 KB 0:244/332.61 KB 4:239/325.64 KB 2:248/338.44 KB 5:255/347.29 KB 1:251/342.25 KB MemoryUsage(500.000ms): 0, 194.36 KB, 184.36 KB, 166.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 250.36 KB (256366) - PerHostPeakMemUsage: 250.36 KB (256366) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 451.513us - TotalStorageWaitTime: 22s387ms - TotalThreadsInvoluntaryContextSwitches: 23 (23) - TotalThreadsTotalWallClockTime: 2m8s - TotalThreadsSysTime: 86.969ms - TotalThreadsUserTime: 365.907ms - TotalThreadsVoluntaryContextSwitches: 9.55K (9554) Fragment Instance Lifecycle Timings: - ExecTime: 602.981us - ExecTreeExecTime: 5.202us - OpenTime: 1s510ms - ExecTreeOpenTime: 1s462ms - PrepareTime: 395.604ms - ExecTreePrepareTime: 7.275ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 70.312us, non-child: 70.312us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 290.71 KB/sec - OverallThroughput: 222.22 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.174us - TransmitDataRPCTime: 53.747us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.279ms, non-child: 24.279ms, % non-child: 100.00%) - CodegenTime: 567.829us - CompileTime: 3.951ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.643ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.323ms AGGREGATION_NODE (id=1):(Total: 1s469ms, non-child: 2.929ms, % non-child: 0.20%) ExecOption: Codegen Enabled - BuildTime: 1.435ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s466ms, non-child: 1s466ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:218/296.92 KB 3:255/347.19 KB 6:235/320.27 KB 0:244/332.61 KB 4:239/325.64 KB 2:248/338.44 KB 5:255/347.29 KB 1:251/342.25 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 739.79 KB, 1.40 MB, 2.41 MB - FooterProcessingTime: (Avg: 11.573ms ; Min: 4.977ms ; Max: 198.900ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714227) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714227) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714227) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 210.00 KB (215040) - PerReadThreadRawHdfsThroughput: 110.40 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s424ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.33 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m7s - MaterializeTupleTime(*): 2.160ms - ScannerThreadsSysTime: 55.973ms - ScannerThreadsUserTime: 275.920ms - ScannerThreadsVoluntaryContextSwitches: 7.49K (7486) - TotalRawHdfsReadTime(*): 23.445ms - TotalReadThroughput: 1.61 MB/sec Instance c34cc30188b6a8c9:3 (host=vhs037.foo.fa.com:22000):(Total: 1s904ms, non-child: 413.810ms, % non-child: 21.72%) Hdfs split stats (:<# splits>/): 6:229/312.68 KB 0:252/343.22 KB 7:262/357.40 KB 3:242/330.09 KB 2:225/306.50 KB 4:247/337.10 KB 1:237/323.09 KB 5:250/340.94 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 337.970us - TotalStorageWaitTime: 21s732ms - TotalThreadsInvoluntaryContextSwitches: 19 (19) - TotalThreadsTotalWallClockTime: 2m8s - TotalThreadsSysTime: 86.963ms - TotalThreadsUserTime: 426.900ms - TotalThreadsVoluntaryContextSwitches: 9.46K (9459) Fragment Instance Lifecycle Timings: - ExecTime: 513.537us - ExecTreeExecTime: 5.189us - OpenTime: 1s507ms - ExecTreeOpenTime: 1s459ms - PrepareTime: 397.183ms - ExecTreePrepareTime: 6.726ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 75.229us, non-child: 75.229us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 280.95 KB/sec - OverallThroughput: 207.70 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.665us - TransmitDataRPCTime: 55.614us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.442ms, non-child: 24.442ms, % non-child: 100.00%) - CodegenTime: 596.052us - CompileTime: 4.159ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.738ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.179ms AGGREGATION_NODE (id=1):(Total: 1s466ms, non-child: 5.085ms, % non-child: 0.35%) ExecOption: Codegen Enabled - BuildTime: 1.586ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s461ms, non-child: 1s461ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:229/312.68 KB 0:252/343.22 KB 7:262/357.40 KB 3:242/330.09 KB 2:225/306.50 KB 4:247/337.10 KB 1:237/323.09 KB 5:250/340.94 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 326.94 KB, 1.36 MB, 2.30 MB - FooterProcessingTime: (Avg: 11.249ms ; Min: 5.780ms ; Max: 32.031ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714627) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714627) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714627) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.34 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s415ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.33 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m7s - MaterializeTupleTime(*): 2.397ms - ScannerThreadsSysTime: 54.968ms - ScannerThreadsUserTime: 327.915ms - ScannerThreadsVoluntaryContextSwitches: 7.34K (7337) - TotalRawHdfsReadTime(*): 25.296ms - TotalReadThroughput: 1.53 MB/sec Instance c34cc30188b6a8c9:9 (host=ve1135.foo.fa.com:22000):(Total: 1s904ms, non-child: 328.314ms, % non-child: 17.24%) Hdfs split stats (:<# splits>/): 6:260/354.23 KB 3:238/324.78 KB 0:255/347.68 KB 7:246/335.34 KB 1:240/327.33 KB 2:252/343.57 KB 4:224/305.48 KB 5:228/310.95 KB MemoryUsage(500.000ms): 0, 184.36 KB, 200.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 362.704us - TotalStorageWaitTime: 23s275ms - TotalThreadsInvoluntaryContextSwitches: 10 (10) - TotalThreadsTotalWallClockTime: 2m4s - TotalThreadsSysTime: 84.968ms - TotalThreadsUserTime: 434.910ms - TotalThreadsVoluntaryContextSwitches: 9.34K (9336) Fragment Instance Lifecycle Timings: - ExecTime: 541.523us - ExecTreeExecTime: 11.560us - OpenTime: 1s591ms - ExecTreeOpenTime: 1s545ms - PrepareTime: 312.684ms - ExecTreePrepareTime: 6.513ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 103.036us, non-child: 103.036us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 200.46 KB/sec - OverallThroughput: 151.65 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.657us - TransmitDataRPCTime: 77.946us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.854ms, non-child: 23.854ms, % non-child: 100.00%) - CodegenTime: 571.535us - CompileTime: 4.022ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.523ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 12.937ms AGGREGATION_NODE (id=1):(Total: 1s552ms, non-child: 6.020ms, % non-child: 0.39%) ExecOption: Codegen Enabled - BuildTime: 1.659ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s546ms, non-child: 1s546ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:260/354.23 KB 3:238/324.78 KB 0:255/347.68 KB 7:246/335.34 KB 1:240/327.33 KB 2:252/343.57 KB 4:224/305.48 KB 5:228/310.95 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 440.97 KB, 1.52 MB, 2.23 MB - FooterProcessingTime: (Avg: 12.049ms ; Min: 5.326ms ; Max: 31.449ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2712947) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712947) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712947) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.17 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s502ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.26 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m2s - MaterializeTupleTime(*): 2.609ms - ScannerThreadsSysTime: 57.972ms - ScannerThreadsUserTime: 333.925ms - ScannerThreadsVoluntaryContextSwitches: 7.26K (7260) - TotalRawHdfsReadTime(*): 25.322ms - TotalReadThroughput: 1.48 MB/sec Instance c34cc30188b6a8c9:42 (host=vc1518.foo.fa.com:22000):(Total: 1s896ms, non-child: 409.021ms, % non-child: 21.57%) Hdfs split stats (:<# splits>/): 2:260/354.11 KB 5:259/353.67 KB 0:223/304.03 KB 1:237/323.37 KB 3:243/331.25 KB 6:232/316.26 KB 7:248/337.94 KB 4:241/328.72 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 479.297us - TotalStorageWaitTime: 22s200ms - TotalThreadsInvoluntaryContextSwitches: 16 (16) - TotalThreadsTotalWallClockTime: 2m8s - TotalThreadsSysTime: 87.961ms - TotalThreadsUserTime: 378.901ms - TotalThreadsVoluntaryContextSwitches: 9.54K (9541) Fragment Instance Lifecycle Timings: - ExecTime: 657.084us - ExecTreeExecTime: 6.267us - OpenTime: 1s502ms - ExecTreeOpenTime: 1s455ms - PrepareTime: 392.707ms - ExecTreePrepareTime: 6.812ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 79.907us, non-child: 79.907us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 259.83 KB/sec - OverallThroughput: 195.54 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.486us - TransmitDataRPCTime: 60.136us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.685ms, non-child: 24.685ms, % non-child: 100.00%) - CodegenTime: 561.133us - CompileTime: 3.976ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.571ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.748ms AGGREGATION_NODE (id=1):(Total: 1s462ms, non-child: 4.421ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.670ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s457ms, non-child: 1s457ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:260/354.11 KB 5:259/353.67 KB 0:223/304.03 KB 1:237/323.37 KB 3:243/331.25 KB 6:232/316.26 KB 7:248/337.94 KB 4:241/328.72 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 0, 897.37 KB, 1.84 MB - FooterProcessingTime: (Avg: 11.489ms ; Min: 5.771ms ; Max: 28.513ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2712930) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712930) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712930) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 200.00 KB (204800) - PerReadThreadRawHdfsThroughput: 115.74 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s414ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.33 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 2m6s - MaterializeTupleTime(*): 2.203ms - ScannerThreadsSysTime: 62.965ms - ScannerThreadsUserTime: 282.916ms - ScannerThreadsVoluntaryContextSwitches: 7.47K (7471) - TotalRawHdfsReadTime(*): 22.354ms - TotalReadThroughput: 1.23 MB/sec Instance c34cc30188b6a8c9:a (host=ve1128.foo.fa.com:22000):(Total: 1s895ms, non-child: 342.546ms, % non-child: 18.07%) Hdfs split stats (:<# splits>/): 1:255/347.55 KB 2:234/318.98 KB 4:244/332.73 KB 7:249/339.34 KB 0:246/335.61 KB 6:230/313.23 KB 5:240/326.60 KB 3:246/334.93 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 350.975us - TotalStorageWaitTime: 23s027ms - TotalThreadsInvoluntaryContextSwitches: 24 (24) - TotalThreadsTotalWallClockTime: 2m2s - TotalThreadsSysTime: 85.962ms - TotalThreadsUserTime: 401.910ms - TotalThreadsVoluntaryContextSwitches: 9.52K (9519) Fragment Instance Lifecycle Timings: - ExecTime: 522.299us - ExecTreeExecTime: 5.880us - OpenTime: 1s567ms - ExecTreeOpenTime: 1s520ms - PrepareTime: 327.610ms - ExecTreePrepareTime: 6.919ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 82.272us, non-child: 82.272us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 249.86 KB/sec - OverallThroughput: 189.92 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.234us - TransmitDataRPCTime: 62.534us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.049ms, non-child: 25.049ms, % non-child: 100.00%) - CodegenTime: 615.930us - CompileTime: 4.123ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.789ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.734ms AGGREGATION_NODE (id=1):(Total: 1s527ms, non-child: 3.562ms, % non-child: 0.23%) ExecOption: Codegen Enabled - BuildTime: 1.348ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s524ms, non-child: 1s524ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:255/347.55 KB 2:234/318.98 KB 4:244/332.73 KB 7:249/339.34 KB 0:246/335.61 KB 6:230/313.23 KB 5:240/326.60 KB 3:246/334.93 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 65.30 KB, 1.10 MB, 2.08 MB - FooterProcessingTime: (Avg: 11.910ms ; Min: 5.613ms ; Max: 33.921ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2712540) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712540) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712540) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 108.65 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s481ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.27 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m - MaterializeTupleTime(*): 2.133ms - ScannerThreadsSysTime: 60.966ms - ScannerThreadsUserTime: 300.925ms - ScannerThreadsVoluntaryContextSwitches: 7.37K (7371) - TotalRawHdfsReadTime(*): 23.810ms - TotalReadThroughput: 1.38 MB/sec Instance c34cc30188b6a8c9:58 (host=vd1336.foo.fa.com:22000):(Total: 1s883ms, non-child: 335.530ms, % non-child: 17.82%) Hdfs split stats (:<# splits>/): 5:166/226.13 KB 7:154/209.97 KB 4:182/248.43 KB 2:166/226.17 KB 6:176/240.03 KB 3:161/219.20 KB 0:180/245.56 KB 1:176/239.82 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 184.36 KB, 60.36 KB ThreadUsage(500.000ms): 81, 81, 81, 3 - AverageThreadTokens: 61.50 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 363.207us - TotalStorageWaitTime: 23s303ms - TotalThreadsInvoluntaryContextSwitches: 18 (18) - TotalThreadsTotalWallClockTime: 2m1s - TotalThreadsSysTime: 67.966ms - TotalThreadsUserTime: 294.911ms - TotalThreadsVoluntaryContextSwitches: 6.58K (6581) Fragment Instance Lifecycle Timings: - ExecTime: 549.164us - ExecTreeExecTime: 6.615us - OpenTime: 1s564ms - ExecTreeOpenTime: 1s518ms - PrepareTime: 317.738ms - ExecTreePrepareTime: 4.912ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 81.584us, non-child: 81.584us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 286.75 KB/sec - OverallThroughput: 191.52 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.088us - TransmitDataRPCTime: 54.489us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.074ms, non-child: 24.074ms, % non-child: 100.00%) - CodegenTime: 590.867us - CompileTime: 4.031ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.498ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.130ms AGGREGATION_NODE (id=1):(Total: 1s523ms, non-child: 4.083ms, % non-child: 0.27%) ExecOption: Codegen Enabled - BuildTime: 1.174ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s519ms, non-child: 1s519ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:166/226.13 KB 7:154/209.97 KB 4:182/248.43 KB 2:166/226.17 KB 6:176/240.03 KB 3:161/219.20 KB 0:180/245.56 KB 1:176/239.82 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1361 out of 1361 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1361 BytesRead(500.000ms): 0, 959.82 KB, 1.40 MB, 1.81 MB - FooterProcessingTime: (Avg: 17.191ms ; Min: 6.004ms ; Max: 378.469ms ; Number of samples: 1361) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 60.50 - BytesRead: 1.81 MB (1899826) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1899826) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1899826) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.23 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s484ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1361) - RowsReturned: 1.36K (1361) - RowsReturnedRate: 895.00 /sec - ScanRangesComplete: 1.36K (1361) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 2m - MaterializeTupleTime(*): 1.714ms - ScannerThreadsSysTime: 44.970ms - ScannerThreadsUserTime: 207.924ms - ScannerThreadsVoluntaryContextSwitches: 5.09K (5095) - TotalRawHdfsReadTime(*): 17.723ms - TotalReadThroughput: 926.27 KB/sec Instance c34cc30188b6a8c9:3d (host=vc1542.foo.fa.com:22000):(Total: 1s880ms, non-child: 412.116ms, % non-child: 21.91%) Hdfs split stats (:<# splits>/): 3:229/312.14 KB 7:245/334.50 KB 4:246/335.04 KB 6:226/307.65 KB 5:263/358.19 KB 2:252/343.54 KB 0:242/329.66 KB 1:241/328.16 KB MemoryUsage(500.000ms): 0, 184.36 KB, 200.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 452.509us - TotalStorageWaitTime: 21s481ms - TotalThreadsInvoluntaryContextSwitches: 28 (28) - TotalThreadsTotalWallClockTime: 2m6s - TotalThreadsSysTime: 72.966ms - TotalThreadsUserTime: 421.900ms - TotalThreadsVoluntaryContextSwitches: 9.41K (9410) Fragment Instance Lifecycle Timings: - ExecTime: 637.718us - ExecTreeExecTime: 5.713us - OpenTime: 1s484ms - ExecTreeOpenTime: 1s437ms - PrepareTime: 395.294ms - ExecTreePrepareTime: 6.738ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 80.142us, non-child: 80.142us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 278.92 KB/sec - OverallThroughput: 194.97 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.892us - TransmitDataRPCTime: 56.019us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.764ms, non-child: 24.764ms, % non-child: 100.00%) - CodegenTime: 554.869us - CompileTime: 4.009ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.726ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.658ms AGGREGATION_NODE (id=1):(Total: 1s443ms, non-child: 4.112ms, % non-child: 0.28%) ExecOption: Codegen Enabled - BuildTime: 1.458ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s439ms, non-child: 1s439ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:229/312.14 KB 7:245/334.50 KB 4:246/335.04 KB 6:226/307.65 KB 5:263/358.19 KB 2:252/343.54 KB 0:242/329.66 KB 1:241/328.16 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 305.46 KB, 1.29 MB, 2.17 MB - FooterProcessingTime: (Avg: 11.118ms ; Min: 5.448ms ; Max: 31.044ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2712447) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712447) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712447) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 104.63 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s395ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.35 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m5s - MaterializeTupleTime(*): 2.388ms - ScannerThreadsSysTime: 47.969ms - ScannerThreadsUserTime: 321.915ms - ScannerThreadsVoluntaryContextSwitches: 7.28K (7279) - TotalRawHdfsReadTime(*): 24.722ms - TotalReadThroughput: 1.45 MB/sec Instance c34cc30188b6a8c9:85 (host=vc1539.foo.fa.com:22000):(Total: 1s871ms, non-child: 409.694ms, % non-child: 21.90%) Hdfs split stats (:<# splits>/): 4:248/338.09 KB 5:254/346.39 KB 0:252/343.40 KB 1:231/314.74 KB 3:264/359.91 KB 6:243/331.35 KB 7:229/312.14 KB 2:224/305.11 KB MemoryUsage(500.000ms): 28.00 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89 - AverageThreadTokens: 59.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 471.957us - TotalStorageWaitTime: 21s706ms - TotalThreadsInvoluntaryContextSwitches: 17 (17) - TotalThreadsTotalWallClockTime: 2m6s - TotalThreadsSysTime: 89.954ms - TotalThreadsUserTime: 360.902ms - TotalThreadsVoluntaryContextSwitches: 9.63K (9627) Fragment Instance Lifecycle Timings: - ExecTime: 665.372us - ExecTreeExecTime: 4.838us - OpenTime: 1s477ms - ExecTreeOpenTime: 1s430ms - PrepareTime: 392.647ms - ExecTreePrepareTime: 7.057ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 78.259us, non-child: 78.259us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 262.74 KB/sec - OverallThroughput: 199.66 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.514us - TransmitDataRPCTime: 59.470us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.156ms, non-child: 24.156ms, % non-child: 100.00%) - CodegenTime: 619.104us - CompileTime: 3.958ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.628ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.216ms AGGREGATION_NODE (id=1):(Total: 1s437ms, non-child: 3.560ms, % non-child: 0.25%) ExecOption: Codegen Enabled - BuildTime: 1.482ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s433ms, non-child: 1s433ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:248/338.09 KB 5:254/346.39 KB 0:252/343.40 KB 1:231/314.74 KB 3:264/359.91 KB 6:243/331.35 KB 7:229/312.14 KB 2:224/305.11 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 0, 1.02 MB, 1.69 MB - FooterProcessingTime: (Avg: 11.220ms ; Min: 4.556ms ; Max: 213.173ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714746) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714746) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714746) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 113.80 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s390ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.36 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 2m4s - MaterializeTupleTime(*): 2.135ms - ScannerThreadsSysTime: 63.958ms - ScannerThreadsUserTime: 266.917ms - ScannerThreadsVoluntaryContextSwitches: 7.55K (7548) - TotalRawHdfsReadTime(*): 22.750ms - TotalReadThroughput: 1.13 MB/sec Instance c34cc30188b6a8c9:2d (host=vc1520.foo.fa.com:22000):(Total: 1s868ms, non-child: 407.275ms, % non-child: 21.79%) Hdfs split stats (:<# splits>/): 7:263/358.71 KB 0:245/333.81 KB 1:224/305.62 KB 5:242/329.90 KB 2:245/334.51 KB 6:250/341.08 KB 3:246/334.93 KB 4:229/312.36 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 192.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 402.592us - TotalStorageWaitTime: 21s355ms - TotalThreadsInvoluntaryContextSwitches: 21 (21) - TotalThreadsTotalWallClockTime: 2m6s - TotalThreadsSysTime: 58.974ms - TotalThreadsUserTime: 425.899ms - TotalThreadsVoluntaryContextSwitches: 9.50K (9498) Fragment Instance Lifecycle Timings: - ExecTime: 643.550us - ExecTreeExecTime: 5.278us - OpenTime: 1s476ms - ExecTreeOpenTime: 1s430ms - PrepareTime: 391.352ms - ExecTreePrepareTime: 6.539ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 84.666us, non-child: 84.666us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 247.03 KB/sec - OverallThroughput: 184.55 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.013us - TransmitDataRPCTime: 63.252us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.984ms, non-child: 23.984ms, % non-child: 100.00%) - CodegenTime: 593.863us - CompileTime: 3.985ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.674ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 12.984ms AGGREGATION_NODE (id=1):(Total: 1s437ms, non-child: 4.742ms, % non-child: 0.33%) ExecOption: Codegen Enabled - BuildTime: 1.476ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s432ms, non-child: 1s432ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:263/358.71 KB 0:245/333.81 KB 1:224/305.62 KB 5:242/329.90 KB 2:245/334.51 KB 6:250/341.08 KB 3:246/334.93 KB 4:229/312.36 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 201.52 KB, 1.11 MB, 2.11 MB - FooterProcessingTime: (Avg: 11.053ms ; Min: 5.558ms ; Max: 32.135ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714554) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714554) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714554) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 200.00 KB (204800) - PerReadThreadRawHdfsThroughput: 101.27 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s386ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.36 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 2m4s - MaterializeTupleTime(*): 2.313ms - ScannerThreadsSysTime: 31.978ms - ScannerThreadsUserTime: 328.913ms - ScannerThreadsVoluntaryContextSwitches: 7.43K (7432) - TotalRawHdfsReadTime(*): 25.563ms - TotalReadThroughput: 1.41 MB/sec Instance c34cc30188b6a8c9:17 (host=vhs032.foo.fa.com:22000):(Total: 1s866ms, non-child: 331.973ms, % non-child: 17.78%) Hdfs split stats (:<# splits>/): 4:245/333.81 KB 6:256/349.22 KB 5:244/332.79 KB 1:233/317.97 KB 0:226/308.69 KB 7:248/338.32 KB 2:254/346.26 KB 3:237/322.90 KB MemoryUsage(500.000ms): 0, 186.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 303.151us - TotalStorageWaitTime: 23s091ms - TotalThreadsInvoluntaryContextSwitches: 26 (26) - TotalThreadsTotalWallClockTime: 2m12s - TotalThreadsSysTime: 64.973ms - TotalThreadsUserTime: 397.901ms - TotalThreadsVoluntaryContextSwitches: 9.58K (9583) Fragment Instance Lifecycle Timings: - ExecTime: 467.011us - ExecTreeExecTime: 5.600us - OpenTime: 1s549ms - ExecTreeOpenTime: 1s503ms - PrepareTime: 316.741ms - ExecTreePrepareTime: 6.573ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 75.041us, non-child: 75.041us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 334.58 KB/sec - OverallThroughput: 208.22 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.415us - TransmitDataRPCTime: 46.700us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.518ms, non-child: 24.518ms, % non-child: 100.00%) - CodegenTime: 593.040us - CompileTime: 4.254ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.578ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.311ms AGGREGATION_NODE (id=1):(Total: 1s510ms, non-child: 3.506ms, % non-child: 0.23%) ExecOption: Codegen Enabled - BuildTime: 1.328ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s506ms, non-child: 1s506ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:245/333.81 KB 6:256/349.22 KB 5:244/332.79 KB 1:233/317.97 KB 0:226/308.69 KB 7:248/338.32 KB 2:254/346.26 KB 3:237/322.90 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 321.82 KB, 1.29 MB, 2.14 MB - FooterProcessingTime: (Avg: 11.946ms ; Min: 5.097ms ; Max: 198.519ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713554) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713554) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713554) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 110.55 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s464ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.29 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m11s - MaterializeTupleTime(*): 2.191ms - ScannerThreadsSysTime: 37.977ms - ScannerThreadsUserTime: 304.915ms - ScannerThreadsVoluntaryContextSwitches: 7.51K (7510) - TotalRawHdfsReadTime(*): 23.408ms - TotalReadThroughput: 1.43 MB/sec Instance c34cc30188b6a8c9:14 (host=vd1114.foo.fa.com:22000):(Total: 1s857ms, non-child: 425.209ms, % non-child: 22.89%) Hdfs split stats (:<# splits>/): 3:239/326.16 KB 6:225/306.84 KB 0:235/320.62 KB 5:241/328.20 KB 7:235/319.82 KB 1:245/334.10 KB 4:262/357.07 KB 2:263/358.25 KB MemoryUsage(500.000ms): 28.00 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89 - AverageThreadTokens: 59.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 466.531us - TotalStorageWaitTime: 20s883ms - TotalThreadsInvoluntaryContextSwitches: 11 (11) - TotalThreadsTotalWallClockTime: 2m3s - TotalThreadsSysTime: 75.970ms - TotalThreadsUserTime: 421.898ms - TotalThreadsVoluntaryContextSwitches: 9.44K (9439) Fragment Instance Lifecycle Timings: - ExecTime: 695.769us - ExecTreeExecTime: 5.637us - OpenTime: 1s449ms - ExecTreeOpenTime: 1s400ms - PrepareTime: 407.288ms - ExecTreePrepareTime: 6.452ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 82.099us, non-child: 82.099us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 274.78 KB/sec - OverallThroughput: 190.32 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.743us - TransmitDataRPCTime: 56.864us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.411ms, non-child: 25.411ms, % non-child: 100.00%) - CodegenTime: 561.923us - CompileTime: 4.245ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.934ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.820ms AGGREGATION_NODE (id=1):(Total: 1s406ms, non-child: 6.179ms, % non-child: 0.44%) ExecOption: Codegen Enabled - BuildTime: 1.705ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s400ms, non-child: 1s400ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:239/326.16 KB 6:225/306.84 KB 0:235/320.62 KB 5:241/328.20 KB 7:235/319.82 KB 1:245/334.10 KB 4:262/357.07 KB 2:263/358.25 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 0, 977.42 KB, 1.91 MB - FooterProcessingTime: (Avg: 10.803ms ; Min: 4.976ms ; Max: 40.933ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714681) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714681) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714681) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 200.00 KB (204800) - PerReadThreadRawHdfsThroughput: 92.16 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s351ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.39 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m1s - MaterializeTupleTime(*): 2.280ms - ScannerThreadsSysTime: 45.974ms - ScannerThreadsUserTime: 319.913ms - ScannerThreadsVoluntaryContextSwitches: 7.33K (7332) - TotalRawHdfsReadTime(*): 28.090ms - TotalReadThroughput: 1.27 MB/sec Instance c34cc30188b6a8c9:40 (host=vhs029.foo.fa.com:22000):(Total: 1s852ms, non-child: 343.689ms, % non-child: 18.55%) Hdfs split stats (:<# splits>/): 0:243/331.48 KB 4:259/353.33 KB 1:228/310.77 KB 5:261/354.97 KB 2:233/317.53 KB 3:234/318.96 KB 7:238/324.34 KB 6:247/336.71 KB MemoryUsage(500.000ms): 0, 194.36 KB, 208.36 KB, 192.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 309.894us - TotalStorageWaitTime: 22s319ms - TotalThreadsInvoluntaryContextSwitches: 21 (21) - TotalThreadsTotalWallClockTime: 2m10s - TotalThreadsSysTime: 88.962ms - TotalThreadsUserTime: 405.902ms - TotalThreadsVoluntaryContextSwitches: 9.35K (9345) Fragment Instance Lifecycle Timings: - ExecTime: 508.386us - ExecTreeExecTime: 5.295us - OpenTime: 1s524ms - ExecTreeOpenTime: 1s478ms - PrepareTime: 327.803ms - ExecTreePrepareTime: 6.385ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 73.388us, non-child: 73.388us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 285.74 KB/sec - OverallThroughput: 212.91 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.005us - TransmitDataRPCTime: 54.682us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.989ms, non-child: 23.989ms, % non-child: 100.00%) - CodegenTime: 555.474us - CompileTime: 4.040ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.626ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 12.936ms AGGREGATION_NODE (id=1):(Total: 1s484ms, non-child: 5.070ms, % non-child: 0.34%) ExecOption: Codegen Enabled - BuildTime: 1.572ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s479ms, non-child: 1s479ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:243/331.48 KB 4:259/353.33 KB 1:228/310.77 KB 5:261/354.97 KB 2:233/317.53 KB 3:234/318.96 KB 7:238/324.34 KB 6:247/336.71 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 241.26 KB, 1.14 MB, 2.11 MB - FooterProcessingTime: (Avg: 11.553ms ; Min: 6.072ms ; Max: 33.138ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2711628) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2711628) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2711628) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 106.55 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s436ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.31 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 10 (10) - ScannerThreadsTotalWallClockTime: 2m8s - MaterializeTupleTime(*): 2.238ms - ScannerThreadsSysTime: 62.966ms - ScannerThreadsUserTime: 306.917ms - ScannerThreadsVoluntaryContextSwitches: 7.25K (7253) - TotalRawHdfsReadTime(*): 24.271ms - TotalReadThroughput: 1.40 MB/sec Instance c34cc30188b6a8c9:44 (host=vd1135.foo.fa.com:22000):(Total: 1s838ms, non-child: 418.774ms, % non-child: 22.77%) Hdfs split stats (:<# splits>/): 4:244/332.63 KB 5:255/348.01 KB 1:223/303.87 KB 6:249/339.57 KB 7:233/317.76 KB 0:259/352.57 KB 2:224/305.34 KB 3:257/350.04 KB MemoryUsage(500.000ms): 0, 194.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 487.717us - TotalStorageWaitTime: 20s989ms - TotalThreadsInvoluntaryContextSwitches: 13 (13) - TotalThreadsTotalWallClockTime: 1m51s - TotalThreadsSysTime: 61.968ms - TotalThreadsUserTime: 417.906ms - TotalThreadsVoluntaryContextSwitches: 9.60K (9600) Fragment Instance Lifecycle Timings: - ExecTime: 669.572us - ExecTreeExecTime: 5.749us - OpenTime: 1s436ms - ExecTreeOpenTime: 1s388ms - PrepareTime: 401.278ms - ExecTreePrepareTime: 7.053ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 90.079us, non-child: 90.079us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 277.73 KB/sec - OverallThroughput: 173.46 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.175us - TransmitDataRPCTime: 56.260us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.668ms, non-child: 24.668ms, % non-child: 100.00%) - CodegenTime: 549.383us - CompileTime: 3.997ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.648ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.644ms AGGREGATION_NODE (id=1):(Total: 1s395ms, non-child: 3.827ms, % non-child: 0.27%) ExecOption: Codegen Enabled - BuildTime: 1.537ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s391ms, non-child: 1s391ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:244/332.63 KB 5:255/348.01 KB 1:223/303.87 KB 6:249/339.57 KB 7:233/317.76 KB 0:259/352.57 KB 2:224/305.34 KB 3:257/350.04 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:66.67% 1:33.33% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 79.06 KB, 1007.30 KB, 2.00 MB - FooterProcessingTime: (Avg: 10.864ms ; Min: 5.353ms ; Max: 31.388ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2713380) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713380) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713380) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 101.21 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s346ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.40 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 1m50s - MaterializeTupleTime(*): 2.314ms - ScannerThreadsSysTime: 42.971ms - ScannerThreadsUserTime: 316.921ms - ScannerThreadsVoluntaryContextSwitches: 7.53K (7530) - TotalRawHdfsReadTime(*): 25.567ms - TotalReadThroughput: 1.33 MB/sec Instance c34cc30188b6a8c9:31 (host=vc1501.foo.fa.com:22000):(Total: 1s835ms, non-child: 336.351ms, % non-child: 18.32%) Hdfs split stats (:<# splits>/): 1:244/332.37 KB 5:230/313.49 KB 6:240/327.88 KB 0:240/327.31 KB 7:249/339.82 KB 2:264/359.73 KB 3:247/337.17 KB 4:229/311.86 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 541.173us - TotalStorageWaitTime: 21s867ms - TotalThreadsInvoluntaryContextSwitches: 33 (33) - TotalThreadsTotalWallClockTime: 2m9s - TotalThreadsSysTime: 61.967ms - TotalThreadsUserTime: 419.898ms - TotalThreadsVoluntaryContextSwitches: 9.39K (9392) Fragment Instance Lifecycle Timings: - ExecTime: 737.363us - ExecTreeExecTime: 5.927us - OpenTime: 1s514ms - ExecTreeOpenTime: 1s468ms - PrepareTime: 320.543ms - ExecTreePrepareTime: 6.542ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 94.946us, non-child: 94.946us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 224.42 KB/sec - OverallThroughput: 164.57 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.917us - TransmitDataRPCTime: 69.624us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.250ms, non-child: 24.250ms, % non-child: 100.00%) - CodegenTime: 586.935us - CompileTime: 4.061ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.677ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.118ms AGGREGATION_NODE (id=1):(Total: 1s475ms, non-child: 2.785ms, % non-child: 0.19%) ExecOption: Codegen Enabled - BuildTime: 1.200ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s472ms, non-child: 1s472ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:244/332.37 KB 5:230/313.49 KB 6:240/327.88 KB 0:240/327.31 KB 7:249/339.82 KB 2:264/359.73 KB 3:247/337.17 KB 4:229/311.86 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 0, 1.03 MB, 1.93 MB - FooterProcessingTime: (Avg: 11.320ms ; Min: 5.269ms ; Max: 33.637ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713228) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713228) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713228) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 110.11 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s428ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.32 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 2m7s - MaterializeTupleTime(*): 2.260ms - ScannerThreadsSysTime: 42.970ms - ScannerThreadsUserTime: 321.913ms - ScannerThreadsVoluntaryContextSwitches: 7.28K (7276) - TotalRawHdfsReadTime(*): 23.499ms - TotalReadThroughput: 1.28 MB/sec Instance c34cc30188b6a8c9:41 (host=vhs039.foo.fa.com:22000):(Total: 1s826ms, non-child: 334.522ms, % non-child: 18.32%) Hdfs split stats (:<# splits>/): 5:254/346.01 KB 4:250/340.89 KB 7:229/311.95 KB 1:236/321.73 KB 2:248/338.22 KB 6:246/335.08 KB 3:233/317.72 KB 0:249/339.18 KB MemoryUsage(500.000ms): 28.00 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89 - AverageThreadTokens: 59.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 449.488us - TotalStorageWaitTime: 22s474ms - TotalThreadsInvoluntaryContextSwitches: 16 (16) - TotalThreadsTotalWallClockTime: 2m8s - TotalThreadsSysTime: 81.965ms - TotalThreadsUserTime: 412.903ms - TotalThreadsVoluntaryContextSwitches: 9.66K (9665) Fragment Instance Lifecycle Timings: - ExecTime: 654.471us - ExecTreeExecTime: 6.836us - OpenTime: 1s505ms - ExecTreeOpenTime: 1s460ms - PrepareTime: 319.892ms - ExecTreePrepareTime: 6.628ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 92.512us, non-child: 92.512us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 212.16 KB/sec - OverallThroughput: 168.90 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.061us - TransmitDataRPCTime: 73.646us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.930ms, non-child: 24.930ms, % non-child: 100.00%) - CodegenTime: 555.718us - CompileTime: 4.206ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.809ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.500ms AGGREGATION_NODE (id=1):(Total: 1s466ms, non-child: 5.737ms, % non-child: 0.39%) ExecOption: Codegen Enabled - BuildTime: 1.806ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s460ms, non-child: 1s460ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:254/346.01 KB 4:250/340.89 KB 7:229/311.95 KB 1:236/321.73 KB 2:248/338.22 KB 6:246/335.08 KB 3:233/317.72 KB 0:249/339.18 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 0, 880.02 KB, 1.74 MB - FooterProcessingTime: (Avg: 11.623ms ; Min: 6.024ms ; Max: 36.837ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714393) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714393) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714393) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 99.67 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s414ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.33 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m7s - MaterializeTupleTime(*): 2.222ms - ScannerThreadsSysTime: 53.970ms - ScannerThreadsUserTime: 315.918ms - ScannerThreadsVoluntaryContextSwitches: 7.57K (7565) - TotalRawHdfsReadTime(*): 25.972ms - TotalReadThroughput: 1.16 MB/sec Instance c34cc30188b6a8c9:19 (host=vd1132.foo.fa.com:22000):(Total: 1s820ms, non-child: 411.799ms, % non-child: 22.63%) Hdfs split stats (:<# splits>/): 1:252/343.22 KB 6:233/317.40 KB 3:253/344.83 KB 2:242/330.04 KB 4:240/327.43 KB 7:229/311.96 KB 5:261/355.43 KB 0:234/318.90 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 200.36 KB ThreadUsage(500.000ms): 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 426.134us - TotalStorageWaitTime: 20s742ms - TotalThreadsInvoluntaryContextSwitches: 14 (14) - TotalThreadsTotalWallClockTime: 1m50s - TotalThreadsSysTime: 65.972ms - TotalThreadsUserTime: 422.908ms - TotalThreadsVoluntaryContextSwitches: 9.59K (9591) Fragment Instance Lifecycle Timings: - ExecTime: 637.775us - ExecTreeExecTime: 4.891us - OpenTime: 1s424ms - ExecTreeOpenTime: 1s376ms - PrepareTime: 395.053ms - ExecTreePrepareTime: 6.955ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 61.901us, non-child: 61.901us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 356.26 KB/sec - OverallThroughput: 252.42 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.585us - TransmitDataRPCTime: 43.858us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.452ms, non-child: 24.452ms, % non-child: 100.00%) - CodegenTime: 601.327us - CompileTime: 4.009ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.737ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.282ms AGGREGATION_NODE (id=1):(Total: 1s383ms, non-child: 3.964ms, % non-child: 0.29%) ExecOption: Codegen Enabled - BuildTime: 1.603ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s379ms, non-child: 1s379ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:252/343.22 KB 6:233/317.40 KB 3:253/344.83 KB 2:242/330.04 KB 4:240/327.43 KB 7:229/311.96 KB 5:261/355.43 KB 0:234/318.90 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 19.20 KB, 990.85 KB, 1.97 MB - FooterProcessingTime: (Avg: 10.738ms ; Min: 5.693ms ; Max: 35.587ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2712792) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712792) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712792) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.92 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s334ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.41 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 1m49s - MaterializeTupleTime(*): 2.476ms - ScannerThreadsSysTime: 41.975ms - ScannerThreadsUserTime: 325.922ms - ScannerThreadsVoluntaryContextSwitches: 7.52K (7524) - TotalRawHdfsReadTime(*): 25.136ms - TotalReadThroughput: 1.32 MB/sec Instance c34cc30188b6a8c9:20 (host=vd1117.foo.fa.com:22000):(Total: 1s813ms, non-child: 420.854ms, % non-child: 23.21%) Hdfs split stats (:<# splits>/): 4:226/307.94 KB 6:249/339.39 KB 7:260/354.10 KB 3:232/316.79 KB 5:247/336.92 KB 0:253/344.64 KB 2:233/318.04 KB 1:244/332.64 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 94.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 447.419us - TotalStorageWaitTime: 20s278ms - TotalThreadsInvoluntaryContextSwitches: 20 (20) - TotalThreadsTotalWallClockTime: 1m59s - TotalThreadsSysTime: 94.957ms - TotalThreadsUserTime: 404.900ms - TotalThreadsVoluntaryContextSwitches: 9.46K (9463) Fragment Instance Lifecycle Timings: - ExecTime: 711.921us - ExecTreeExecTime: 5.834us - OpenTime: 1s407ms - ExecTreeOpenTime: 1s359ms - PrepareTime: 404.825ms - ExecTreePrepareTime: 7.159ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 76.267us, non-child: 76.267us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 292.11 KB/sec - OverallThroughput: 204.87 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.509us - TransmitDataRPCTime: 53.490us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.407ms, non-child: 25.407ms, % non-child: 100.00%) - CodegenTime: 633.622us - CompileTime: 4.173ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.760ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.078ms AGGREGATION_NODE (id=1):(Total: 1s366ms, non-child: 5.205ms, % non-child: 0.38%) ExecOption: Codegen Enabled - BuildTime: 1.686ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s361ms, non-child: 1s361ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:226/307.94 KB 6:249/339.39 KB 7:260/354.10 KB 3:232/316.79 KB 5:247/336.92 KB 0:253/344.64 KB 2:233/318.04 KB 1:244/332.64 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 564.21 KB, 1.56 MB, 2.55 MB - FooterProcessingTime: (Avg: 10.501ms ; Min: 5.503ms ; Max: 36.062ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714072) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714072) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714072) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 100.72 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s313ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.43 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 1m58s - MaterializeTupleTime(*): 2.514ms - ScannerThreadsSysTime: 61.962ms - ScannerThreadsUserTime: 309.915ms - ScannerThreadsVoluntaryContextSwitches: 7.37K (7369) - TotalRawHdfsReadTime(*): 25.699ms - TotalReadThroughput: 1.70 MB/sec Instance c34cc30188b6a8c9:1b (host=vhs043.foo.fa.com:22000):(Total: 1s811ms, non-child: 328.099ms, % non-child: 18.11%) Hdfs split stats (:<# splits>/): 2:248/337.92 KB 7:235/319.84 KB 1:243/330.87 KB 6:251/342.23 KB 5:238/324.08 KB 4:242/329.82 KB 3:231/315.25 KB 0:258/351.94 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 194.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 250.36 KB (256366) - PerHostPeakMemUsage: 250.36 KB (256366) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 331.104us - TotalStorageWaitTime: 22s277ms - TotalThreadsInvoluntaryContextSwitches: 8 (8) - TotalThreadsTotalWallClockTime: 2m8s - TotalThreadsSysTime: 65.969ms - TotalThreadsUserTime: 396.900ms - TotalThreadsVoluntaryContextSwitches: 9.65K (9655) Fragment Instance Lifecycle Timings: - ExecTime: 487.760us - ExecTreeExecTime: 4.277us - OpenTime: 1s497ms - ExecTreeOpenTime: 1s453ms - PrepareTime: 313.056ms - ExecTreePrepareTime: 6.621ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 55.825us, non-child: 55.825us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 412.44 KB/sec - OverallThroughput: 279.89 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.509us - TransmitDataRPCTime: 37.884us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.576ms, non-child: 23.576ms, % non-child: 100.00%) - CodegenTime: 623.668us - CompileTime: 3.985ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.476ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 12.753ms AGGREGATION_NODE (id=1):(Total: 1s459ms, non-child: 5.111ms, % non-child: 0.35%) ExecOption: Codegen Enabled - BuildTime: 1.635ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s454ms, non-child: 1s454ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:248/337.92 KB 7:235/319.84 KB 1:243/330.87 KB 6:251/342.23 KB 5:238/324.08 KB 4:242/329.82 KB 3:231/315.25 KB 0:258/351.94 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1946 out of 1946 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1946 BytesRead(500.000ms): 183.77 KB, 1.15 MB, 2.11 MB - FooterProcessingTime: (Avg: 11.512ms ; Min: 5.474ms ; Max: 105.253ms ; Number of samples: 1946) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2715581) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2715581) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2715581) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 210.00 KB (215040) - PerReadThreadRawHdfsThroughput: 102.79 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s410ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1946) - RowsReturned: 1.95K (1946) - RowsReturnedRate: 1.34 K/sec - ScanRangesComplete: 1.95K (1946) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 2m6s - MaterializeTupleTime(*): 2.298ms - ScannerThreadsSysTime: 37.973ms - ScannerThreadsUserTime: 307.913ms - ScannerThreadsVoluntaryContextSwitches: 7.57K (7566) - TotalRawHdfsReadTime(*): 25.195ms - TotalReadThroughput: 1.41 MB/sec Instance c34cc30188b6a8c9:1c (host=vd1116.foo.fa.com:22000):(Total: 1s809ms, non-child: 426.754ms, % non-child: 23.58%) Hdfs split stats (:<# splits>/): 0:212/288.98 KB 1:267/364.39 KB 3:247/336.72 KB 6:229/312.53 KB 7:255/347.60 KB 2:253/345.28 KB 4:245/333.74 KB 5:235/320.69 KB MemoryUsage(500.000ms): 40.36 KB, 184.36 KB, 192.36 KB ThreadUsage(500.000ms): 47, 89, 89 - AverageThreadTokens: 75.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 482.691us - TotalStorageWaitTime: 20s110ms - TotalThreadsInvoluntaryContextSwitches: 22 (22) - TotalThreadsTotalWallClockTime: 1m59s - TotalThreadsSysTime: 69.964ms - TotalThreadsUserTime: 426.897ms - TotalThreadsVoluntaryContextSwitches: 9.40K (9401) Fragment Instance Lifecycle Timings: - ExecTime: 664.506us - ExecTreeExecTime: 5.683us - OpenTime: 1s399ms - ExecTreeOpenTime: 1s350ms - PrepareTime: 409.950ms - ExecTreePrepareTime: 7.125ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 92.377us, non-child: 92.377us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 253.89 KB/sec - OverallThroughput: 169.14 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.185us - TransmitDataRPCTime: 61.542us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.549ms, non-child: 25.549ms, % non-child: 100.00%) - CodegenTime: 598.697us - CompileTime: 4.169ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.909ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.066ms AGGREGATION_NODE (id=1):(Total: 1s357ms, non-child: 4.081ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.585ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s353ms, non-child: 1s353ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:212/288.98 KB 1:267/364.39 KB 3:247/336.72 KB 6:229/312.53 KB 7:255/347.60 KB 2:253/345.28 KB 4:245/333.74 KB 5:235/320.69 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 0, 963.35 KB, 1.94 MB - FooterProcessingTime: (Avg: 10.418ms ; Min: 5.067ms ; Max: 33.990ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 74.00 - BytesRead: 2.59 MB (2713539) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713539) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713539) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.10 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s308ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.44 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 1m57s - MaterializeTupleTime(*): 2.297ms - ScannerThreadsSysTime: 50.967ms - ScannerThreadsUserTime: 319.914ms - ScannerThreadsVoluntaryContextSwitches: 7.30K (7302) - TotalRawHdfsReadTime(*): 25.347ms - TotalReadThroughput: 1.30 MB/sec Instance c34cc30188b6a8c9:24 (host=vc1541.foo.fa.com:22000):(Total: 1s808ms, non-child: 412.959ms, % non-child: 22.83%) Hdfs split stats (:<# splits>/): 1:247/336.75 KB 4:242/330.21 KB 5:232/316.01 KB 7:239/325.80 KB 2:249/339.52 KB 0:250/341.24 KB 3:240/327.30 KB 6:246/335.03 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 130.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 280.36 KB (287086) - PerHostPeakMemUsage: 280.36 KB (287086) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 431.695us - TotalStorageWaitTime: 21s147ms - TotalThreadsInvoluntaryContextSwitches: 34 (34) - TotalThreadsTotalWallClockTime: 2m - TotalThreadsSysTime: 74.964ms - TotalThreadsUserTime: 424.900ms - TotalThreadsVoluntaryContextSwitches: 9.51K (9513) Fragment Instance Lifecycle Timings: - ExecTime: 652.441us - ExecTreeExecTime: 6.378us - OpenTime: 1s410ms - ExecTreeOpenTime: 1s363ms - PrepareTime: 397.816ms - ExecTreePrepareTime: 7.039ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 88.146us, non-child: 88.146us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 258.32 KB/sec - OverallThroughput: 177.26 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.452us - TransmitDataRPCTime: 60.486us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.257ms, non-child: 25.257ms, % non-child: 100.00%) - CodegenTime: 583.023us - CompileTime: 4.154ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.741ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.907ms AGGREGATION_NODE (id=1):(Total: 1s370ms, non-child: 4.240ms, % non-child: 0.31%) ExecOption: Codegen Enabled - BuildTime: 1.516ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s365ms, non-child: 1s365ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:247/336.75 KB 4:242/330.21 KB 5:232/316.01 KB 7:239/325.80 KB 2:249/339.52 KB 0:250/341.24 KB 3:240/327.30 KB 6:246/335.03 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 632.72 KB, 1.57 MB, 2.57 MB - FooterProcessingTime: (Avg: 10.941ms ; Min: 5.812ms ; Max: 38.162ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2715514) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2715514) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2715514) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 240.00 KB (245760) - PerReadThreadRawHdfsThroughput: 102.59 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s318ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.42 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 13 (13) - ScannerThreadsTotalWallClockTime: 1m58s - MaterializeTupleTime(*): 2.386ms - ScannerThreadsSysTime: 46.969ms - ScannerThreadsUserTime: 328.914ms - ScannerThreadsVoluntaryContextSwitches: 7.42K (7419) - TotalRawHdfsReadTime(*): 25.242ms - TotalReadThroughput: 1.71 MB/sec Instance c34cc30188b6a8c9:61 (host=ve1127.foo.fa.com:22000):(Total: 1s799ms, non-child: 351.254ms, % non-child: 19.52%) Hdfs split stats (:<# splits>/): 7:245/334.41 KB 0:232/316.42 KB 5:245/333.91 KB 2:265/361.69 KB 3:251/342.60 KB 6:229/312.12 KB 1:237/323.33 KB 4:238/324.44 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 58.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 345.854us - TotalStorageWaitTime: 20s545ms - TotalThreadsInvoluntaryContextSwitches: 23 (23) - TotalThreadsTotalWallClockTime: 1m53s - TotalThreadsSysTime: 82.964ms - TotalThreadsUserTime: 377.913ms - TotalThreadsVoluntaryContextSwitches: 9.49K (9488) Fragment Instance Lifecycle Timings: - ExecTime: 513.975us - ExecTreeExecTime: 6.007us - OpenTime: 1s465ms - ExecTreeOpenTime: 1s417ms - PrepareTime: 334.120ms - ExecTreePrepareTime: 6.742ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 89.284us, non-child: 89.284us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 279.11 KB/sec - OverallThroughput: 175.00 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.104us - TransmitDataRPCTime: 55.982us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.398ms, non-child: 24.398ms, % non-child: 100.00%) - CodegenTime: 588.818us - CompileTime: 4.056ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.577ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.378ms AGGREGATION_NODE (id=1):(Total: 1s424ms, non-child: 3.363ms, % non-child: 0.24%) ExecOption: Codegen Enabled - BuildTime: 1.453ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s420ms, non-child: 1s420ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:245/334.41 KB 0:232/316.42 KB 5:245/333.91 KB 2:265/361.69 KB 3:251/342.60 KB 6:229/312.12 KB 1:237/323.33 KB 4:238/324.44 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1942 out of 1942 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1942 BytesRead(500.000ms): 776.01 KB, 1.79 MB, 2.57 MB - FooterProcessingTime: (Avg: 10.644ms ; Min: 5.163ms ; Max: 28.984ms ; Number of samples: 1942) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2712484) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712484) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712484) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 107.31 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s377ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1942) - RowsReturned: 1.94K (1942) - RowsReturnedRate: 1.37 K/sec - ScanRangesComplete: 1.94K (1942) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 1m52s - MaterializeTupleTime(*): 2.153ms - ScannerThreadsSysTime: 56.968ms - ScannerThreadsUserTime: 288.926ms - ScannerThreadsVoluntaryContextSwitches: 7.39K (7390) - TotalRawHdfsReadTime(*): 24.106ms - TotalReadThroughput: 1.71 MB/sec Instance c34cc30188b6a8c9:2f (host=vd1101.foo.fa.com:22000):(Total: 1s793ms, non-child: 410.692ms, % non-child: 22.90%) Hdfs split stats (:<# splits>/): 0:304/414.85 KB 5:255/347.15 KB 2:263/358.18 KB 6:293/398.78 KB 1:270/368.00 KB 3:274/373.77 KB 4:285/388.52 KB MemoryUsage(500.000ms): 28.00 KB, 166.36 KB, 166.36 KB ThreadUsage(500.000ms): 1, 89, 89 - AverageThreadTokens: 59.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 230.36 KB (235886) - PerHostPeakMemUsage: 230.36 KB (235886) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 425.797us - TotalStorageWaitTime: 17s738ms - TotalThreadsInvoluntaryContextSwitches: 31 (31) - TotalThreadsTotalWallClockTime: 1m59s - TotalThreadsSysTime: 74.961ms - TotalThreadsUserTime: 419.899ms - TotalThreadsVoluntaryContextSwitches: 9.48K (9483) Fragment Instance Lifecycle Timings: - ExecTime: 629.263us - ExecTreeExecTime: 5.968us - OpenTime: 1s398ms - ExecTreeOpenTime: 1s351ms - PrepareTime: 393.813ms - ExecTreePrepareTime: 6.816ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 64.750us, non-child: 64.750us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 342.04 KB/sec - OverallThroughput: 241.31 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.898us - TransmitDataRPCTime: 45.682us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.512ms, non-child: 24.512ms, % non-child: 100.00%) - CodegenTime: 603.061us - CompileTime: 4.107ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.797ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.242ms AGGREGATION_NODE (id=1):(Total: 1s358ms, non-child: 2.802ms, % non-child: 0.21%) ExecOption: Codegen Enabled - BuildTime: 1.284ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s355ms, non-child: 1s355ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:304/414.85 KB 5:255/347.15 KB 2:263/358.18 KB 6:293/398.78 KB 1:270/368.00 KB 3:274/373.77 KB 4:285/388.52 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 0, 929.29 KB, 1.89 MB - FooterProcessingTime: (Avg: 9.194ms ; Min: 5.410ms ; Max: 31.867ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2712840) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712840) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712840) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 7 (7) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 190.00 KB (194560) - PerReadThreadRawHdfsThroughput: 103.37 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s311ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.43 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 1m57s - MaterializeTupleTime(*): 2.412ms - ScannerThreadsSysTime: 51.965ms - ScannerThreadsUserTime: 325.914ms - ScannerThreadsVoluntaryContextSwitches: 7.30K (7303) - TotalRawHdfsReadTime(*): 25.028ms - TotalReadThroughput: 1.26 MB/sec Instance c34cc30188b6a8c9:2c (host=vhs046.foo.fa.com:22000):(Total: 1s791ms, non-child: 336.585ms, % non-child: 18.79%) Hdfs split stats (:<# splits>/): 5:227/309.50 KB 1:259/352.76 KB 0:220/300.17 KB 2:256/348.63 KB 7:224/305.46 KB 3:248/338.33 KB 4:247/336.43 KB 6:265/361.12 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 186.36 KB ThreadUsage(500.000ms): 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 501.243us - TotalStorageWaitTime: 21s285ms - TotalThreadsInvoluntaryContextSwitches: 32 (32) - TotalThreadsTotalWallClockTime: 2m5s - TotalThreadsSysTime: 65.968ms - TotalThreadsUserTime: 409.898ms - TotalThreadsVoluntaryContextSwitches: 9.52K (9516) Fragment Instance Lifecycle Timings: - ExecTime: 711.619us - ExecTreeExecTime: 5.242us - OpenTime: 1s468ms - ExecTreeOpenTime: 1s422ms - PrepareTime: 321.871ms - ExecTreePrepareTime: 6.980ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 152.979us, non-child: 152.979us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 126.56 KB/sec - OverallThroughput: 102.14 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.733us - TransmitDataRPCTime: 123.455us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.187ms, non-child: 25.187ms, % non-child: 100.00%) - CodegenTime: 570.786us - CompileTime: 4.145ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.709ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.959ms AGGREGATION_NODE (id=1):(Total: 1s429ms, non-child: 4.397ms, % non-child: 0.31%) ExecOption: Codegen Enabled - BuildTime: 1.426ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s424ms, non-child: 1s424ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:227/309.50 KB 1:259/352.76 KB 0:220/300.17 KB 2:256/348.63 KB 7:224/305.46 KB 3:248/338.33 KB 4:247/336.43 KB 6:265/361.12 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1946 out of 1946 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1946 BytesRead(500.000ms): 66.95 KB, 1005.13 KB, 1.96 MB - FooterProcessingTime: (Avg: 11.003ms ; Min: 5.524ms ; Max: 37.006ms ; Number of samples: 1946) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2716056) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2716056) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2716056) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 111.29 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s380ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1946) - RowsReturned: 1.95K (1946) - RowsReturnedRate: 1.36 K/sec - ScanRangesComplete: 1.95K (1946) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 2m3s - MaterializeTupleTime(*): 2.245ms - ScannerThreadsSysTime: 35.973ms - ScannerThreadsUserTime: 311.913ms - ScannerThreadsVoluntaryContextSwitches: 7.44K (7439) - TotalRawHdfsReadTime(*): 23.274ms - TotalReadThroughput: 1.31 MB/sec Instance c34cc30188b6a8c9:23 (host=vc1519.foo.fa.com:22000):(Total: 1s787ms, non-child: 382.182ms, % non-child: 21.38%) Hdfs split stats (:<# splits>/): 2:237/322.69 KB 6:237/323.49 KB 5:248/338.01 KB 4:269/366.32 KB 7:229/312.70 KB 3:237/322.61 KB 1:246/335.53 KB 0:241/328.30 KB MemoryUsage(500.000ms): 0, 184.36 KB, 194.36 KB ThreadUsage(500.000ms): 1, 89, 89 - AverageThreadTokens: 59.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 500.267us - TotalStorageWaitTime: 20s535ms - TotalThreadsInvoluntaryContextSwitches: 19 (19) - TotalThreadsTotalWallClockTime: 2m1s - TotalThreadsSysTime: 70.967ms - TotalThreadsUserTime: 412.899ms - TotalThreadsVoluntaryContextSwitches: 9.67K (9667) Fragment Instance Lifecycle Timings: - ExecTime: 667.986us - ExecTreeExecTime: 6.441us - OpenTime: 1s421ms - ExecTreeOpenTime: 1s373ms - PrepareTime: 365.813ms - ExecTreePrepareTime: 6.760ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 78.888us, non-child: 78.888us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 273.49 KB/sec - OverallThroughput: 198.07 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.057us - TransmitDataRPCTime: 57.132us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.737ms, non-child: 24.737ms, % non-child: 100.00%) - CodegenTime: 584.529us - CompileTime: 4.104ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.719ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.546ms AGGREGATION_NODE (id=1):(Total: 1s380ms, non-child: 4.360ms, % non-child: 0.32%) ExecOption: Codegen Enabled - BuildTime: 1.548ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s376ms, non-child: 1s376ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:237/322.69 KB 6:237/323.49 KB 5:248/338.01 KB 4:269/366.32 KB 7:229/312.70 KB 3:237/322.61 KB 1:246/335.53 KB 0:241/328.30 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 748.05 KB, 1.74 MB - FooterProcessingTime: (Avg: 10.631ms ; Min: 5.307ms ; Max: 36.208ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713239) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713239) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713239) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 101.47 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s331ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.41 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 1m59s - MaterializeTupleTime(*): 2.333ms - ScannerThreadsSysTime: 42.971ms - ScannerThreadsUserTime: 317.914ms - ScannerThreadsVoluntaryContextSwitches: 7.56K (7563) - TotalRawHdfsReadTime(*): 25.499ms - TotalReadThroughput: 1.74 MB/sec Instance c34cc30188b6a8c9:13 (host=vd1104.foo.fa.com:22000):(Total: 1s782ms, non-child: 338.916ms, % non-child: 19.01%) Hdfs split stats (:<# splits>/): 4:248/338.43 KB 7:245/333.95 KB 2:234/319.02 KB 3:210/286.15 KB 5:248/337.98 KB 6:242/330.59 KB 0:248/338.19 KB 1:269/366.39 KB MemoryUsage(500.000ms): 28.00 KB, 184.36 KB, 194.36 KB ThreadUsage(500.000ms): 1, 89, 89 - AverageThreadTokens: 59.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 232.36 KB (237934) - PerHostPeakMemUsage: 232.36 KB (237934) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 388.900us - TotalStorageWaitTime: 20s651ms - TotalThreadsInvoluntaryContextSwitches: 24 (24) - TotalThreadsTotalWallClockTime: 2m4s - TotalThreadsSysTime: 49.975ms - TotalThreadsUserTime: 433.898ms - TotalThreadsVoluntaryContextSwitches: 9.45K (9452) Fragment Instance Lifecycle Timings: - ExecTime: 622.660us - ExecTreeExecTime: 6.668us - OpenTime: 1s457ms - ExecTreeOpenTime: 1s411ms - PrepareTime: 324.024ms - ExecTreePrepareTime: 6.889ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 108.738us, non-child: 108.738us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 202.66 KB/sec - OverallThroughput: 143.69 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.384us - TransmitDataRPCTime: 77.098us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.357ms, non-child: 25.357ms, % non-child: 100.00%) - CodegenTime: 586.629us - CompileTime: 4.173ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.985ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.814ms AGGREGATION_NODE (id=1):(Total: 1s418ms, non-child: 4.226ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.726ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s413ms, non-child: 1s413ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:248/338.43 KB 7:245/333.95 KB 2:234/319.02 KB 3:210/286.15 KB 5:248/337.98 KB 6:242/330.59 KB 0:248/338.19 KB 1:269/366.39 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 0, 879.80 KB, 1.81 MB - FooterProcessingTime: (Avg: 10.691ms ; Min: 5.488ms ; Max: 36.201ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714309) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714309) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714309) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 192.00 KB (196608) - PerReadThreadRawHdfsThroughput: 103.96 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s368ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.37 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 6 (6) - ScannerThreadsTotalWallClockTime: 2m2s - MaterializeTupleTime(*): 2.439ms - ScannerThreadsSysTime: 30.978ms - ScannerThreadsUserTime: 334.913ms - ScannerThreadsVoluntaryContextSwitches: 7.33K (7333) - TotalRawHdfsReadTime(*): 24.899ms - TotalReadThroughput: 1.20 MB/sec Instance c34cc30188b6a8c9:11 (host=vd1138.foo.fa.com:22000):(Total: 1s779ms, non-child: 406.459ms, % non-child: 22.84%) Hdfs split stats (:<# splits>/): 6:232/316.60 KB 1:239/325.50 KB 0:245/333.84 KB 2:239/326.09 KB 3:270/367.82 KB 4:253/344.84 KB 5:229/312.22 KB 7:238/324.47 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 130.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 434.455us - TotalStorageWaitTime: 19s562ms - TotalThreadsInvoluntaryContextSwitches: 22 (22) - TotalThreadsTotalWallClockTime: 1m47s - TotalThreadsSysTime: 71.970ms - TotalThreadsUserTime: 435.907ms - TotalThreadsVoluntaryContextSwitches: 9.49K (9490) Fragment Instance Lifecycle Timings: - ExecTime: 689.472us - ExecTreeExecTime: 5.473us - OpenTime: 1s389ms - ExecTreeOpenTime: 1s342ms - PrepareTime: 389.819ms - ExecTreePrepareTime: 6.897ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 76.428us, non-child: 76.428us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 271.76 KB/sec - OverallThroughput: 204.44 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.780us - TransmitDataRPCTime: 57.495us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.327ms, non-child: 24.327ms, % non-child: 100.00%) - CodegenTime: 594.107us - CompileTime: 4.065ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.638ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.280ms AGGREGATION_NODE (id=1):(Total: 1s348ms, non-child: 4.476ms, % non-child: 0.33%) ExecOption: Codegen Enabled - BuildTime: 1.612ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s344ms, non-child: 1s344ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:232/316.60 KB 1:239/325.50 KB 0:245/333.84 KB 2:239/326.09 KB 3:270/367.82 KB 4:253/344.84 KB 5:229/312.22 KB 7:238/324.47 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1945 out of 1945 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1945 BytesRead(500.000ms): 398.01 KB, 1.41 MB, 2.50 MB - FooterProcessingTime: (Avg: 10.128ms ; Min: 5.553ms ; Max: 30.454ms ; Number of samples: 1945) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2715015) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2715015) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2715015) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 100.55 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s299ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.95K (1945) - RowsReturned: 1.95K (1945) - RowsReturnedRate: 1.45 K/sec - ScanRangesComplete: 1.95K (1945) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 1m46s - MaterializeTupleTime(*): 2.432ms - ScannerThreadsSysTime: 34.975ms - ScannerThreadsUserTime: 347.921ms - ScannerThreadsVoluntaryContextSwitches: 7.42K (7425) - TotalRawHdfsReadTime(*): 25.751ms - TotalReadThroughput: 1.66 MB/sec Instance c34cc30188b6a8c9:1a (host=vc1502.foo.fa.com:22000):(Total: 1s776ms, non-child: 402.757ms, % non-child: 22.67%) Hdfs split stats (:<# splits>/): 4:249/339.54 KB 3:252/344.25 KB 5:239/326.21 KB 7:236/322.01 KB 6:252/343.02 KB 0:232/316.90 KB 1:236/321.89 KB 2:245/334.00 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89 - AverageThreadTokens: 59.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 387.820us - TotalStorageWaitTime: 20s583ms - TotalThreadsInvoluntaryContextSwitches: 22 (22) - TotalThreadsTotalWallClockTime: 1m58s - TotalThreadsSysTime: 89.960ms - TotalThreadsUserTime: 399.899ms - TotalThreadsVoluntaryContextSwitches: 9.54K (9544) Fragment Instance Lifecycle Timings: - ExecTime: 591.447us - ExecTreeExecTime: 5.636us - OpenTime: 1s388ms - ExecTreeOpenTime: 1s342ms - PrepareTime: 386.984ms - ExecTreePrepareTime: 6.791ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 74.769us, non-child: 74.769us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 278.11 KB/sec - OverallThroughput: 208.98 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.640us - TransmitDataRPCTime: 56.183us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.458ms, non-child: 24.458ms, % non-child: 100.00%) - CodegenTime: 594.978us - CompileTime: 4.080ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.781ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.231ms AGGREGATION_NODE (id=1):(Total: 1s349ms, non-child: 4.713ms, % non-child: 0.35%) ExecOption: Codegen Enabled - BuildTime: 1.503ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s344ms, non-child: 1s344ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:249/339.54 KB 3:252/344.25 KB 5:239/326.21 KB 7:236/322.01 KB 6:252/343.02 KB 0:232/316.90 KB 1:236/321.89 KB 2:245/334.00 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1941 out of 1941 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1941 BytesRead(500.000ms): 823.87 KB, 1.77 MB - FooterProcessingTime: (Avg: 10.671ms ; Min: 5.603ms ; Max: 29.493ms ; Number of samples: 1941) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2711375) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2711375) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2711375) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 200.00 KB (204800) - PerReadThreadRawHdfsThroughput: 102.12 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s298ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1941) - RowsReturned: 1.94K (1941) - RowsReturnedRate: 1.44 K/sec - ScanRangesComplete: 1.94K (1941) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 1m56s - MaterializeTupleTime(*): 2.321ms - ScannerThreadsSysTime: 61.964ms - ScannerThreadsUserTime: 298.914ms - ScannerThreadsVoluntaryContextSwitches: 7.46K (7458) - TotalRawHdfsReadTime(*): 25.320ms - TotalReadThroughput: 1.77 MB/sec Instance c34cc30188b6a8c9:2e (host=vhs027.foo.fa.com:22000):(Total: 1s762ms, non-child: 401.146ms, % non-child: 22.75%) Hdfs split stats (:<# splits>/): 7:216/294.76 KB 5:249/339.22 KB 6:255/348.53 KB 2:246/335.31 KB 4:251/342.10 KB 0:253/344.71 KB 3:240/327.07 KB 1:233/318.02 KB MemoryUsage(500.000ms): 0, 184.36 KB, 192.36 KB, 130.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 411.969us - TotalStorageWaitTime: 20s447ms - TotalThreadsInvoluntaryContextSwitches: 33 (33) - TotalThreadsTotalWallClockTime: 1m57s - TotalThreadsSysTime: 78.964ms - TotalThreadsUserTime: 421.899ms - TotalThreadsVoluntaryContextSwitches: 9.02K (9023) Fragment Instance Lifecycle Timings: - ExecTime: 584.546us - ExecTreeExecTime: 4.196us - OpenTime: 1s386ms - ExecTreeOpenTime: 1s329ms - PrepareTime: 376.002ms - ExecTreePrepareTime: 7.298ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 87.219us, non-child: 87.219us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 249.23 KB/sec - OverallThroughput: 179.15 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.158us - TransmitDataRPCTime: 62.694us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.205ms, non-child: 25.205ms, % non-child: 100.00%) - CodegenTime: 576.548us - CompileTime: 4.029ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.583ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.206ms AGGREGATION_NODE (id=1):(Total: 1s336ms, non-child: 3.020ms, % non-child: 0.23%) ExecOption: Codegen Enabled - BuildTime: 1.329ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s333ms, non-child: 1s333ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:216/294.76 KB 5:249/339.22 KB 6:255/348.53 KB 2:246/335.31 KB 4:251/342.10 KB 0:253/344.71 KB 3:240/327.07 KB 1:233/318.02 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 477.34 KB, 1.46 MB, 2.52 MB - FooterProcessingTime: (Avg: 10.584ms ; Min: 5.037ms ; Max: 36.233ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713312) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713312) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713312) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 118.66 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s290ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.46 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 14 (14) - ScannerThreadsTotalWallClockTime: 1m55s - MaterializeTupleTime(*): 2.198ms - ScannerThreadsSysTime: 50.968ms - ScannerThreadsUserTime: 317.915ms - ScannerThreadsVoluntaryContextSwitches: 6.90K (6898) - TotalRawHdfsReadTime(*): 21.806ms - TotalReadThroughput: 1.68 MB/sec Instance c34cc30188b6a8c9:5a (host=vd1338.foo.fa.com:22000):(Total: 1s754ms, non-child: 354.975ms, % non-child: 20.23%) Hdfs split stats (:<# splits>/): 5:163/221.94 KB 7:174/237.26 KB 4:162/221.17 KB 1:178/243.22 KB 3:164/223.31 KB 0:180/245.43 KB 2:168/228.97 KB 6:171/233.26 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 186.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 379.406us - TotalStorageWaitTime: 21s083ms - TotalThreadsInvoluntaryContextSwitches: 19 (19) - TotalThreadsTotalWallClockTime: 1m50s - TotalThreadsSysTime: 60.975ms - TotalThreadsUserTime: 300.913ms - TotalThreadsVoluntaryContextSwitches: 6.60K (6604) Fragment Instance Lifecycle Timings: - ExecTime: 538.504us - ExecTreeExecTime: 3.390us - OpenTime: 1s415ms - ExecTreeOpenTime: 1s370ms - PrepareTime: 338.082ms - ExecTreePrepareTime: 4.907ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 75.937us, non-child: 75.937us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 289.13 KB/sec - OverallThroughput: 205.76 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 8.806us - TransmitDataRPCTime: 54.041us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.102ms, non-child: 24.102ms, % non-child: 100.00%) - CodegenTime: 536.239us - CompileTime: 3.997ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.490ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.243ms AGGREGATION_NODE (id=1):(Total: 1s375ms, non-child: 4.451ms, % non-child: 0.32%) ExecOption: Codegen Enabled - BuildTime: 1.170ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s370ms, non-child: 1s370ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:163/221.94 KB 7:174/237.26 KB 4:162/221.17 KB 1:178/243.22 KB 3:164/223.31 KB 0:180/245.43 KB 2:168/228.97 KB 6:171/233.26 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1360 out of 1360 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1360 BytesRead(500.000ms): 486.99 KB, 1.25 MB, 1.68 MB - FooterProcessingTime: (Avg: 15.573ms ; Min: 5.697ms ; Max: 214.749ms ; Number of samples: 1360) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1899060) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1899060) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1899060) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.35 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s336ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1360) - RowsReturned: 1.36K (1360) - RowsReturnedRate: 992.00 /sec - ScanRangesComplete: 1.36K (1360) - ScannerThreadsInvoluntaryContextSwitches: 5 (5) - ScannerThreadsTotalWallClockTime: 1m48s - MaterializeTupleTime(*): 1.695ms - ScannerThreadsSysTime: 35.979ms - ScannerThreadsUserTime: 211.926ms - ScannerThreadsVoluntaryContextSwitches: 5.13K (5126) - TotalRawHdfsReadTime(*): 17.694ms - TotalReadThroughput: 1.12 MB/sec Instance c34cc30188b6a8c9:1d (host=vhs044.foo.fa.com:22000):(Total: 1s753ms, non-child: 343.066ms, % non-child: 19.57%) Hdfs split stats (:<# splits>/): 0:232/315.82 KB 3:237/323.51 KB 1:254/346.38 KB 4:259/352.46 KB 7:238/324.28 KB 5:233/317.91 KB 6:255/347.97 KB 2:235/320.69 KB MemoryUsage(500.000ms): 20.00 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89 - AverageThreadTokens: 59.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 527.086us - TotalStorageWaitTime: 20s885ms - TotalThreadsInvoluntaryContextSwitches: 27 (27) - TotalThreadsTotalWallClockTime: 2m1s - TotalThreadsSysTime: 82.959ms - TotalThreadsUserTime: 424.897ms - TotalThreadsVoluntaryContextSwitches: 9.41K (9411) Fragment Instance Lifecycle Timings: - ExecTime: 767.519us - ExecTreeExecTime: 5.219us - OpenTime: 1s425ms - ExecTreeOpenTime: 1s378ms - PrepareTime: 327.315ms - ExecTreePrepareTime: 6.795ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 88.947us, non-child: 88.947us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 228.65 KB/sec - OverallThroughput: 175.67 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.907us - TransmitDataRPCTime: 68.337us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.091ms, non-child: 25.091ms, % non-child: 100.00%) - CodegenTime: 592.790us - CompileTime: 4.185ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.875ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.642ms AGGREGATION_NODE (id=1):(Total: 1s385ms, non-child: 4.161ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.435ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s380ms, non-child: 1s380ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:232/315.82 KB 3:237/323.51 KB 1:254/346.38 KB 4:259/352.46 KB 7:238/324.28 KB 5:233/317.91 KB 6:255/347.97 KB 2:235/320.69 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 0, 781.28 KB, 1.79 MB - FooterProcessingTime: (Avg: 10.818ms ; Min: 5.451ms ; Max: 39.617ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2712607) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712607) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712607) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 101.91 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s334ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.41 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 8 (8) - ScannerThreadsTotalWallClockTime: 2m - MaterializeTupleTime(*): 2.370ms - ScannerThreadsSysTime: 51.964ms - ScannerThreadsUserTime: 322.913ms - ScannerThreadsVoluntaryContextSwitches: 7.31K (7310) - TotalRawHdfsReadTime(*): 25.385ms - TotalReadThroughput: 1.19 MB/sec Instance c34cc30188b6a8c9:54 (host=ve1138.foo.fa.com:22000):(Total: 1s741ms, non-child: 426.902ms, % non-child: 24.51%) Hdfs split stats (:<# splits>/): 6:260/354.64 KB 4:229/312.34 KB 5:245/333.92 KB 1:250/341.36 KB 7:250/341.03 KB 0:222/302.91 KB 2:252/343.65 KB 3:234/318.80 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81 - AverageThreadTokens: 54.33 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 368.673us - TotalStorageWaitTime: 19s367ms - TotalThreadsInvoluntaryContextSwitches: 26 (26) - TotalThreadsTotalWallClockTime: 1m42s - TotalThreadsSysTime: 70.965ms - TotalThreadsUserTime: 421.905ms - TotalThreadsVoluntaryContextSwitches: 9.46K (9457) Fragment Instance Lifecycle Timings: - ExecTime: 534.360us - ExecTreeExecTime: 6.238us - OpenTime: 1s330ms - ExecTreeOpenTime: 1s282ms - PrepareTime: 410.278ms - ExecTreePrepareTime: 7.139ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 83.072us, non-child: 83.072us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 244.08 KB/sec - OverallThroughput: 188.09 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.468us - TransmitDataRPCTime: 64.016us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.109ms, non-child: 25.109ms, % non-child: 100.00%) - CodegenTime: 593.894us - CompileTime: 4.090ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.772ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.843ms AGGREGATION_NODE (id=1):(Total: 1s289ms, non-child: 4.303ms, % non-child: 0.33%) ExecOption: Codegen Enabled - BuildTime: 1.539ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s285ms, non-child: 1s285ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:260/354.64 KB 4:229/312.34 KB 5:245/333.92 KB 1:250/341.36 KB 7:250/341.03 KB 0:222/302.91 KB 2:252/343.65 KB 3:234/318.80 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1942 out of 1942 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1942 BytesRead(500.000ms): 586.83 KB, 1.61 MB - FooterProcessingTime: (Avg: 10.042ms ; Min: 5.262ms ; Max: 33.671ms ; Number of samples: 1942) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2712220) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712220) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712220) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 105.33 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s239ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1942) - RowsReturned: 1.94K (1942) - RowsReturnedRate: 1.51 K/sec - ScanRangesComplete: 1.94K (1942) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 1m41s - MaterializeTupleTime(*): 2.326ms - ScannerThreadsSysTime: 44.969ms - ScannerThreadsUserTime: 316.921ms - ScannerThreadsVoluntaryContextSwitches: 7.35K (7354) - TotalRawHdfsReadTime(*): 24.556ms - TotalReadThroughput: 1.61 MB/sec Instance c34cc30188b6a8c9:46 (host=vd1121.foo.fa.com:22000):(Total: 1s736ms, non-child: 331.408ms, % non-child: 19.09%) Hdfs split stats (:<# splits>/): 4:250/341.28 KB 6:238/324.42 KB 7:239/325.97 KB 5:241/328.51 KB 1:247/336.77 KB 0:246/335.64 KB 2:246/335.90 KB 3:235/320.78 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 242.36 KB (248174) - PerHostPeakMemUsage: 242.36 KB (248174) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 381.736us - TotalStorageWaitTime: 21s366ms - TotalThreadsInvoluntaryContextSwitches: 19 (19) - TotalThreadsTotalWallClockTime: 2m1s - TotalThreadsSysTime: 61.972ms - TotalThreadsUserTime: 423.897ms - TotalThreadsVoluntaryContextSwitches: 9.53K (9534) Fragment Instance Lifecycle Timings: - ExecTime: 597.157us - ExecTreeExecTime: 6.724us - OpenTime: 1s419ms - ExecTreeOpenTime: 1s374ms - PrepareTime: 316.332ms - ExecTreePrepareTime: 6.297ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 81.552us, non-child: 81.552us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 253.26 KB/sec - OverallThroughput: 191.59 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.314us - TransmitDataRPCTime: 61.696us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.019ms, non-child: 24.019ms, % non-child: 100.00%) - CodegenTime: 552.728us - CompileTime: 4.051ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.563ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.021ms AGGREGATION_NODE (id=1):(Total: 1s380ms, non-child: 5.341ms, % non-child: 0.39%) ExecOption: Codegen Enabled - BuildTime: 1.668ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s375ms, non-child: 1s375ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:250/341.28 KB 6:238/324.42 KB 7:239/325.97 KB 5:241/328.51 KB 1:247/336.77 KB 0:246/335.64 KB 2:246/335.90 KB 3:235/320.78 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1942 out of 1942 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1942 BytesRead(500.000ms): 340.86 KB, 1.27 MB, 2.25 MB - FooterProcessingTime: (Avg: 11.070ms ; Min: 5.817ms ; Max: 34.056ms ; Number of samples: 1942) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2712850) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712850) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712850) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 202.00 KB (206848) - PerReadThreadRawHdfsThroughput: 105.16 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s331ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1942) - RowsReturned: 1.94K (1942) - RowsReturnedRate: 1.41 K/sec - ScanRangesComplete: 1.94K (1942) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 1m59s - MaterializeTupleTime(*): 2.408ms - ScannerThreadsSysTime: 36.975ms - ScannerThreadsUserTime: 325.912ms - ScannerThreadsVoluntaryContextSwitches: 7.45K (7449) - TotalRawHdfsReadTime(*): 24.603ms - TotalReadThroughput: 1.50 MB/sec Instance c34cc30188b6a8c9:1f (host=vc1506.foo.fa.com:22000):(Total: 1s735ms, non-child: 412.199ms, % non-child: 23.76%) Hdfs split stats (:<# splits>/): 2:243/331.87 KB 0:246/335.54 KB 7:239/326.07 KB 4:250/340.39 KB 3:245/333.85 KB 5:238/324.10 KB 1:248/338.55 KB 6:235/320.39 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89 - AverageThreadTokens: 59.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 407.975us - TotalStorageWaitTime: 19s999ms - TotalThreadsInvoluntaryContextSwitches: 15 (15) - TotalThreadsTotalWallClockTime: 1m53s - TotalThreadsSysTime: 72.967ms - TotalThreadsUserTime: 418.898ms - TotalThreadsVoluntaryContextSwitches: 9.52K (9521) Fragment Instance Lifecycle Timings: - ExecTime: 686.212us - ExecTreeExecTime: 5.706us - OpenTime: 1s339ms - ExecTreeOpenTime: 1s290ms - PrepareTime: 394.940ms - ExecTreePrepareTime: 6.908ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 91.332us, non-child: 91.332us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 245.09 KB/sec - OverallThroughput: 171.08 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.573us - TransmitDataRPCTime: 63.751us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.896ms, non-child: 24.896ms, % non-child: 100.00%) - CodegenTime: 584.464us - CompileTime: 3.985ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.596ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.899ms AGGREGATION_NODE (id=1):(Total: 1s297ms, non-child: 4.571ms, % non-child: 0.35%) ExecOption: Codegen Enabled - BuildTime: 1.637ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s293ms, non-child: 1s293ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:243/331.87 KB 0:246/335.54 KB 7:239/326.07 KB 4:250/340.39 KB 3:245/333.85 KB 5:238/324.10 KB 1:248/338.55 KB 6:235/320.39 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 652.96 KB, 1.71 MB - FooterProcessingTime: (Avg: 10.352ms ; Min: 5.585ms ; Max: 31.036ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714375) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714375) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714375) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 110.76 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s248ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.50 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 1m52s - MaterializeTupleTime(*): 2.369ms - ScannerThreadsSysTime: 45.971ms - ScannerThreadsUserTime: 317.914ms - ScannerThreadsVoluntaryContextSwitches: 7.42K (7419) - TotalRawHdfsReadTime(*): 23.372ms - TotalReadThroughput: 1.71 MB/sec Instance c34cc30188b6a8c9:12 (host=ve1132.foo.fa.com:22000):(Total: 1s729ms, non-child: 330.152ms, % non-child: 19.09%) Hdfs split stats (:<# splits>/): 7:259/353.05 KB 6:250/340.28 KB 0:243/331.30 KB 4:237/323.05 KB 2:242/329.88 KB 5:237/323.32 KB 3:237/323.33 KB 1:239/325.63 KB MemoryUsage(500.000ms): 0, 184.36 KB, 194.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 345.031us - TotalStorageWaitTime: 20s650ms - TotalThreadsInvoluntaryContextSwitches: 15 (15) - TotalThreadsTotalWallClockTime: 1m49s - TotalThreadsSysTime: 68.967ms - TotalThreadsUserTime: 429.904ms - TotalThreadsVoluntaryContextSwitches: 9.52K (9519) Fragment Instance Lifecycle Timings: - ExecTime: 539.262us - ExecTreeExecTime: 5.202us - OpenTime: 1s413ms - ExecTreeOpenTime: 1s367ms - PrepareTime: 315.955ms - ExecTreePrepareTime: 6.869ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 91.798us, non-child: 91.798us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 222.85 KB/sec - OverallThroughput: 170.21 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.840us - TransmitDataRPCTime: 70.115us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.847ms, non-child: 24.847ms, % non-child: 100.00%) - CodegenTime: 561.814us - CompileTime: 4.225ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.663ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.555ms AGGREGATION_NODE (id=1):(Total: 1s374ms, non-child: 5.163ms, % non-child: 0.38%) ExecOption: Codegen Enabled - BuildTime: 1.607ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s369ms, non-child: 1s369ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:259/353.05 KB 6:250/340.28 KB 0:243/331.30 KB 4:237/323.05 KB 2:242/329.88 KB 5:237/323.32 KB 3:237/323.33 KB 1:239/325.63 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 313.90 KB, 1.34 MB, 2.31 MB - FooterProcessingTime: (Avg: 10.692ms ; Min: 5.279ms ; Max: 35.530ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2713434) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713434) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713434) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 101.08 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s321ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.42 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 1m48s - MaterializeTupleTime(*): 2.333ms - ScannerThreadsSysTime: 46.971ms - ScannerThreadsUserTime: 321.921ms - ScannerThreadsVoluntaryContextSwitches: 7.41K (7411) - TotalRawHdfsReadTime(*): 25.601ms - TotalReadThroughput: 1.54 MB/sec Instance c34cc30188b6a8c9:74 (host=vd1331.foo.fa.com:22000):(Total: 1s723ms, non-child: 373.687ms, % non-child: 21.69%) Hdfs split stats (:<# splits>/): 4:169/230.21 KB 6:159/216.68 KB 0:181/246.87 KB 2:178/242.83 KB 7:176/240.03 KB 5:164/223.71 KB 1:170/231.84 KB 3:164/223.52 KB MemoryUsage(500.000ms): 28.00 KB, 184.36 KB, 194.36 KB ThreadUsage(500.000ms): 1, 81, 81 - AverageThreadTokens: 54.33 - BloomFilterBytes: 0 - PeakMemoryUsage: 264.36 KB (270702) - PerHostPeakMemUsage: 264.36 KB (270702) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 380.185us - TotalStorageWaitTime: 20s283ms - TotalThreadsInvoluntaryContextSwitches: 15 (15) - TotalThreadsTotalWallClockTime: 1m45s - TotalThreadsSysTime: 60.974ms - TotalThreadsUserTime: 303.913ms - TotalThreadsVoluntaryContextSwitches: 6.58K (6583) Fragment Instance Lifecycle Timings: - ExecTime: 552.392us - ExecTreeExecTime: 6.271us - OpenTime: 1s366ms - ExecTreeOpenTime: 1s319ms - PrepareTime: 356.490ms - ExecTreePrepareTime: 4.776ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 77.641us, non-child: 77.641us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 288.29 KB/sec - OverallThroughput: 201.25 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.770us - TransmitDataRPCTime: 54.199us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.699ms, non-child: 24.699ms, % non-child: 100.00%) - CodegenTime: 577.145us - CompileTime: 4.033ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.585ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.723ms AGGREGATION_NODE (id=1):(Total: 1s324ms, non-child: 4.244ms, % non-child: 0.32%) ExecOption: Codegen Enabled - BuildTime: 1.161ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s320ms, non-child: 1s320ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:169/230.21 KB 6:159/216.68 KB 0:181/246.87 KB 2:178/242.83 KB 7:176/240.03 KB 5:164/223.71 KB 1:170/231.84 KB 3:164/223.52 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1361 out of 1361 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1361 BytesRead(500.000ms): 0, 594.60 KB, 1.27 MB - FooterProcessingTime: (Avg: 14.972ms ; Min: 6.169ms ; Max: 33.650ms ; Number of samples: 1361) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1900234) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1900234) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1900234) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 224.00 KB (229376) - PerReadThreadRawHdfsThroughput: 94.84 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s285ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1361) - RowsReturned: 1.36K (1361) - RowsReturnedRate: 1.03 K/sec - ScanRangesComplete: 1.36K (1361) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 1m44s - MaterializeTupleTime(*): 1.628ms - ScannerThreadsSysTime: 38.978ms - ScannerThreadsUserTime: 215.926ms - ScannerThreadsVoluntaryContextSwitches: 5.13K (5132) - TotalRawHdfsReadTime(*): 19.107ms - TotalReadThroughput: 869.01 KB/sec Instance c34cc30188b6a8c9:10 (host=vd1124.foo.fa.com:22000):(Total: 1s717ms, non-child: 409.246ms, % non-child: 23.83%) Hdfs split stats (:<# splits>/): 7:230/313.77 KB 3:252/343.65 KB 4:244/332.19 KB 6:253/345.32 KB 5:249/339.45 KB 1:208/284.08 KB 0:253/345.27 KB 2:254/346.67 KB MemoryUsage(500.000ms): 0, 186.36 KB, 184.36 KB, 130.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 373.128us - TotalStorageWaitTime: 19s343ms - TotalThreadsInvoluntaryContextSwitches: 29 (29) - TotalThreadsTotalWallClockTime: 1m52s - TotalThreadsSysTime: 108.951ms - TotalThreadsUserTime: 412.902ms - TotalThreadsVoluntaryContextSwitches: 9.45K (9445) Fragment Instance Lifecycle Timings: - ExecTime: 552.327us - ExecTreeExecTime: 3.772us - OpenTime: 1s323ms - ExecTreeOpenTime: 1s276ms - PrepareTime: 392.612ms - ExecTreePrepareTime: 6.796ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 62.955us, non-child: 62.955us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 331.56 KB/sec - OverallThroughput: 248.19 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 8.907us - TransmitDataRPCTime: 47.125us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.070ms, non-child: 24.070ms, % non-child: 100.00%) - CodegenTime: 606.765us - CompileTime: 3.922ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.488ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.293ms AGGREGATION_NODE (id=1):(Total: 1s283ms, non-child: 4.022ms, % non-child: 0.31%) ExecOption: Codegen Enabled - BuildTime: 1.469ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s279ms, non-child: 1s279ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:230/313.77 KB 3:252/343.65 KB 4:244/332.19 KB 6:253/345.32 KB 5:249/339.45 KB 1:208/284.08 KB 0:253/345.27 KB 2:254/346.67 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 427.14 KB, 1.44 MB, 2.52 MB - FooterProcessingTime: (Avg: 10.025ms ; Min: 5.021ms ; Max: 32.685ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714025) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714025) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714025) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 103.92 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s233ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.52 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 15 (15) - ScannerThreadsTotalWallClockTime: 1m51s - MaterializeTupleTime(*): 2.344ms - ScannerThreadsSysTime: 69.957ms - ScannerThreadsUserTime: 327.915ms - ScannerThreadsVoluntaryContextSwitches: 7.34K (7339) - TotalRawHdfsReadTime(*): 24.905ms - TotalReadThroughput: 1.68 MB/sec Instance c34cc30188b6a8c9:45 (host=ve1126.foo.fa.com:22000):(Total: 1s711ms, non-child: 333.780ms, % non-child: 19.50%) Hdfs split stats (:<# splits>/): 3:245/333.91 KB 6:238/324.61 KB 1:264/359.92 KB 5:245/334.04 KB 0:232/316.44 KB 2:225/306.71 KB 7:254/345.92 KB 4:241/328.33 KB MemoryUsage(500.000ms): 40.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81 - AverageThreadTokens: 54.33 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 467.373us - TotalStorageWaitTime: 20s315ms - TotalThreadsInvoluntaryContextSwitches: 12 (12) - TotalThreadsTotalWallClockTime: 1m48s - TotalThreadsSysTime: 61.969ms - TotalThreadsUserTime: 443.906ms - TotalThreadsVoluntaryContextSwitches: 9.41K (9408) Fragment Instance Lifecycle Timings: - ExecTime: 669.251us - ExecTreeExecTime: 6.451us - OpenTime: 1s392ms - ExecTreeOpenTime: 1s346ms - PrepareTime: 318.782ms - ExecTreePrepareTime: 6.618ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 98.365us, non-child: 98.365us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 219.46 KB/sec - OverallThroughput: 158.85 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.093us - TransmitDataRPCTime: 71.197us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.685ms, non-child: 24.685ms, % non-child: 100.00%) - CodegenTime: 567.289us - CompileTime: 4.098ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.682ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.472ms AGGREGATION_NODE (id=1):(Total: 1s353ms, non-child: 4.435ms, % non-child: 0.33%) ExecOption: Codegen Enabled - BuildTime: 1.520ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s348ms, non-child: 1s348ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:245/333.91 KB 6:238/324.61 KB 1:264/359.92 KB 5:245/334.04 KB 0:232/316.44 KB 2:225/306.71 KB 7:254/345.92 KB 4:241/328.33 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 0, 904.12 KB, 1.92 MB - FooterProcessingTime: (Avg: 10.522ms ; Min: 4.839ms ; Max: 33.373ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 53.33 - BytesRead: 2.59 MB (2713495) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713495) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713495) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 106.15 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s304ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.44 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 1m46s - MaterializeTupleTime(*): 2.429ms - ScannerThreadsSysTime: 36.973ms - ScannerThreadsUserTime: 347.921ms - ScannerThreadsVoluntaryContextSwitches: 7.33K (7334) - TotalRawHdfsReadTime(*): 24.378ms - TotalReadThroughput: 1.28 MB/sec Instance c34cc30188b6a8c9:47 (host=vhs045.foo.fa.com:22000):(Total: 1s711ms, non-child: 332.698ms, % non-child: 19.44%) Hdfs split stats (:<# splits>/): 4:238/324.84 KB 6:246/335.11 KB 1:232/317.01 KB 0:244/332.64 KB 2:248/337.85 KB 3:255/347.69 KB 7:234/318.78 KB 5:247/336.92 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89 - AverageThreadTokens: 59.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 272.36 KB (278894) - PerHostPeakMemUsage: 272.36 KB (278894) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 429.934us - TotalStorageWaitTime: 20s766ms - TotalThreadsInvoluntaryContextSwitches: 19 (19) - TotalThreadsTotalWallClockTime: 1m58s - TotalThreadsSysTime: 65.970ms - TotalThreadsUserTime: 390.902ms - TotalThreadsVoluntaryContextSwitches: 9.46K (9461) Fragment Instance Lifecycle Timings: - ExecTime: 611.863us - ExecTreeExecTime: 7.848us - OpenTime: 1s393ms - ExecTreeOpenTime: 1s348ms - PrepareTime: 317.078ms - ExecTreePrepareTime: 6.329ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 64.617us, non-child: 64.617us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 348.75 KB/sec - OverallThroughput: 241.81 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.418us - TransmitDataRPCTime: 44.803us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.952ms, non-child: 23.952ms, % non-child: 100.00%) - CodegenTime: 582.413us - CompileTime: 4.062ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.508ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.012ms AGGREGATION_NODE (id=1):(Total: 1s354ms, non-child: 3.605ms, % non-child: 0.27%) ExecOption: Codegen Enabled - BuildTime: 1.378ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s350ms, non-child: 1s350ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:238/324.84 KB 6:246/335.11 KB 1:232/317.01 KB 0:244/332.64 KB 2:248/337.85 KB 3:255/347.69 KB 7:234/318.78 KB 5:247/336.92 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 643.35 KB, 1.66 MB - FooterProcessingTime: (Avg: 10.746ms ; Min: 5.194ms ; Max: 35.351ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714452) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714452) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714452) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 232.00 KB (237568) - PerReadThreadRawHdfsThroughput: 111.35 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s309ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.44 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 1m57s - MaterializeTupleTime(*): 2.321ms - ScannerThreadsSysTime: 45.973ms - ScannerThreadsUserTime: 294.917ms - ScannerThreadsVoluntaryContextSwitches: 7.39K (7386) - TotalRawHdfsReadTime(*): 23.247ms - TotalReadThroughput: 1.66 MB/sec Instance c34cc30188b6a8c9:f (host=vd1113.foo.fa.com:22000):(Total: 1s709ms, non-child: 392.683ms, % non-child: 22.97%) Hdfs split stats (:<# splits>/): 7:233/317.63 KB 1:246/335.39 KB 6:248/338.33 KB 0:243/331.70 KB 3:234/318.85 KB 4:238/324.89 KB 5:258/352.17 KB 2:242/329.89 KB MemoryUsage(500.000ms): 0, 194.36 KB, 184.36 KB, 166.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 266.36 KB (272750) - PerHostPeakMemUsage: 266.36 KB (272750) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 499.916us - TotalStorageWaitTime: 19s535ms - TotalThreadsInvoluntaryContextSwitches: 23 (23) - TotalThreadsTotalWallClockTime: 1m53s - TotalThreadsSysTime: 63.977ms - TotalThreadsUserTime: 384.901ms - TotalThreadsVoluntaryContextSwitches: 9.56K (9564) Fragment Instance Lifecycle Timings: - ExecTime: 717.393us - ExecTreeExecTime: 5.813us - OpenTime: 1s330ms - ExecTreeOpenTime: 1s284ms - PrepareTime: 378.637ms - ExecTreePrepareTime: 6.821ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 127.346us, non-child: 127.346us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 198.03 KB/sec - OverallThroughput: 122.70 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 15.928us - TransmitDataRPCTime: 78.901us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.402ms, non-child: 25.402ms, % non-child: 100.00%) - CodegenTime: 824.985us - CompileTime: 4.152ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.726ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.894ms AGGREGATION_NODE (id=1):(Total: 1s291ms, non-child: 4.250ms, % non-child: 0.33%) ExecOption: Codegen Enabled - BuildTime: 1.513ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s287ms, non-child: 1s287ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:233/317.63 KB 1:246/335.39 KB 6:248/338.33 KB 0:243/331.70 KB 3:234/318.85 KB 4:238/324.89 KB 5:258/352.17 KB 2:242/329.89 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1942 out of 1942 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1942 BytesRead(500.000ms): 395.38 KB, 1.48 MB, 2.50 MB - FooterProcessingTime: (Avg: 10.119ms ; Min: 5.000ms ; Max: 31.143ms ; Number of samples: 1942) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2712412) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2712412) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2712412) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 226.00 KB (231424) - PerReadThreadRawHdfsThroughput: 125.22 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s241ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1942) - RowsReturned: 1.94K (1942) - RowsReturnedRate: 1.51 K/sec - ScanRangesComplete: 1.94K (1942) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 1m51s - MaterializeTupleTime(*): 2.100ms - ScannerThreadsSysTime: 32.981ms - ScannerThreadsUserTime: 294.915ms - ScannerThreadsVoluntaryContextSwitches: 7.46K (7457) - TotalRawHdfsReadTime(*): 20.657ms - TotalReadThroughput: 1.66 MB/sec Instance c34cc30188b6a8c9:50 (host=vd1329.foo.fa.com:22000):(Total: 1s708ms, non-child: 327.442ms, % non-child: 19.17%) Hdfs split stats (:<# splits>/): 1:181/246.44 KB 6:165/224.83 KB 0:163/222.35 KB 5:168/228.98 KB 3:175/238.44 KB 7:168/229.09 KB 2:181/246.42 KB 4:160/218.34 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 86.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 240.36 KB (246126) - PerHostPeakMemUsage: 240.36 KB (246126) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 393.122us - TotalStorageWaitTime: 20s779ms - TotalThreadsInvoluntaryContextSwitches: 18 (18) - TotalThreadsTotalWallClockTime: 1m48s - TotalThreadsSysTime: 64.973ms - TotalThreadsUserTime: 314.911ms - TotalThreadsVoluntaryContextSwitches: 6.65K (6646) Fragment Instance Lifecycle Timings: - ExecTime: 647.350us - ExecTreeExecTime: 6.165us - OpenTime: 1s396ms - ExecTreeOpenTime: 1s351ms - PrepareTime: 310.925ms - ExecTreePrepareTime: 4.752ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 91.899us, non-child: 91.899us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 269.80 KB/sec - OverallThroughput: 170.02 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 14.744us - TransmitDataRPCTime: 57.914us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.470ms, non-child: 24.470ms, % non-child: 100.00%) - CodegenTime: 564.716us - CompileTime: 4.090ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.566ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.441ms AGGREGATION_NODE (id=1):(Total: 1s356ms, non-child: 5.033ms, % non-child: 0.37%) ExecOption: Codegen Enabled - BuildTime: 1.271ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s351ms, non-child: 1s351ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:181/246.44 KB 6:165/224.83 KB 0:163/222.35 KB 5:168/228.98 KB 3:175/238.44 KB 7:168/229.09 KB 2:181/246.42 KB 4:160/218.34 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1361 out of 1361 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1361 BytesRead(500.000ms): 401.25 KB, 947.61 KB, 1.80 MB - FooterProcessingTime: (Avg: 15.339ms ; Min: 5.596ms ; Max: 35.883ms ; Number of samples: 1361) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1899406) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1899406) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1899406) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 200.00 KB (204800) - PerReadThreadRawHdfsThroughput: 99.92 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s315ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1361) - RowsReturned: 1.36K (1361) - RowsReturnedRate: 1.01 K/sec - ScanRangesComplete: 1.36K (1361) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 1m47s - MaterializeTupleTime(*): 1.685ms - ScannerThreadsSysTime: 37.977ms - ScannerThreadsUserTime: 227.924ms - ScannerThreadsVoluntaryContextSwitches: 5.18K (5178) - TotalRawHdfsReadTime(*): 18.128ms - TotalReadThroughput: 1.20 MB/sec Instance c34cc30188b6a8c9:e (host=vd1125.foo.fa.com:22000):(Total: 1s705ms, non-child: 352.360ms, % non-child: 20.66%) Hdfs split stats (:<# splits>/): 1:254/346.54 KB 6:246/335.51 KB 2:237/323.14 KB 0:254/345.99 KB 3:263/358.55 KB 5:214/291.77 KB 4:255/347.83 KB 7:221/301.29 KB MemoryUsage(500.000ms): 113.50 KB, 186.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89 - AverageThreadTokens: 59.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 258.36 KB (264558) - PerHostPeakMemUsage: 258.36 KB (264558) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 514.205us - TotalStorageWaitTime: 19s815ms - TotalThreadsInvoluntaryContextSwitches: 25 (25) - TotalThreadsTotalWallClockTime: 1m56s - TotalThreadsSysTime: 69.967ms - TotalThreadsUserTime: 418.899ms - TotalThreadsVoluntaryContextSwitches: 9.56K (9556) Fragment Instance Lifecycle Timings: - ExecTime: 725.313us - ExecTreeExecTime: 4.738us - OpenTime: 1s369ms - ExecTreeOpenTime: 1s321ms - PrepareTime: 335.215ms - ExecTreePrepareTime: 6.576ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 78.445us, non-child: 78.445us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 313.43 KB/sec - OverallThroughput: 199.18 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.529us - TransmitDataRPCTime: 49.852us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.392ms, non-child: 24.392ms, % non-child: 100.00%) - CodegenTime: 551.396us - CompileTime: 4.120ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.538ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.370ms AGGREGATION_NODE (id=1):(Total: 1s328ms, non-child: 4.868ms, % non-child: 0.37%) ExecOption: Codegen Enabled - BuildTime: 1.550ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s323ms, non-child: 1s323ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:254/346.54 KB 6:246/335.51 KB 2:237/323.14 KB 0:254/345.99 KB 3:263/358.55 KB 5:214/291.77 KB 4:255/347.83 KB 7:221/301.29 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 0, 987.39 KB, 1.98 MB - FooterProcessingTime: (Avg: 10.260ms ; Min: 5.046ms ; Max: 32.983ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714246) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714246) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714246) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 218.00 KB (223232) - PerReadThreadRawHdfsThroughput: 103.34 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s277ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.47 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 1m55s - MaterializeTupleTime(*): 2.400ms - ScannerThreadsSysTime: 42.971ms - ScannerThreadsUserTime: 320.914ms - ScannerThreadsVoluntaryContextSwitches: 7.46K (7465) - TotalRawHdfsReadTime(*): 25.049ms - TotalReadThroughput: 1.32 MB/sec Instance c34cc30188b6a8c9:22 (host=vc1514.foo.fa.com:22000):(Total: 1s679ms, non-child: 340.562ms, % non-child: 20.27%) Hdfs split stats (:<# splits>/): 7:229/312.14 KB 5:230/313.69 KB 0:241/328.39 KB 6:252/343.84 KB 2:237/323.33 KB 3:238/324.14 KB 1:258/352.39 KB 4:257/350.18 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 112.36 KB ThreadUsage(500.000ms): 1, 89, 89, 89 - AverageThreadTokens: 67.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 361.576us - TotalStorageWaitTime: 19s951ms - TotalThreadsInvoluntaryContextSwitches: 15 (15) - TotalThreadsTotalWallClockTime: 1m55s - TotalThreadsSysTime: 64.971ms - TotalThreadsUserTime: 416.900ms - TotalThreadsVoluntaryContextSwitches: 9.44K (9439) Fragment Instance Lifecycle Timings: - ExecTime: 570.878us - ExecTreeExecTime: 6.050us - OpenTime: 1s355ms - ExecTreeOpenTime: 1s308ms - PrepareTime: 323.529ms - ExecTreePrepareTime: 6.552ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 79.443us, non-child: 79.443us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 252.51 KB/sec - OverallThroughput: 196.68 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.688us - TransmitDataRPCTime: 61.879us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.869ms, non-child: 23.869ms, % non-child: 100.00%) - CodegenTime: 558.514us - CompileTime: 4.097ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.553ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 12.850ms AGGREGATION_NODE (id=1):(Total: 1s315ms, non-child: 4.937ms, % non-child: 0.38%) ExecOption: Codegen Enabled - BuildTime: 1.510ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s310ms, non-child: 1s310ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 7:229/312.14 KB 5:230/313.69 KB 0:241/328.39 KB 6:252/343.84 KB 2:237/323.33 KB 3:238/324.14 KB 1:258/352.39 KB 4:257/350.18 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1942 out of 1942 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1942 BytesRead(500.000ms): 469.14 KB, 1.46 MB, 2.53 MB - FooterProcessingTime: (Avg: 10.338ms ; Min: 4.905ms ; Max: 34.097ms ; Number of samples: 1942) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2711659) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2711659) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2711659) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 113.04 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s266ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1942) - RowsReturned: 1.94K (1942) - RowsReturnedRate: 1.48 K/sec - ScanRangesComplete: 1.94K (1942) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 1m54s - MaterializeTupleTime(*): 2.338ms - ScannerThreadsSysTime: 35.976ms - ScannerThreadsUserTime: 319.915ms - ScannerThreadsVoluntaryContextSwitches: 7.35K (7346) - TotalRawHdfsReadTime(*): 22.878ms - TotalReadThroughput: 1.69 MB/sec Instance c34cc30188b6a8c9:7a (host=vd1337.foo.fa.com:22000):(Total: 1s669ms, non-child: 345.749ms, % non-child: 20.71%) Hdfs split stats (:<# splits>/): 5:170/232.13 KB 3:166/226.18 KB 0:178/242.81 KB 6:176/239.45 KB 1:162/221.02 KB 4:171/233.06 KB 7:171/233.11 KB 2:167/227.51 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 402.164us - TotalStorageWaitTime: 19s820ms - TotalThreadsInvoluntaryContextSwitches: 10 (10) - TotalThreadsTotalWallClockTime: 1m43s - TotalThreadsSysTime: 47.978ms - TotalThreadsUserTime: 310.910ms - TotalThreadsVoluntaryContextSwitches: 6.67K (6670) Fragment Instance Lifecycle Timings: - ExecTime: 619.056us - ExecTreeExecTime: 6.676us - OpenTime: 1s342ms - ExecTreeOpenTime: 1s294ms - PrepareTime: 326.808ms - ExecTreePrepareTime: 4.930ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 88.425us, non-child: 88.425us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 271.72 KB/sec - OverallThroughput: 176.70 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 13.100us - TransmitDataRPCTime: 57.503us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.888ms, non-child: 23.888ms, % non-child: 100.00%) - CodegenTime: 584.677us - CompileTime: 4.114ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.477ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 12.936ms AGGREGATION_NODE (id=1):(Total: 1s299ms, non-child: 4.073ms, % non-child: 0.31%) ExecOption: Codegen Enabled - BuildTime: 1.087ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s295ms, non-child: 1s295ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:170/232.13 KB 3:166/226.18 KB 0:178/242.81 KB 6:176/239.45 KB 1:162/221.02 KB 4:171/233.06 KB 7:171/233.11 KB 2:167/227.51 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1361 out of 1361 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1361 BytesRead(500.000ms): 109.24 KB, 737.59 KB, 1.55 MB - FooterProcessingTime: (Avg: 14.632ms ; Min: 5.563ms ; Max: 202.840ms ; Number of samples: 1361) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1899801) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1899801) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1899801) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 97.95 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s261ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1361) - RowsReturned: 1.36K (1361) - RowsReturnedRate: 1.05 K/sec - ScanRangesComplete: 1.36K (1361) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 1m42s - MaterializeTupleTime(*): 1.671ms - ScannerThreadsSysTime: 27.981ms - ScannerThreadsUserTime: 224.923ms - ScannerThreadsVoluntaryContextSwitches: 5.18K (5184) - TotalRawHdfsReadTime(*): 18.497ms - TotalReadThroughput: 1.03 MB/sec Instance c34cc30188b6a8c9:48 (host=ve1131.foo.fa.com:22000):(Total: 1s663ms, non-child: 354.337ms, % non-child: 21.30%) Hdfs split stats (:<# splits>/): 4:230/313.33 KB 7:258/351.51 KB 6:247/337.14 KB 2:246/335.31 KB 3:237/323.11 KB 5:242/329.53 KB 1:232/316.04 KB 0:252/343.70 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB, 58.36 KB ThreadUsage(500.000ms): 1, 81, 81, 81 - AverageThreadTokens: 61.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 258.36 KB (264558) - PerHostPeakMemUsage: 258.36 KB (264558) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 377.796us - TotalStorageWaitTime: 19s360ms - TotalThreadsInvoluntaryContextSwitches: 26 (26) - TotalThreadsTotalWallClockTime: 1m42s - TotalThreadsSysTime: 77.963ms - TotalThreadsUserTime: 417.907ms - TotalThreadsVoluntaryContextSwitches: 9.38K (9380) Fragment Instance Lifecycle Timings: - ExecTime: 554.621us - ExecTreeExecTime: 4.798us - OpenTime: 1s325ms - ExecTreeOpenTime: 1s277ms - PrepareTime: 337.792ms - ExecTreePrepareTime: 6.825ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 67.720us, non-child: 67.720us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 310.19 KB/sec - OverallThroughput: 230.73 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.605us - TransmitDataRPCTime: 50.372us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.842ms, non-child: 24.842ms, % non-child: 100.00%) - CodegenTime: 581.708us - CompileTime: 4.031ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.656ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.777ms AGGREGATION_NODE (id=1):(Total: 1s284ms, non-child: 4.495ms, % non-child: 0.35%) ExecOption: Codegen Enabled - BuildTime: 1.511ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s279ms, non-child: 1s279ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:230/313.33 KB 7:258/351.51 KB 6:247/337.14 KB 2:246/335.31 KB 3:237/323.11 KB 5:242/329.53 KB 1:232/316.04 KB 0:252/343.70 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 463.08 KB, 1.53 MB, 2.58 MB - FooterProcessingTime: (Avg: 10.026ms ; Min: 4.763ms ; Max: 30.989ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2713272) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713272) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713272) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 218.00 KB (223232) - PerReadThreadRawHdfsThroughput: 106.29 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s234ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.52 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 10 (10) - ScannerThreadsTotalWallClockTime: 1m41s - MaterializeTupleTime(*): 2.362ms - ScannerThreadsSysTime: 54.967ms - ScannerThreadsUserTime: 315.922ms - ScannerThreadsVoluntaryContextSwitches: 7.30K (7301) - TotalRawHdfsReadTime(*): 24.345ms - TotalReadThroughput: 1.72 MB/sec Instance c34cc30188b6a8c9:78 (host=vc1516.foo.fa.com:22000):(Total: 1s641ms, non-child: 349.044ms, % non-child: 21.27%) Hdfs split stats (:<# splits>/): 5:246/335.61 KB 0:258/351.68 KB 2:220/299.97 KB 3:230/314.09 KB 7:240/326.86 KB 4:265/361.35 KB 6:249/339.81 KB 1:236/321.79 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 194.36 KB ThreadUsage(500.000ms): 89, 89, 89 - AverageThreadTokens: 89.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 281.240us - TotalStorageWaitTime: 18s851ms - TotalThreadsInvoluntaryContextSwitches: 20 (20) - TotalThreadsTotalWallClockTime: 1m51s - TotalThreadsSysTime: 73.961ms - TotalThreadsUserTime: 384.900ms - TotalThreadsVoluntaryContextSwitches: 9.49K (9486) Fragment Instance Lifecycle Timings: - ExecTime: 471.849us - ExecTreeExecTime: 4.605us - OpenTime: 1s308ms - ExecTreeOpenTime: 1s261ms - PrepareTime: 331.885ms - ExecTreePrepareTime: 6.602ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 69.002us, non-child: 69.002us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 308.88 KB/sec - OverallThroughput: 226.44 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.596us - TransmitDataRPCTime: 50.586us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.081ms, non-child: 24.081ms, % non-child: 100.00%) - CodegenTime: 569.473us - CompileTime: 4.030ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.518ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.188ms AGGREGATION_NODE (id=1):(Total: 1s268ms, non-child: 4.505ms, % non-child: 0.36%) ExecOption: Codegen Enabled - BuildTime: 1.633ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s263ms, non-child: 1s263ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:246/335.61 KB 0:258/351.68 KB 2:220/299.97 KB 3:230/314.09 KB 7:240/326.86 KB 4:265/361.35 KB 6:249/339.81 KB 1:236/321.79 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 64.11 KB, 1.14 MB, 2.24 MB - FooterProcessingTime: (Avg: 9.762ms ; Min: 4.839ms ; Max: 33.388ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2714775) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2714775) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2714775) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 115.81 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s220ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.54 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 1m49s - MaterializeTupleTime(*): 2.146ms - ScannerThreadsSysTime: 55.964ms - ScannerThreadsUserTime: 282.915ms - ScannerThreadsVoluntaryContextSwitches: 7.39K (7391) - TotalRawHdfsReadTime(*): 22.356ms - TotalReadThroughput: 1.49 MB/sec Instance c34cc30188b6a8c9:6c (host=ve1129.foo.fa.com:22000):(Total: 1s630ms, non-child: 353.137ms, % non-child: 21.66%) Hdfs split stats (:<# splits>/): 3:241/328.27 KB 5:244/332.56 KB 1:244/332.62 KB 0:219/298.74 KB 2:232/316.00 KB 4:252/343.53 KB 7:259/353.22 KB 6:253/345.12 KB MemoryUsage(500.000ms): 28.00 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81 - AverageThreadTokens: 54.33 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 477.513us - TotalStorageWaitTime: 18s798ms - TotalThreadsInvoluntaryContextSwitches: 15 (15) - TotalThreadsTotalWallClockTime: 1m39s - TotalThreadsSysTime: 85.962ms - TotalThreadsUserTime: 410.908ms - TotalThreadsVoluntaryContextSwitches: 9.62K (9617) Fragment Instance Lifecycle Timings: - ExecTime: 692.713us - ExecTreeExecTime: 5.085us - OpenTime: 1s294ms - ExecTreeOpenTime: 1s246ms - PrepareTime: 335.065ms - ExecTreePrepareTime: 6.673ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 76.986us, non-child: 76.986us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 293.23 KB/sec - OverallThroughput: 202.96 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.797us - TransmitDataRPCTime: 53.286us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.583ms, non-child: 24.583ms, % non-child: 100.00%) - CodegenTime: 584.832us - CompileTime: 4.041ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.653ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.521ms AGGREGATION_NODE (id=1):(Total: 1s252ms, non-child: 5.028ms, % non-child: 0.40%) ExecOption: Codegen Enabled - BuildTime: 1.621ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s247ms, non-child: 1s247ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:241/328.27 KB 5:244/332.56 KB 1:244/332.62 KB 0:219/298.74 KB 2:232/316.00 KB 4:252/343.53 KB 7:259/353.22 KB 6:253/345.12 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1944 out of 1944 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1944 BytesRead(500.000ms): 0, 1016.90 KB, 2.10 MB - FooterProcessingTime: (Avg: 9.741ms ; Min: 5.127ms ; Max: 30.704ms ; Number of samples: 1944) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 2.59 MB (2713660) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713660) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713660) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 101.38 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s203ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1944) - RowsReturned: 1.94K (1944) - RowsReturnedRate: 1.56 K/sec - ScanRangesComplete: 1.94K (1944) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 1m38s - MaterializeTupleTime(*): 2.254ms - ScannerThreadsSysTime: 55.966ms - ScannerThreadsUserTime: 318.922ms - ScannerThreadsVoluntaryContextSwitches: 7.53K (7525) - TotalRawHdfsReadTime(*): 25.527ms - TotalReadThroughput: 1.40 MB/sec Instance c34cc30188b6a8c9:81 (host=vd1342.foo.fa.com:22000):(Total: 1s629ms, non-child: 419.355ms, % non-child: 25.74%) Hdfs split stats (:<# splits>/): 4:185/252.58 KB 6:168/229.00 KB 7:155/211.74 KB 1:177/241.01 KB 2:180/245.60 KB 0:160/218.17 KB 3:169/229.95 KB 5:167/227.70 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81 - AverageThreadTokens: 54.33 - BloomFilterBytes: 0 - PeakMemoryUsage: 250.36 KB (256366) - PerHostPeakMemUsage: 250.36 KB (256366) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 374.400us - TotalStorageWaitTime: 17s835ms - TotalThreadsInvoluntaryContextSwitches: 23 (23) - TotalThreadsTotalWallClockTime: 1m34s - TotalThreadsSysTime: 58.971ms - TotalThreadsUserTime: 308.911ms - TotalThreadsVoluntaryContextSwitches: 6.64K (6644) Fragment Instance Lifecycle Timings: - ExecTime: 528.925us - ExecTreeExecTime: 5.132us - OpenTime: 1s226ms - ExecTreeOpenTime: 1s179ms - PrepareTime: 402.095ms - ExecTreePrepareTime: 5.354ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 78.443us, non-child: 78.443us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 334.17 KB/sec - OverallThroughput: 199.19 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.671us - TransmitDataRPCTime: 46.758us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.553ms, non-child: 25.553ms, % non-child: 100.00%) - CodegenTime: 606.259us - CompileTime: 4.281ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.823ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.052ms AGGREGATION_NODE (id=1):(Total: 1s184ms, non-child: 3.223ms, % non-child: 0.27%) ExecOption: Codegen Enabled - BuildTime: 1.062ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s181ms, non-child: 1s181ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:185/252.58 KB 6:168/229.00 KB 7:155/211.74 KB 1:177/241.01 KB 2:180/245.60 KB 0:160/218.17 KB 3:169/229.95 KB 5:167/227.70 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1361 out of 1361 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1361 BytesRead(500.000ms): 524.70 KB, 1.26 MB - FooterProcessingTime: (Avg: 13.171ms ; Min: 5.779ms ; Max: 33.447ms ; Number of samples: 1361) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1900296) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1900296) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1900296) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 210.00 KB (215040) - PerReadThreadRawHdfsThroughput: 95.92 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s144ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1361) - RowsReturned: 1.36K (1361) - RowsReturnedRate: 1.15 K/sec - ScanRangesComplete: 1.36K (1361) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 1m33s - MaterializeTupleTime(*): 1.678ms - ScannerThreadsSysTime: 39.973ms - ScannerThreadsUserTime: 219.924ms - ScannerThreadsVoluntaryContextSwitches: 5.16K (5162) - TotalRawHdfsReadTime(*): 18.893ms - TotalReadThroughput: 1.26 MB/sec Instance c34cc30188b6a8c9:86 (host=vhs041.foo.fa.com:22000):(Total: 1s608ms, non-child: 348.686ms, % non-child: 21.68%) Hdfs split stats (:<# splits>/): 2:226/308.15 KB 0:260/354.64 KB 7:253/345.19 KB 4:240/327.46 KB 5:250/340.70 KB 1:238/324.51 KB 6:236/321.92 KB 3:240/326.90 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 89, 89 - AverageThreadTokens: 59.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 408.988us - TotalStorageWaitTime: 18s654ms - TotalThreadsInvoluntaryContextSwitches: 12 (12) - TotalThreadsTotalWallClockTime: 1m48s - TotalThreadsSysTime: 61.975ms - TotalThreadsUserTime: 377.903ms - TotalThreadsVoluntaryContextSwitches: 9.67K (9668) Fragment Instance Lifecycle Timings: - ExecTime: 585.745us - ExecTreeExecTime: 5.994us - OpenTime: 1s275ms - ExecTreeOpenTime: 1s227ms - PrepareTime: 332.318ms - ExecTreePrepareTime: 6.788ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 79.631us, non-child: 79.631us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 288.26 KB/sec - OverallThroughput: 196.22 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.342us - TransmitDataRPCTime: 54.205us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.871ms, non-child: 24.871ms, % non-child: 100.00%) - CodegenTime: 595.002us - CompileTime: 4.102ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.592ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.783ms AGGREGATION_NODE (id=1):(Total: 1s234ms, non-child: 3.705ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.456ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s230ms, non-child: 1s230ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:226/308.15 KB 0:260/354.64 KB 7:253/345.19 KB 4:240/327.46 KB 5:250/340.70 KB 1:238/324.51 KB 6:236/321.92 KB 3:240/326.90 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1943 out of 1943 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1943 BytesRead(500.000ms): 707.33 KB, 1.80 MB - FooterProcessingTime: (Avg: 9.662ms ; Min: 4.800ms ; Max: 30.316ms ; Number of samples: 1943) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 88.00 - BytesRead: 2.59 MB (2713046) - BytesReadDataNodeCache: 0 - BytesReadLocal: 2.59 MB (2713046) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2.59 MB (2713046) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 88 (88) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 108.36 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s188ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.94K (1943) - RowsReturned: 1.94K (1943) - RowsReturnedRate: 1.58 K/sec - ScanRangesComplete: 1.94K (1943) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 1m46s - MaterializeTupleTime(*): 2.090ms - ScannerThreadsSysTime: 40.978ms - ScannerThreadsUserTime: 285.917ms - ScannerThreadsVoluntaryContextSwitches: 7.57K (7567) - TotalRawHdfsReadTime(*): 23.877ms - TotalReadThroughput: 1.80 MB/sec Instance c34cc30188b6a8c9:35 (host=vd1326.foo.fa.com:22000):(Total: 1s562ms, non-child: 468.819ms, % non-child: 30.01%) Hdfs split stats (:<# splits>/): 4:166/226.37 KB 3:170/231.54 KB 1:179/244.14 KB 7:181/246.78 KB 0:180/245.42 KB 2:158/215.59 KB 5:162/220.48 KB 6:166/226.66 KB MemoryUsage(500.000ms): 0, 194.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81 - AverageThreadTokens: 54.33 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 576.942us - TotalStorageWaitTime: 16s144ms - TotalThreadsInvoluntaryContextSwitches: 29 (29) - TotalThreadsTotalWallClockTime: 1m24s - TotalThreadsSysTime: 69.972ms - TotalThreadsUserTime: 312.912ms - TotalThreadsVoluntaryContextSwitches: 6.76K (6762) Fragment Instance Lifecycle Timings: - ExecTime: 769.491us - ExecTreeExecTime: 5.556us - OpenTime: 1s126ms - ExecTreeOpenTime: 1s061ms - PrepareTime: 434.931ms - ExecTreePrepareTime: 6.052ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 102.529us, non-child: 102.529us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 234.07 KB/sec - OverallThroughput: 152.40 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 19.010us - TransmitDataRPCTime: 66.754us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 25.571ms, non-child: 25.571ms, % non-child: 100.00%) - CodegenTime: 645.561us - CompileTime: 4.053ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.774ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.340ms AGGREGATION_NODE (id=1):(Total: 1s067ms, non-child: 2.082ms, % non-child: 0.20%) ExecOption: Codegen Enabled - BuildTime: 996.827us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s065ms, non-child: 1s065ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:166/226.37 KB 3:170/231.54 KB 1:179/244.14 KB 7:181/246.78 KB 0:180/245.42 KB 2:158/215.59 KB 5:162/220.48 KB 6:166/226.66 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1362 out of 1362 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1362 BytesRead(500.000ms): 166.17 KB, 1.09 MB - FooterProcessingTime: (Avg: 11.922ms ; Min: 5.810ms ; Max: 33.320ms ; Number of samples: 1362) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1901551) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1901551) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1901551) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 100.97 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s027ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1362) - RowsReturned: 1.36K (1362) - RowsReturnedRate: 1.28 K/sec - ScanRangesComplete: 1.36K (1362) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 1m23s - MaterializeTupleTime(*): 1.660ms - ScannerThreadsSysTime: 43.976ms - ScannerThreadsUserTime: 212.927ms - ScannerThreadsVoluntaryContextSwitches: 5.25K (5247) - TotalRawHdfsReadTime(*): 17.961ms - TotalReadThroughput: 1.09 MB/sec Instance c34cc30188b6a8c9:26 (host=vd1333.foo.fa.com:22000):(Total: 1s545ms, non-child: 335.393ms, % non-child: 21.70%) Hdfs split stats (:<# splits>/): 1:156/212.33 KB 7:182/248.32 KB 4:174/237.21 KB 6:177/241.60 KB 5:156/212.72 KB 2:188/256.27 KB 0:169/230.24 KB 3:159/216.92 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB, 104.36 KB ThreadUsage(500.000ms): 81, 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 486.805us - TotalStorageWaitTime: 17s616ms - TotalThreadsInvoluntaryContextSwitches: 12 (12) - TotalThreadsTotalWallClockTime: 1m34s - TotalThreadsSysTime: 65.965ms - TotalThreadsUserTime: 302.910ms - TotalThreadsVoluntaryContextSwitches: 6.75K (6753) Fragment Instance Lifecycle Timings: - ExecTime: 699.946us - ExecTreeExecTime: 6.110us - OpenTime: 1s225ms - ExecTreeOpenTime: 1s180ms - PrepareTime: 319.126ms - ExecTreePrepareTime: 5.336ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 82.752us, non-child: 82.752us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 269.85 KB/sec - OverallThroughput: 188.82 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.673us - TransmitDataRPCTime: 57.903us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.548ms, non-child: 24.548ms, % non-child: 100.00%) - CodegenTime: 629.711us - CompileTime: 4.163ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.645ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.347ms AGGREGATION_NODE (id=1):(Total: 1s185ms, non-child: 4.693ms, % non-child: 0.40%) ExecOption: Codegen Enabled - BuildTime: 1.300ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s181ms, non-child: 1s181ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:156/212.33 KB 7:182/248.32 KB 4:174/237.21 KB 6:177/241.60 KB 5:156/212.72 KB 2:188/256.27 KB 0:169/230.24 KB 3:159/216.92 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1361 out of 1361 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1361 BytesRead(500.000ms): 189.78 KB, 1.23 MB, 1.77 MB - FooterProcessingTime: (Avg: 13.012ms ; Min: 6.094ms ; Max: 207.343ms ; Number of samples: 1361) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1900150) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1900150) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1900150) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 101.10 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s143ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1361) - RowsReturned: 1.36K (1361) - RowsReturnedRate: 1.15 K/sec - ScanRangesComplete: 1.36K (1361) - ScannerThreadsInvoluntaryContextSwitches: 4 (4) - ScannerThreadsTotalWallClockTime: 1m33s - MaterializeTupleTime(*): 1.679ms - ScannerThreadsSysTime: 40.969ms - ScannerThreadsUserTime: 217.923ms - ScannerThreadsVoluntaryContextSwitches: 5.27K (5271) - TotalRawHdfsReadTime(*): 17.924ms - TotalReadThroughput: 1.18 MB/sec Instance c34cc30188b6a8c9:5f (host=vd1340.foo.fa.com:22000):(Total: 1s521ms, non-child: 334.804ms, % non-child: 22.01%) Hdfs split stats (:<# splits>/): 1:168/229.38 KB 7:159/216.73 KB 2:172/234.27 KB 0:157/213.93 KB 4:186/253.53 KB 6:175/238.81 KB 3:163/222.30 KB 5:182/248.34 KB MemoryUsage(500.000ms): 32.36 KB, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81 - AverageThreadTokens: 54.33 - BloomFilterBytes: 0 - PeakMemoryUsage: 236.36 KB (242030) - PerHostPeakMemUsage: 236.36 KB (242030) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 479.092us - TotalStorageWaitTime: 17s315ms - TotalThreadsInvoluntaryContextSwitches: 18 (18) - TotalThreadsTotalWallClockTime: 1m32s - TotalThreadsSysTime: 62.972ms - TotalThreadsUserTime: 307.912ms - TotalThreadsVoluntaryContextSwitches: 6.64K (6637) Fragment Instance Lifecycle Timings: - ExecTime: 708.451us - ExecTreeExecTime: 5.266us - OpenTime: 1s203ms - ExecTreeOpenTime: 1s157ms - PrepareTime: 317.364ms - ExecTreePrepareTime: 4.949ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 92.605us, non-child: 92.605us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 239.22 KB/sec - OverallThroughput: 168.73 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.355us - TransmitDataRPCTime: 65.316us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.078ms, non-child: 24.078ms, % non-child: 100.00%) - CodegenTime: 587.260us - CompileTime: 4.017ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.516ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.176ms AGGREGATION_NODE (id=1):(Total: 1s162ms, non-child: 3.927ms, % non-child: 0.34%) ExecOption: Codegen Enabled - BuildTime: 1.153ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s158ms, non-child: 1s158ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:168/229.38 KB 7:159/216.73 KB 2:172/234.27 KB 0:157/213.93 KB 4:186/253.53 KB 6:175/238.81 KB 3:163/222.30 KB 5:182/248.34 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1362 out of 1362 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1362 BytesRead(500.000ms): 0, 632.13 KB, 1.58 MB - FooterProcessingTime: (Avg: 12.783ms ; Min: 6.106ms ; Max: 41.087ms ; Number of samples: 1362) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1901869) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1901869) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1901869) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 196.00 KB (200704) - PerReadThreadRawHdfsThroughput: 100.48 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 1s122ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1362) - RowsReturned: 1.36K (1362) - RowsReturnedRate: 1.18 K/sec - ScanRangesComplete: 1.36K (1362) - ScannerThreadsInvoluntaryContextSwitches: 5 (5) - ScannerThreadsTotalWallClockTime: 1m31s - MaterializeTupleTime(*): 1.795ms - ScannerThreadsSysTime: 39.975ms - ScannerThreadsUserTime: 217.926ms - ScannerThreadsVoluntaryContextSwitches: 5.17K (5166) - TotalRawHdfsReadTime(*): 18.050ms - TotalReadThroughput: 1.05 MB/sec Instance c34cc30188b6a8c9:30 (host=vd1334.foo.fa.com:22000):(Total: 1s484ms, non-child: 476.821ms, % non-child: 32.12%) Hdfs split stats (:<# splits>/): 0:172/234.55 KB 2:158/215.58 KB 6:173/235.96 KB 7:176/240.44 KB 1:165/224.81 KB 4:167/227.82 KB 3:161/219.60 KB 5:189/257.69 KB MemoryUsage(500.000ms): 0, 208.36 KB, 166.36 KB ThreadUsage(500.000ms): 1, 81, 81 - AverageThreadTokens: 54.33 - BloomFilterBytes: 0 - PeakMemoryUsage: 258.36 KB (264558) - PerHostPeakMemUsage: 258.36 KB (264558) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 384.573us - TotalStorageWaitTime: 14s445ms - TotalThreadsInvoluntaryContextSwitches: 24 (24) - TotalThreadsTotalWallClockTime: 1m18s - TotalThreadsSysTime: 67.965ms - TotalThreadsUserTime: 309.908ms - TotalThreadsVoluntaryContextSwitches: 6.80K (6796) Fragment Instance Lifecycle Timings: - ExecTime: 553.040us - ExecTreeExecTime: 4.577us - OpenTime: 1s044ms - ExecTreeOpenTime: 974.284ms - PrepareTime: 439.791ms - ExecTreePrepareTime: 6.893ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 109.031us, non-child: 109.031us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 233.78 KB/sec - OverallThroughput: 143.31 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.695us - TransmitDataRPCTime: 66.837us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 26.525ms, non-child: 26.525ms, % non-child: 100.00%) - CodegenTime: 614.202us - CompileTime: 5.072ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.920ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 14.117ms AGGREGATION_NODE (id=1):(Total: 981.177ms, non-child: 119.348us, % non-child: 0.01%) ExecOption: Codegen Enabled - BuildTime: 863.218us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 981.057ms, non-child: 981.057ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:172/234.55 KB 2:158/215.58 KB 6:173/235.96 KB 7:176/240.44 KB 1:165/224.81 KB 4:167/227.82 KB 3:161/219.60 KB 5:189/257.69 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1361 out of 1361 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1361 BytesRead(500.000ms): 657.54 KB, 1.68 MB - FooterProcessingTime: (Avg: 10.681ms ; Min: 4.666ms ; Max: 33.512ms ; Number of samples: 1361) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1901010) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1901010) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1901010) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 218.00 KB (223232) - PerReadThreadRawHdfsThroughput: 98.52 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 943.730ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1361) - RowsReturned: 1.36K (1361) - RowsReturnedRate: 1.39 K/sec - ScanRangesComplete: 1.36K (1361) - ScannerThreadsInvoluntaryContextSwitches: 7 (7) - ScannerThreadsTotalWallClockTime: 1m16s - MaterializeTupleTime(*): 1.653ms - ScannerThreadsSysTime: 41.969ms - ScannerThreadsUserTime: 210.923ms - ScannerThreadsVoluntaryContextSwitches: 5.29K (5294) - TotalRawHdfsReadTime(*): 18.401ms - TotalReadThroughput: 1.68 MB/sec Instance c34cc30188b6a8c9:3f (host=vd1327.foo.fa.com:22000):(Total: 1s430ms, non-child: 407.677ms, % non-child: 28.50%) Hdfs split stats (:<# splits>/): 0:163/222.49 KB 7:166/226.56 KB 1:168/229.29 KB 5:174/236.95 KB 3:174/237.18 KB 2:168/228.88 KB 6:172/234.19 KB 4:176/240.01 KB MemoryUsage(500.000ms): 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 375.057us - TotalStorageWaitTime: 15s229ms - TotalThreadsInvoluntaryContextSwitches: 10 (10) - TotalThreadsTotalWallClockTime: 1m19s - TotalThreadsSysTime: 57.976ms - TotalThreadsUserTime: 294.914ms - TotalThreadsVoluntaryContextSwitches: 6.68K (6684) Fragment Instance Lifecycle Timings: - ExecTime: 551.544us - ExecTreeExecTime: 5.623us - OpenTime: 1s040ms - ExecTreeOpenTime: 993.595ms - PrepareTime: 389.138ms - ExecTreePrepareTime: 5.002ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 86.863us, non-child: 86.863us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 288.61 KB/sec - OverallThroughput: 179.88 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.740us - TransmitDataRPCTime: 54.138us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.893ms, non-child: 23.893ms, % non-child: 100.00%) - CodegenTime: 575.690us - CompileTime: 4.012ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.568ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 12.943ms AGGREGATION_NODE (id=1):(Total: 998.597ms, non-child: 3.043ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.101ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 995.553ms, non-child: 995.553ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:163/222.49 KB 7:166/226.56 KB 1:168/229.29 KB 5:174/236.95 KB 3:174/237.18 KB 2:168/228.88 KB 6:172/234.19 KB 4:176/240.01 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1361 out of 1361 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1361 BytesRead(500.000ms): 17.74 KB, 1.05 MB - FooterProcessingTime: (Avg: 11.258ms ; Min: 5.680ms ; Max: 31.765ms ; Number of samples: 1361) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1900087) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1900087) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1900087) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 103.96 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 961.367ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1361) - RowsReturned: 1.36K (1361) - RowsReturnedRate: 1.37 K/sec - ScanRangesComplete: 1.36K (1361) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 1m18s - MaterializeTupleTime(*): 1.636ms - ScannerThreadsSysTime: 36.979ms - ScannerThreadsUserTime: 215.926ms - ScannerThreadsVoluntaryContextSwitches: 5.21K (5211) - TotalRawHdfsReadTime(*): 17.430ms - TotalReadThroughput: 1.05 MB/sec Instance c34cc30188b6a8c9:3e (host=vd1335.foo.fa.com:22000):(Total: 1s393ms, non-child: 348.354ms, % non-child: 25.00%) Hdfs split stats (:<# splits>/): 3:165/225.18 KB 2:176/239.70 KB 4:167/227.78 KB 7:176/239.99 KB 0:165/224.89 KB 6:173/236.25 KB 5:181/246.61 KB 1:159/216.64 KB MemoryUsage(500.000ms): 184.36 KB, 208.36 KB ThreadUsage(500.000ms): 81, 81 - AverageThreadTokens: 81.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 490.222us - TotalStorageWaitTime: 15s388ms - TotalThreadsInvoluntaryContextSwitches: 12 (12) - TotalThreadsTotalWallClockTime: 1m21s - TotalThreadsSysTime: 64.968ms - TotalThreadsUserTime: 286.915ms - TotalThreadsVoluntaryContextSwitches: 6.69K (6693) Fragment Instance Lifecycle Timings: - ExecTime: 660.108us - ExecTreeExecTime: 5.944us - OpenTime: 1s063ms - ExecTreeOpenTime: 1s015ms - PrepareTime: 329.055ms - ExecTreePrepareTime: 4.953ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 123.386us, non-child: 123.386us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 156.68 KB/sec - OverallThroughput: 126.63 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.938us - TransmitDataRPCTime: 99.724us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.171ms, non-child: 24.171ms, % non-child: 100.00%) - CodegenTime: 592.673us - CompileTime: 3.976ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.677ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.116ms AGGREGATION_NODE (id=1):(Total: 1s020ms, non-child: 3.047ms, % non-child: 0.30%) ExecOption: Codegen Enabled - BuildTime: 1.029ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1s017ms, non-child: 1s017ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:165/225.18 KB 2:176/239.70 KB 4:167/227.78 KB 7:176/239.99 KB 0:165/224.89 KB 6:173/236.25 KB 5:181/246.61 KB 1:159/216.64 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1362 out of 1362 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1362 BytesRead(500.000ms): 141.93 KB, 1.07 MB - FooterProcessingTime: (Avg: 11.366ms ; Min: 6.104ms ; Max: 32.647ms ; Number of samples: 1362) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1901610) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1901610) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1901610) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 110.97 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 984.194ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1362) - RowsReturned: 1.36K (1362) - RowsReturnedRate: 1.34 K/sec - ScanRangesComplete: 1.36K (1362) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 1m20s - MaterializeTupleTime(*): 1.600ms - ScannerThreadsSysTime: 45.971ms - ScannerThreadsUserTime: 200.928ms - ScannerThreadsVoluntaryContextSwitches: 5.19K (5194) - TotalRawHdfsReadTime(*): 16.342ms - TotalReadThroughput: 1.07 MB/sec Instance c34cc30188b6a8c9:3c (host=vd1339.foo.fa.com:22000):(Total: 1s318ms, non-child: 335.151ms, % non-child: 25.42%) Hdfs split stats (:<# splits>/): 3:189/257.76 KB 0:185/252.39 KB 4:168/229.47 KB 5:162/220.89 KB 7:175/238.23 KB 6:156/212.47 KB 2:167/227.47 KB 1:159/216.56 KB MemoryUsage(500.000ms): 0, 184.36 KB, 112.36 KB ThreadUsage(500.000ms): 1, 81, 81 - AverageThreadTokens: 54.33 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 574.472us - TotalStorageWaitTime: 14s041ms - TotalThreadsInvoluntaryContextSwitches: 21 (21) - TotalThreadsTotalWallClockTime: 1m16s - TotalThreadsSysTime: 63.974ms - TotalThreadsUserTime: 303.913ms - TotalThreadsVoluntaryContextSwitches: 6.70K (6701) Fragment Instance Lifecycle Timings: - ExecTime: 826.171us - ExecTreeExecTime: 4.527us - OpenTime: 999.439ms - ExecTreeOpenTime: 953.261ms - PrepareTime: 318.002ms - ExecTreePrepareTime: 4.793ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 94.022us, non-child: 94.022us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 225.71 KB/sec - OverallThroughput: 166.18 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 11.256us - TransmitDataRPCTime: 69.227us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.995ms, non-child: 24.995ms, % non-child: 100.00%) - CodegenTime: 582.516us - CompileTime: 4.101ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.591ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.920ms AGGREGATION_NODE (id=1):(Total: 958.054ms, non-child: 3.808ms, % non-child: 0.40%) ExecOption: Codegen Enabled - BuildTime: 1.127ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 954.246ms, non-child: 954.246ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 3:189/257.76 KB 0:185/252.39 KB 4:168/229.47 KB 5:162/220.89 KB 7:175/238.23 KB 6:156/212.47 KB 2:167/227.47 KB 1:159/216.56 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1361 out of 1361 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1361 BytesRead(500.000ms): 754.96 KB, 1.75 MB - FooterProcessingTime: (Avg: 10.386ms ; Min: 5.446ms ; Max: 33.895ms ; Number of samples: 1361) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1899762) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1899762) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1899762) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.19 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 920.695ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1361) - RowsReturned: 1.36K (1361) - RowsReturnedRate: 1.43 K/sec - ScanRangesComplete: 1.36K (1361) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 1m15s - MaterializeTupleTime(*): 1.641ms - ScannerThreadsSysTime: 39.978ms - ScannerThreadsUserTime: 214.927ms - ScannerThreadsVoluntaryContextSwitches: 5.24K (5237) - TotalRawHdfsReadTime(*): 17.729ms - TotalReadThroughput: 1.75 MB/sec Instance c34cc30188b6a8c9:6 (host=vd1141.foo.fa.com:22000):(Total: 1s283ms, non-child: 336.838ms, % non-child: 26.24%) Hdfs split stats (:<# splits>/): 4:167/227.93 KB 6:177/241.31 KB 1:156/212.59 KB 0:172/234.29 KB 7:168/229.43 KB 5:168/229.20 KB 2:179/243.99 KB 3:175/238.22 KB MemoryUsage(500.000ms): 0, 184.36 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81, 81 - AverageThreadTokens: 54.33 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 579.304us - TotalStorageWaitTime: 13s958ms - TotalThreadsInvoluntaryContextSwitches: 9 (9) - TotalThreadsTotalWallClockTime: 1m13s - TotalThreadsSysTime: 69.965ms - TotalThreadsUserTime: 278.916ms - TotalThreadsVoluntaryContextSwitches: 6.71K (6707) Fragment Instance Lifecycle Timings: - ExecTime: 901.834us - ExecTreeExecTime: 5.570us - OpenTime: 963.209ms - ExecTreeOpenTime: 917.699ms - PrepareTime: 319.494ms - ExecTreePrepareTime: 4.780ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 105.050us, non-child: 105.050us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 256.06 KB/sec - OverallThroughput: 148.74 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 18.961us - TransmitDataRPCTime: 61.021us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 24.212ms, non-child: 24.212ms, % non-child: 100.00%) - CodegenTime: 540.319us - CompileTime: 4.020ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.667ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.144ms AGGREGATION_NODE (id=1):(Total: 922.480ms, non-child: 3.359ms, % non-child: 0.36%) ExecOption: Codegen Enabled - BuildTime: 1.126ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 919.120ms, non-child: 919.120ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:167/227.93 KB 6:177/241.31 KB 1:156/212.59 KB 0:172/234.29 KB 7:168/229.43 KB 5:168/229.20 KB 2:179/243.99 KB 3:175/238.22 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1362 out of 1362 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1362 BytesRead(500.000ms): 293.00 KB, 1.31 MB - FooterProcessingTime: (Avg: 10.318ms ; Min: 5.568ms ; Max: 32.376ms ; Number of samples: 1362) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1901524) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1901524) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1901524) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 97.86 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 886.424ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1362) - RowsReturned: 1.36K (1362) - RowsReturnedRate: 1.48 K/sec - ScanRangesComplete: 1.36K (1362) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 1m12s - MaterializeTupleTime(*): 1.651ms - ScannerThreadsSysTime: 47.969ms - ScannerThreadsUserTime: 204.927ms - ScannerThreadsVoluntaryContextSwitches: 5.20K (5200) - TotalRawHdfsReadTime(*): 18.531ms - TotalReadThroughput: 1.31 MB/sec Instance c34cc30188b6a8c9:84 (host=vd1332.foo.fa.com:22000):(Total: 1s263ms, non-child: 349.243ms, % non-child: 27.65%) Hdfs split stats (:<# splits>/): 6:174/237.09 KB 7:153/208.81 KB 2:160/218.16 KB 4:181/246.73 KB 3:173/236.08 KB 1:161/219.96 KB 5:183/249.36 KB 0:174/237.71 KB MemoryUsage(500.000ms): 0, 184.36 KB, 130.36 KB ThreadUsage(500.000ms): 1, 81, 81 - AverageThreadTokens: 54.33 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 449.933us - TotalStorageWaitTime: 13s301ms - TotalThreadsInvoluntaryContextSwitches: 26 (26) - TotalThreadsTotalWallClockTime: 1m10s - TotalThreadsSysTime: 57.972ms - TotalThreadsUserTime: 296.915ms - TotalThreadsVoluntaryContextSwitches: 6.79K (6785) Fragment Instance Lifecycle Timings: - ExecTime: 675.502us - ExecTreeExecTime: 5.436us - OpenTime: 934.539ms - ExecTreeOpenTime: 882.672ms - PrepareTime: 327.984ms - ExecTreePrepareTime: 5.169ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 106.721us, non-child: 106.721us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 194.40 KB/sec - OverallThroughput: 146.41 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.278us - TransmitDataRPCTime: 80.375us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 26.037ms, non-child: 26.037ms, % non-child: 100.00%) - CodegenTime: 583.272us - CompileTime: 4.526ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 7.372ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 13.726ms AGGREGATION_NODE (id=1):(Total: 887.842ms, non-child: 1.431ms, % non-child: 0.16%) ExecOption: Codegen Enabled - BuildTime: 851.973us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 886.410ms, non-child: 886.410ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 6:174/237.09 KB 7:153/208.81 KB 2:160/218.16 KB 4:181/246.73 KB 3:173/236.08 KB 1:161/219.96 KB 5:183/249.36 KB 0:174/237.71 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1359 out of 1359 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1359 BytesRead(500.000ms): 657.61 KB, 1.75 MB - FooterProcessingTime: (Avg: 9.858ms ; Min: 5.380ms ; Max: 36.573ms ; Number of samples: 1359) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1898378) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1898378) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1898378) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 104.87 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 851.898ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1359) - RowsReturned: 1.36K (1359) - RowsReturnedRate: 1.53 K/sec - ScanRangesComplete: 1.36K (1359) - ScannerThreadsInvoluntaryContextSwitches: 3 (3) - ScannerThreadsTotalWallClockTime: 1m9s - MaterializeTupleTime(*): 1.706ms - ScannerThreadsSysTime: 42.974ms - ScannerThreadsUserTime: 208.928ms - ScannerThreadsVoluntaryContextSwitches: 5.25K (5255) - TotalRawHdfsReadTime(*): 17.263ms - TotalReadThroughput: 1.75 MB/sec Instance c34cc30188b6a8c9:60 (host=vd1341.foo.fa.com:22000):(Total: 1s255ms, non-child: 336.147ms, % non-child: 26.78%) Hdfs split stats (:<# splits>/): 2:170/232.03 KB 3:162/220.72 KB 6:180/245.43 KB 4:183/249.47 KB 1:180/245.55 KB 0:175/238.62 KB 7:149/203.39 KB 5:161/219.43 KB MemoryUsage(500.000ms): 28.00 KB, 184.36 KB ThreadUsage(500.000ms): 1, 81 - AverageThreadTokens: 41.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 248.36 KB (254318) - PerHostPeakMemUsage: 248.36 KB (254318) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 377.711us - TotalStorageWaitTime: 13s380ms - TotalThreadsInvoluntaryContextSwitches: 12 (12) - TotalThreadsTotalWallClockTime: 1m11s - TotalThreadsSysTime: 65.974ms - TotalThreadsUserTime: 282.916ms - TotalThreadsVoluntaryContextSwitches: 6.89K (6895) Fragment Instance Lifecycle Timings: - ExecTime: 556.703us - ExecTreeExecTime: 5.863us - OpenTime: 936.140ms - ExecTreeOpenTime: 890.527ms - PrepareTime: 318.611ms - ExecTreePrepareTime: 4.841ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.39 GB (173287063552) - PeakMemoryUsage: 0 - ScratchFileUsedBytes: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=2):(Total: 89.748us, non-child: 89.748us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 253.15 KB/sec - OverallThroughput: 174.10 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 12.787us - TransmitDataRPCTime: 61.722us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 23.734ms, non-child: 23.734ms, % non-child: 100.00%) - CodegenTime: 643.755us - CompileTime: 4.061ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.91 MB (1997620) - NumFunctions: 10 (10) - NumInstructions: 171 (171) - OptimizationTime: 6.513ms - PeakMemoryUsage: 85.50 KB (87552) - PrepareTime: 12.794ms AGGREGATION_NODE (id=1):(Total: 895.368ms, non-child: 3.908ms, % non-child: 0.44%) ExecOption: Codegen Enabled - BuildTime: 1.226ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 16.00 KB (16384) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 891.459ms, non-child: 891.459ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:170/232.03 KB 3:162/220.72 KB 6:180/245.43 KB 4:183/249.47 KB 1:180/245.55 KB 0:175/238.62 KB 7:149/203.39 KB 5:161/219.43 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1360 out of 1360 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:1360 BytesRead(500.000ms): 0, 965.30 KB - FooterProcessingTime: (Avg: 9.909ms ; Min: 5.234ms ; Max: 32.841ms ; Number of samples: 1360) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 80.00 - BytesRead: 1.81 MB (1899150) - BytesReadDataNodeCache: 0 - BytesReadLocal: 1.81 MB (1899150) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1.81 MB (1899150) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 0 (0) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 208.00 KB (212992) - PerReadThreadRawHdfsThroughput: 102.26 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 856.662ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.36K (1360) - RowsReturned: 1.36K (1360) - RowsReturnedRate: 1.52 K/sec - ScanRangesComplete: 1.36K (1360) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 1m10s - MaterializeTupleTime(*): 1.682ms - ScannerThreadsSysTime: 38.978ms - ScannerThreadsUserTime: 207.927ms - ScannerThreadsVoluntaryContextSwitches: 5.39K (5395) - TotalRawHdfsReadTime(*): 17.711ms - TotalReadThroughput: 965.30 KB/sec