Query (id=3b44783e08b26695:e9a6c65800000000): Summary: Session ID: 4b482dd805424524:de09a2f106845ad Session Type: BEESWAX Start Time: 2017-02-26 19:40:10.727075000 End Time: Query Type: DML Query State: CREATED Query Status: OK Impala Version: impalad version 2.9.0-SNAPSHOT RELEASE (build 15eaecec51639e9a090711529bca965eba8e8cbc) User: mmulder Connected User: mmulder Delegated User: Network Address: 172.21.0.207:45091 Default Db: impala_2328_no_minmax_meta Sql Statement: insert into partsupp_sorted /*+ sortby(ps_availqty) */ select * from partsupp Coordinator: vd0234.halxg.cloudera.com:22000 Query Options (non default): Plan: ---------------- Estimated Per-Host Requirements: Memory=1.00GB VCores=1 WARNING: The following tables are missing relevant table and/or column statistics. impala_2328_no_minmax_meta.partsupp WRITE TO HDFS [impala_2328_no_minmax_meta.partsupp_sorted, OVERWRITE=false] | partitions=1 | hosts=10 per-host-mem=1.00GB | 01:SORT | order by: ps_availqty DESC NULLS LAST | hosts=10 per-host-mem=0B | tuple-ids=1 row-size=25B cardinality=unavailable | 00:SCAN HDFS [impala_2328_no_minmax_meta.partsupp, RANDOM] partitions=1/1 files=44052 size=318.36GB table stats: unavailable column stats: unavailable hosts=10 per-host-mem=64.00MB tuple-ids=0 row-size=25B cardinality=unavailable ---------------- Estimated Per-Host Mem: 1073741824 Estimated Per-Host VCores: 1 Tables Missing Stats: impala_2328_no_minmax_meta.partsupp Request Pool: root.mmulder Admission result: Admitted immediately Planner Timeline: 221.289ms - Analysis finished: 7.009ms (7.009ms) - Equivalence classes computed: 7.404ms (394.533us) - Single node plan created: 184.174ms (176.770ms) - Runtime filters computed: 184.242ms (68.861us) - Distributed plan created: 203.851ms (19.608ms) - Lineage info computed: 204.149ms (298.679us) - Planning finished: 221.289ms (17.139ms) Query Timeline: 1m23s - Query submitted: 89.250us (89.250us) - Planning finished: 475.586ms (475.497ms) - Submit for admission: 582.200ms (106.613ms) - Completed admission: 582.657ms (457.540us) - Ready to start 10 fragment instances: 595.473ms (12.815ms) - All 10 fragment instances started: 624.327ms (28.854ms) - ComputeScanRangeAssignmentTimer: 72.073ms ImpalaServer: - ClientFetchWaitTimer: 0.000ns - RowMaterializationTimer: 0.000ns Execution Profile 3b44783e08b26695:e9a6c65800000000:(Total: 29.491ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed Enabled ---------------------------------------------------------------------------------------------------------------------------- Fragment instance start latencies: Count: 10, 25th %-ile: 12ms, 50th %-ile: 15ms, 75th %-ile: 21ms, 90th %-ile: 23ms, 95th %-ile: 25ms, 99.9th %-ile: 25ms - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Averaged Fragment F00:(Total: 63.060ms, non-child: 0.000ns, % non-child: 0.00%) - AverageThreadTokens: 32.96 - BloomFilterBytes: 0 - PeakMemoryUsage: 28.00 GB (30063685279) - PerHostPeakMemUsage: 28.00 GB (30063685279) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 37s998ms - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 0.000ns - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 0 (0) Fragment Instance Lifecycle Timings: - OpenTime: 0.000ns - ExecTreeOpenTime: 0.000ns - PrepareTime: 62.986ms - ExecTreePrepareTime: 42.475ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 3.53K (3530) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.38 GB (173281017856) - PeakMemoryUsage: 27.58 GB (29611786240) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns HdfsTableSink:(Total: 2.100ms, non-child: 2.100ms, % non-child: 100.00%) - BytesWritten: 0 - CompressTimer: 0.000ns - EncodeTimer: 0.000ns - FilesCreated: 0 (0) - HdfsWriteTimer: 0.000ns - PartitionsCreated: 0 (0) - PeakMemoryUsage: 0 - RowsInserted: 0 (0) CodeGen:(Total: 56.939ms, non-child: 56.939ms, % non-child: 100.00%) - CodegenTime: 1.014ms - CompileTime: 7.077ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1995628) - NumFunctions: 33 (33) - NumInstructions: 621 (621) - OptimizationTime: 32.436ms - PeakMemoryUsage: 310.50 KB (317952) - PrepareTime: 16.790ms SORT_NODE (id=1):(Total: 42.474ms, non-child: 0.000ns, % non-child: 0.00%) - InMemorySortTime: 0.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 27.58 GB (29612019279) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 3s754ms, non-child: 3s754ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.19 - AverageScannerThreadConcurrency: 32.00 - BytesRead: 12.88 GB (13825102217) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 15s849ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 0 (0) - NumRowGroups: 1.76K (1759) - NumScannerThreadsStarted: 32 (32) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 434.15 MB (455242560) - PerReadThreadRawHdfsThroughput: 1.03 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.14B (1138982204) - RowsReturned: 1.14B (1138759155) - RowsReturnedRate: 303.75 M/sec - ScanRangesComplete: 1.73K (1727) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 1m35s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 16s164ms - TotalReadThroughput: 163.10 MB/sec Fragment F00: Instance 3b44783e08b26695:e9a6c65800000001 (host=vd0338.halxg.cloudera.com:22000):(Total: 64.829ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:304/2.20 GB 7:293/2.12 GB 8:310/2.24 GB 0:314/2.27 GB 4:330/2.38 GB 2:332/2.40 GB 6:311/2.25 GB 10:309/2.24 GB 11:314/2.27 GB 9:311/2.24 GB 5:302/2.18 GB 13:349/2.52 GB 3:305/2.20 GB 12:322/2.33 GB MemoryUsage(2s000ms): 518.29 MB, 1.29 GB, 2.00 GB, 2.72 GB, 3.41 GB, 4.12 GB, 4.80 GB, 5.49 GB, 6.20 GB, 6.92 GB, 7.63 GB, 8.34 GB, 8.98 GB, 9.65 GB, 10.32 GB, 10.94 GB, 11.56 GB, 12.19 GB, 12.85 GB, 13.53 GB, 14.22 GB, 14.89 GB, 15.57 GB, 16.26 GB, 16.94 GB, 17.61 GB, 18.31 GB, 19.01 GB, 19.68 GB, 20.36 GB, 21.05 GB, 21.64 GB, 22.33 GB, 23.03 GB, 23.76 GB, 24.46 GB, 25.15 GB, 25.84 GB, 26.56 GB, 27.27 GB ThreadUsage(2s000ms): 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33 - AverageThreadTokens: 33.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 27.70 GB (29746599111) - PerHostPeakMemUsage: 27.70 GB (29746599111) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 32s780ms - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 0.000ns - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 0 (0) Fragment Instance Lifecycle Timings: - OpenTime: 0.000ns - ExecTreeOpenTime: 0.000ns - PrepareTime: 64.583ms - ExecTreePrepareTime: 45.940ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 3.49K (3492) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.38 GB (173281017856) - PeakMemoryUsage: 27.28 GB (29293019136) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns HdfsTableSink:(Total: 214.186us, non-child: 214.186us, % non-child: 100.00%) - BytesWritten: 0 - CompressTimer: 0.000ns - EncodeTimer: 0.000ns - FilesCreated: 0 (0) - HdfsWriteTimer: 0.000ns - PartitionsCreated: 0 (0) - PeakMemoryUsage: 0 - RowsInserted: 0 (0) CodeGen:(Total: 58.953ms, non-child: 58.953ms, % non-child: 100.00%) - CodegenTime: 944.939us - CompileTime: 7.532ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1995628) - NumFunctions: 33 (33) - NumInstructions: 621 (621) - OptimizationTime: 33.877ms - PeakMemoryUsage: 310.50 KB (317952) - PrepareTime: 16.901ms SORT_NODE (id=1):(Total: 45.939ms, non-child: 0.000ns, % non-child: 0.00%) ExecOption: Codegen Enabled - InMemorySortTime: 0.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 27.28 GB (29293987328) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 3s773ms, non-child: 3s773ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:304/2.20 GB 7:293/2.12 GB 8:310/2.24 GB 0:314/2.27 GB 4:330/2.38 GB 2:332/2.40 GB 6:311/2.25 GB 10:309/2.24 GB 11:314/2.27 GB 9:311/2.24 GB 5:302/2.18 GB 13:349/2.52 GB 3:305/2.20 GB 12:322/2.33 GB ExecOption: PARQUET Codegen Enabled BytesRead(2s000ms): 180.67 MB, 542.92 MB, 863.75 MB, 1.19 GB, 1.50 GB, 1.81 GB, 2.15 GB, 2.46 GB, 2.79 GB, 3.13 GB, 3.45 GB, 3.79 GB, 4.08 GB, 4.37 GB, 4.69 GB, 4.99 GB, 5.27 GB, 5.55 GB, 5.87 GB, 6.18 GB, 6.50 GB, 6.81 GB, 7.12 GB, 7.44 GB, 7.75 GB, 8.07 GB, 8.39 GB, 8.72 GB, 9.03 GB, 9.34 GB, 9.66 GB, 9.93 GB, 10.25 GB, 10.58 GB, 10.91 GB, 11.24 GB, 11.56 GB, 11.88 GB, 12.20 GB, 12.53 GB - FooterProcessingTime: (Avg: 7.481ms ; Min: 1.979ms ; Max: 239.551ms ; Number of samples: 1741) - AverageHdfsReadThreadConcurrency: 0.16 - AverageScannerThreadConcurrency: 32.00 - BytesRead: 12.75 GB (13686164990) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 16s346ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 0 (0) - NumRowGroups: 1.74K (1741) - NumScannerThreadsStarted: 32 (32) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 434.67 MB (455787333) - PerReadThreadRawHdfsThroughput: 1.09 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.13B (1126606654) - RowsReturned: 1.13B (1126382325) - RowsReturnedRate: 298.53 M/sec - ScanRangesComplete: 1.71K (1709) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 1m39s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 11s682ms - TotalReadThroughput: 161.87 MB/sec Instance 3b44783e08b26695:e9a6c65800000005 (host=vd0238.halxg.cloudera.com:22000):(Total: 49.002ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:349/2.52 GB 7:307/2.22 GB 8:308/2.23 GB 13:327/2.36 GB 12:319/2.31 GB 9:326/2.35 GB 0:326/2.36 GB 1:298/2.15 GB 10:309/2.24 GB 3:314/2.27 GB 5:310/2.24 GB 4:276/2.00 GB 11:304/2.19 GB 6:333/2.41 GB MemoryUsage(2s000ms): 723.87 MB, 1.41 GB, 2.12 GB, 2.83 GB, 3.54 GB, 4.24 GB, 4.93 GB, 5.64 GB, 6.33 GB, 7.01 GB, 7.70 GB, 8.38 GB, 9.07 GB, 9.75 GB, 10.46 GB, 11.14 GB, 11.83 GB, 12.50 GB, 13.19 GB, 13.86 GB, 14.55 GB, 15.24 GB, 15.93 GB, 16.62 GB, 17.29 GB, 17.98 GB, 18.66 GB, 19.35 GB, 20.06 GB, 20.73 GB, 21.35 GB, 22.06 GB, 22.76 GB, 23.48 GB, 24.20 GB, 24.90 GB, 25.61 GB, 26.32 GB, 27.02 GB, 27.74 GB ThreadUsage(2s000ms): 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33 - AverageThreadTokens: 33.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 28.42 GB (30510851494) - PerHostPeakMemUsage: 28.42 GB (30510851494) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 31s190ms - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 0.000ns - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 0 (0) Fragment Instance Lifecycle Timings: - OpenTime: 0.000ns - ExecTreeOpenTime: 0.000ns - PrepareTime: 48.945ms - ExecTreePrepareTime: 32.557ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 3.58K (3583) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.38 GB (173281017856) - PeakMemoryUsage: 27.99 GB (30056382464) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns HdfsTableSink:(Total: 195.177us, non-child: 195.177us, % non-child: 100.00%) - BytesWritten: 0 - CompressTimer: 0.000ns - EncodeTimer: 0.000ns - FilesCreated: 0 (0) - HdfsWriteTimer: 0.000ns - PartitionsCreated: 0 (0) - PeakMemoryUsage: 0 - RowsInserted: 0 (0) CodeGen:(Total: 54.740ms, non-child: 54.740ms, % non-child: 100.00%) - CodegenTime: 912.493us - CompileTime: 7.268ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1995628) - NumFunctions: 33 (33) - NumInstructions: 621 (621) - OptimizationTime: 31.978ms - PeakMemoryUsage: 310.50 KB (317952) - PrepareTime: 14.951ms SORT_NODE (id=1):(Total: 32.556ms, non-child: 0.000ns, % non-child: 0.00%) ExecOption: Codegen Enabled - InMemorySortTime: 0.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 27.99 GB (30056548358) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 3s682ms, non-child: 3s682ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:349/2.52 GB 7:307/2.22 GB 8:308/2.23 GB 13:327/2.36 GB 12:319/2.31 GB 9:326/2.35 GB 0:326/2.36 GB 1:298/2.15 GB 10:309/2.24 GB 3:314/2.27 GB 5:310/2.24 GB 4:276/2.00 GB 11:304/2.19 GB 6:333/2.41 GB ExecOption: PARQUET Codegen Enabled BytesRead(2s000ms): 296.43 MB, 596.60 MB, 904.27 MB, 1.25 GB, 1.55 GB, 1.89 GB, 2.21 GB, 2.51 GB, 2.85 GB, 3.16 GB, 3.47 GB, 3.81 GB, 4.12 GB, 4.44 GB, 4.77 GB, 5.07 GB, 5.39 GB, 5.71 GB, 6.01 GB, 6.34 GB, 6.66 GB, 6.96 GB, 7.29 GB, 7.61 GB, 7.92 GB, 8.24 GB, 8.55 GB, 8.88 GB, 9.20 GB, 9.51 GB, 9.80 GB, 10.12 GB, 10.45 GB, 10.79 GB, 11.11 GB, 11.44 GB, 11.77 GB, 12.09 GB, 12.42 GB, 12.75 GB - FooterProcessingTime: (Avg: 6.616ms ; Min: 2.143ms ; Max: 203.976ms ; Number of samples: 1784) - AverageHdfsReadThreadConcurrency: 0.13 - AverageScannerThreadConcurrency: 32.00 - BytesRead: 13.05 GB (14017592676) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 16s630ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 0 (0) - NumRowGroups: 1.78K (1784) - NumScannerThreadsStarted: 32 (32) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 434.05 MB (455136410) - PerReadThreadRawHdfsThroughput: 1.15 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1156227947) - RowsReturned: 1.16B (1156002667) - RowsReturnedRate: 313.93 M/sec - ScanRangesComplete: 1.75K (1752) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 1m40s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 11s381ms - TotalReadThroughput: 165.04 MB/sec Instance 3b44783e08b26695:e9a6c65800000004 (host=vd0234.halxg.cloudera.com:22000):(Total: 59.162ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:310/2.24 GB 3:323/2.34 GB 4:325/2.35 GB 6:306/2.21 GB 13:311/2.25 GB 1:320/2.31 GB 2:307/2.22 GB 0:316/2.28 GB 9:318/2.30 GB 11:331/2.39 GB 7:301/2.17 GB 8:303/2.19 GB 10:318/2.30 GB 12:315/2.28 GB MemoryUsage(2s000ms): 502.96 MB, 1.22 GB, 1.88 GB, 2.54 GB, 3.19 GB, 3.85 GB, 4.51 GB, 5.21 GB, 5.88 GB, 6.56 GB, 7.23 GB, 7.91 GB, 8.60 GB, 9.29 GB, 9.98 GB, 10.65 GB, 11.31 GB, 11.99 GB, 12.66 GB, 13.33 GB, 14.00 GB, 14.70 GB, 15.38 GB, 16.08 GB, 16.78 GB, 17.47 GB, 18.16 GB, 18.84 GB, 19.54 GB, 20.18 GB, 20.82 GB, 21.51 GB, 22.20 GB, 22.88 GB, 23.55 GB, 24.21 GB, 24.88 GB, 25.55 GB, 26.22 GB, 26.91 GB, 27.57 GB ThreadUsage(2s000ms): 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33 - AverageThreadTokens: 33.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 28.32 GB (30409055587) - PerHostPeakMemUsage: 28.32 GB (30409055587) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 1m5s - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 0.000ns - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 0 (0) Fragment Instance Lifecycle Timings: - OpenTime: 0.000ns - ExecTreeOpenTime: 0.000ns - PrepareTime: 59.114ms - ExecTreePrepareTime: 38.787ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 3.57K (3571) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.38 GB (173281017856) - PeakMemoryUsage: 27.90 GB (29955719168) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns HdfsTableSink:(Total: 167.341us, non-child: 167.341us, % non-child: 100.00%) - BytesWritten: 0 - CompressTimer: 0.000ns - EncodeTimer: 0.000ns - FilesCreated: 0 (0) - HdfsWriteTimer: 0.000ns - PartitionsCreated: 0 (0) - PeakMemoryUsage: 0 - RowsInserted: 0 (0) CodeGen:(Total: 55.607ms, non-child: 55.607ms, % non-child: 100.00%) - CodegenTime: 855.576us - CompileTime: 5.476ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1995628) - NumFunctions: 33 (33) - NumInstructions: 621 (621) - OptimizationTime: 32.259ms - PeakMemoryUsage: 310.50 KB (317952) - PrepareTime: 17.230ms SORT_NODE (id=1):(Total: 38.786ms, non-child: 0.000ns, % non-child: 0.00%) ExecOption: Codegen Enabled - InMemorySortTime: 0.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 27.90 GB (29955950606) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 3s748ms, non-child: 3s748ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:310/2.24 GB 3:323/2.34 GB 4:325/2.35 GB 6:306/2.21 GB 13:311/2.25 GB 1:320/2.31 GB 2:307/2.22 GB 0:316/2.28 GB 9:318/2.30 GB 11:331/2.39 GB 7:301/2.17 GB 8:303/2.19 GB 10:318/2.30 GB 12:315/2.28 GB ExecOption: PARQUET Codegen Enabled BytesRead(2s000ms): 180.11 MB, 519.57 MB, 821.09 MB, 1.09 GB, 1.40 GB, 1.72 GB, 2.01 GB, 2.32 GB, 2.66 GB, 2.95 GB, 3.26 GB, 3.60 GB, 3.89 GB, 4.21 GB, 4.54 GB, 4.83 GB, 5.15 GB, 5.48 GB, 5.77 GB, 6.08 GB, 6.41 GB, 6.71 GB, 7.04 GB, 7.37 GB, 7.68 GB, 8.01 GB, 8.32 GB, 8.63 GB, 8.96 GB, 9.26 GB, 9.54 GB, 9.87 GB, 10.19 GB, 10.50 GB, 10.82 GB, 11.12 GB, 11.42 GB, 11.75 GB, 12.05 GB, 12.35 GB, 12.68 GB - FooterProcessingTime: (Avg: 11.076ms ; Min: 1.573ms ; Max: 370.388ms ; Number of samples: 1781) - AverageHdfsReadThreadConcurrency: 0.48 - AverageScannerThreadConcurrency: 32.00 - BytesRead: 13.03 GB (13995922455) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 16s605ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 0 (0) - NumRowGroups: 1.78K (1781) - NumScannerThreadsStarted: 32 (32) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 434.08 MB (455160855) - PerReadThreadRawHdfsThroughput: 341.13 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.15B (1152343234) - RowsReturned: 1.15B (1152118978) - RowsReturnedRate: 307.37 M/sec - ScanRangesComplete: 1.75K (1749) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 1m40s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 39s127ms - TotalReadThroughput: 159.31 MB/sec Instance 3b44783e08b26695:e9a6c65800000007 (host=vd0334.halxg.cloudera.com:22000):(Total: 66.682ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:298/2.16 GB 5:309/2.23 GB 13:300/2.17 GB 3:315/2.27 GB 10:301/2.18 GB 12:314/2.27 GB 4:337/2.44 GB 2:311/2.25 GB 8:333/2.41 GB 9:317/2.29 GB 0:326/2.36 GB 11:320/2.31 GB 6:322/2.32 GB 7:302/2.18 GB MemoryUsage(2s000ms): 584.66 MB, 1.35 GB, 1.97 GB, 2.60 GB, 3.21 GB, 3.84 GB, 4.48 GB, 5.09 GB, 5.71 GB, 6.33 GB, 6.97 GB, 7.66 GB, 8.33 GB, 9.03 GB, 9.72 GB, 10.43 GB, 11.13 GB, 11.83 GB, 12.52 GB, 13.23 GB, 13.92 GB, 14.62 GB, 15.31 GB, 16.01 GB, 16.70 GB, 17.39 GB, 18.10 GB, 18.80 GB, 19.51 GB, 20.22 GB, 20.92 GB, 21.58 GB, 22.20 GB, 22.91 GB, 23.62 GB, 24.32 GB, 25.03 GB, 25.74 GB, 26.44 GB, 27.14 GB, 27.83 GB ThreadUsage(2s000ms): 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33 - AverageThreadTokens: 33.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 28.52 GB (30626260732) - PerHostPeakMemUsage: 28.52 GB (30626260732) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 34s851ms - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 0.000ns - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 0 (0) Fragment Instance Lifecycle Timings: - OpenTime: 0.000ns - ExecTreeOpenTime: 0.000ns - PrepareTime: 66.627ms - ExecTreePrepareTime: 47.594ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 3.60K (3599) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.38 GB (173281017856) - PeakMemoryUsage: 28.12 GB (30190600192) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns HdfsTableSink:(Total: 184.925us, non-child: 184.925us, % non-child: 100.00%) - BytesWritten: 0 - CompressTimer: 0.000ns - EncodeTimer: 0.000ns - FilesCreated: 0 (0) - HdfsWriteTimer: 0.000ns - PartitionsCreated: 0 (0) - PeakMemoryUsage: 0 - RowsInserted: 0 (0) CodeGen:(Total: 58.847ms, non-child: 58.847ms, % non-child: 100.00%) - CodegenTime: 1.046ms - CompileTime: 7.223ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1995628) - NumFunctions: 33 (33) - NumInstructions: 621 (621) - OptimizationTime: 33.672ms - PeakMemoryUsage: 310.50 KB (317952) - PrepareTime: 17.274ms SORT_NODE (id=1):(Total: 47.593ms, non-child: 0.000ns, % non-child: 0.00%) ExecOption: Codegen Enabled - InMemorySortTime: 0.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 28.12 GB (30190700552) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 4s088ms, non-child: 4s088ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:298/2.16 GB 5:309/2.23 GB 13:300/2.17 GB 3:315/2.27 GB 10:301/2.18 GB 12:314/2.27 GB 4:337/2.44 GB 2:311/2.25 GB 8:333/2.41 GB 9:317/2.29 GB 0:326/2.36 GB 11:320/2.31 GB 6:322/2.32 GB 7:302/2.18 GB ExecOption: PARQUET Codegen Enabled BytesRead(2s000ms): 224.60 MB, 577.18 MB, 847.99 MB, 1.11 GB, 1.42 GB, 1.71 GB, 2.00 GB, 2.27 GB, 2.57 GB, 2.86 GB, 3.15 GB, 3.45 GB, 3.79 GB, 4.11 GB, 4.41 GB, 4.76 GB, 5.06 GB, 5.39 GB, 5.72 GB, 6.03 GB, 6.37 GB, 6.69 GB, 7.01 GB, 7.34 GB, 7.64 GB, 7.97 GB, 8.30 GB, 8.61 GB, 8.96 GB, 9.27 GB, 9.59 GB, 9.91 GB, 10.19 GB, 10.51 GB, 10.85 GB, 11.17 GB, 11.50 GB, 11.83 GB, 12.15 GB, 12.47 GB, 12.79 GB - FooterProcessingTime: (Avg: 7.490ms ; Min: 2.101ms ; Max: 226.749ms ; Number of samples: 1793) - AverageHdfsReadThreadConcurrency: 0.19 - AverageScannerThreadConcurrency: 32.00 - BytesRead: 13.12 GB (14090057050) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 16s913ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 0 (0) - NumRowGroups: 1.79K (1793) - NumScannerThreadsStarted: 32 (32) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 434.23 MB (455320704) - PerReadThreadRawHdfsThroughput: 1.02 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1161081177) - RowsReturned: 1.16B (1160858899) - RowsReturnedRate: 283.95 M/sec - ScanRangesComplete: 1.76K (1761) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 1m42s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 12s815ms - TotalReadThroughput: 161.28 MB/sec Instance 3b44783e08b26695:e9a6c65800000002 (host=vd0340.halxg.cloudera.com:22000):(Total: 71.052ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:329/2.37 GB 7:322/2.32 GB 6:299/2.16 GB 10:299/2.16 GB 11:334/2.42 GB 12:300/2.17 GB 5:328/2.37 GB 8:291/2.10 GB 2:309/2.23 GB 1:312/2.25 GB 3:326/2.36 GB 9:319/2.31 GB 13:323/2.33 GB 4:314/2.27 GB MemoryUsage(2s000ms): 703.98 MB, 1.40 GB, 2.10 GB, 2.79 GB, 3.50 GB, 4.18 GB, 4.83 GB, 5.53 GB, 6.20 GB, 6.89 GB, 7.59 GB, 8.27 GB, 9.00 GB, 9.69 GB, 10.38 GB, 11.07 GB, 11.76 GB, 12.45 GB, 13.13 GB, 13.79 GB, 14.43 GB, 15.08 GB, 15.73 GB, 16.38 GB, 17.02 GB, 17.68 GB, 18.34 GB, 19.00 GB, 19.67 GB, 20.33 GB, 20.99 GB, 21.66 GB, 22.32 GB, 23.00 GB, 23.68 GB, 24.37 GB, 24.96 GB, 25.65 GB, 26.35 GB, 27.03 GB, 27.74 GB ThreadUsage(2s000ms): 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33 - AverageThreadTokens: 33.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 28.40 GB (30492155015) - PerHostPeakMemUsage: 28.40 GB (30492155015) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 1m7s - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 0.000ns - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 0 (0) Fragment Instance Lifecycle Timings: - OpenTime: 0.000ns - ExecTreeOpenTime: 0.000ns - PrepareTime: 70.992ms - ExecTreePrepareTime: 51.088ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 3.58K (3581) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.38 GB (173281017856) - PeakMemoryUsage: 27.98 GB (30039605248) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns HdfsTableSink:(Total: 204.118us, non-child: 204.118us, % non-child: 100.00%) - BytesWritten: 0 - CompressTimer: 0.000ns - EncodeTimer: 0.000ns - FilesCreated: 0 (0) - HdfsWriteTimer: 0.000ns - PartitionsCreated: 0 (0) - PeakMemoryUsage: 0 - RowsInserted: 0 (0) CodeGen:(Total: 59.574ms, non-child: 59.574ms, % non-child: 100.00%) - CodegenTime: 1.096ms - CompileTime: 7.664ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1995628) - NumFunctions: 33 (33) - NumInstructions: 621 (621) - OptimizationTime: 33.178ms - PeakMemoryUsage: 310.50 KB (317952) - PrepareTime: 18.035ms SORT_NODE (id=1):(Total: 51.087ms, non-child: 0.000ns, % non-child: 0.00%) ExecOption: Codegen Enabled - InMemorySortTime: 0.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 27.98 GB (30039705608) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 3s890ms, non-child: 3s890ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:329/2.37 GB 7:322/2.32 GB 6:299/2.16 GB 10:299/2.16 GB 11:334/2.42 GB 12:300/2.17 GB 5:328/2.37 GB 8:291/2.10 GB 2:309/2.23 GB 1:312/2.25 GB 3:326/2.36 GB 9:319/2.31 GB 13:323/2.33 GB 4:314/2.27 GB ExecOption: PARQUET Codegen Enabled BytesRead(2s000ms): 279.85 MB, 594.72 MB, 900.53 MB, 1.23 GB, 1.53 GB, 1.85 GB, 2.17 GB, 2.47 GB, 2.78 GB, 3.12 GB, 3.42 GB, 3.75 GB, 4.09 GB, 4.39 GB, 4.73 GB, 5.04 GB, 5.36 GB, 5.69 GB, 5.99 GB, 6.29 GB, 6.60 GB, 6.90 GB, 7.19 GB, 7.51 GB, 7.80 GB, 8.09 GB, 8.41 GB, 8.72 GB, 9.02 GB, 9.33 GB, 9.64 GB, 9.94 GB, 10.24 GB, 10.56 GB, 10.87 GB, 11.19 GB, 11.48 GB, 11.79 GB, 12.11 GB, 12.43 GB, 12.75 GB - FooterProcessingTime: (Avg: 16.401ms ; Min: 1.801ms ; Max: 281.281ms ; Number of samples: 1784) - AverageHdfsReadThreadConcurrency: 0.34 - AverageScannerThreadConcurrency: 32.00 - BytesRead: 13.06 GB (14023721007) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 17s180ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 0 (0) - NumRowGroups: 1.78K (1784) - NumScannerThreadsStarted: 32 (32) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 434.06 MB (455144582) - PerReadThreadRawHdfsThroughput: 398.36 MB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1155272387) - RowsReturned: 1.16B (1155047107) - RowsReturnedRate: 296.92 M/sec - ScanRangesComplete: 1.75K (1752) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 1m41s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 33s572ms - TotalReadThroughput: 160.77 MB/sec Instance 3b44783e08b26695:e9a6c65800000008 (host=vd0336.halxg.cloudera.com:22000):(Total: 55.338ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:288/2.08 GB 7:296/2.14 GB 1:303/2.19 GB 3:328/2.37 GB 13:322/2.33 GB 0:313/2.26 GB 6:312/2.26 GB 12:320/2.31 GB 2:317/2.29 GB 4:315/2.28 GB 8:325/2.35 GB 9:296/2.14 GB 11:344/2.49 GB 10:327/2.36 GB MemoryUsage(2s000ms): 517.06 MB, 1.23 GB, 1.85 GB, 2.48 GB, 3.12 GB, 3.78 GB, 4.45 GB, 5.14 GB, 5.81 GB, 6.50 GB, 7.18 GB, 7.87 GB, 8.55 GB, 9.24 GB, 9.96 GB, 10.67 GB, 11.37 GB, 12.07 GB, 12.76 GB, 13.45 GB, 14.15 GB, 14.82 GB, 15.52 GB, 16.22 GB, 16.91 GB, 17.61 GB, 18.31 GB, 19.00 GB, 19.72 GB, 20.41 GB, 21.08 GB, 21.78 GB, 22.50 GB, 23.12 GB, 23.77 GB, 24.49 GB, 25.21 GB, 25.93 GB, 26.65 GB ThreadUsage(2s000ms): 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33 - AverageThreadTokens: 33.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 27.44 GB (29461819783) - PerHostPeakMemUsage: 27.44 GB (29461819783) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 32s057ms - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 0.000ns - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 0 (0) Fragment Instance Lifecycle Timings: - OpenTime: 0.000ns - ExecTreeOpenTime: 0.000ns - PrepareTime: 55.287ms - ExecTreePrepareTime: 36.372ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 3.46K (3458) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.38 GB (173281017856) - PeakMemoryUsage: 27.02 GB (29007806464) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns HdfsTableSink:(Total: 241.276us, non-child: 241.276us, % non-child: 100.00%) - BytesWritten: 0 - CompressTimer: 0.000ns - EncodeTimer: 0.000ns - FilesCreated: 0 (0) - HdfsWriteTimer: 0.000ns - PartitionsCreated: 0 (0) - PeakMemoryUsage: 0 - RowsInserted: 0 (0) CodeGen:(Total: 58.091ms, non-child: 58.091ms, % non-child: 100.00%) - CodegenTime: 1.033ms - CompileTime: 7.416ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1995628) - NumFunctions: 33 (33) - NumInstructions: 621 (621) - OptimizationTime: 32.729ms - PeakMemoryUsage: 310.50 KB (317952) - PrepareTime: 17.268ms SORT_NODE (id=1):(Total: 36.371ms, non-child: 0.000ns, % non-child: 0.00%) ExecOption: Codegen Enabled - InMemorySortTime: 0.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 27.02 GB (29007906824) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 3s711ms, non-child: 3s711ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:288/2.08 GB 7:296/2.14 GB 1:303/2.19 GB 3:328/2.37 GB 13:322/2.33 GB 0:313/2.26 GB 6:312/2.26 GB 12:320/2.31 GB 2:317/2.29 GB 4:315/2.28 GB 8:325/2.35 GB 9:296/2.14 GB 11:344/2.49 GB 10:327/2.36 GB ExecOption: PARQUET Codegen Enabled BytesRead(2s000ms): 180.11 MB, 531.54 MB, 806.12 MB, 1.06 GB, 1.35 GB, 1.69 GB, 1.98 GB, 2.29 GB, 2.61 GB, 2.92 GB, 3.23 GB, 3.57 GB, 3.87 GB, 4.20 GB, 4.53 GB, 4.85 GB, 5.19 GB, 5.50 GB, 5.82 GB, 6.16 GB, 6.46 GB, 6.77 GB, 7.10 GB, 7.42 GB, 7.74 GB, 8.07 GB, 8.38 GB, 8.73 GB, 9.04 GB, 9.36 GB, 9.68 GB, 9.99 GB, 10.33 GB, 10.61 GB, 10.92 GB, 11.25 GB, 11.59 GB, 11.91 GB, 12.26 GB - FooterProcessingTime: (Avg: 7.196ms ; Min: 1.862ms ; Max: 224.580ms ; Number of samples: 1726) - AverageHdfsReadThreadConcurrency: 0.11 - AverageScannerThreadConcurrency: 32.00 - BytesRead: 12.64 GB (13568348980) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 16s536ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 0 (0) - NumRowGroups: 1.73K (1726) - NumScannerThreadsStarted: 32 (32) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 434.00 MB (455083164) - PerReadThreadRawHdfsThroughput: 1.09 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.12B (1115690324) - RowsReturned: 1.12B (1115464020) - RowsReturnedRate: 300.58 M/sec - ScanRangesComplete: 1.69K (1694) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 1m37s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 11s549ms - TotalReadThroughput: 162.75 MB/sec Instance 3b44783e08b26695:e9a6c65800000000 (host=vd0242.halxg.cloudera.com:22000):(Total: 59.101ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:315/2.28 GB 9:315/2.28 GB 6:326/2.35 GB 5:314/2.26 GB 0:331/2.39 GB 13:290/2.10 GB 3:304/2.20 GB 11:303/2.19 GB 1:327/2.37 GB 10:305/2.21 GB 8:319/2.30 GB 7:305/2.21 GB 2:315/2.27 GB 12:336/2.43 GB MemoryUsage(2s000ms): 746.29 MB, 1.43 GB, 2.13 GB, 2.82 GB, 3.52 GB, 4.23 GB, 4.94 GB, 5.65 GB, 6.36 GB, 7.05 GB, 7.75 GB, 8.48 GB, 9.20 GB, 9.90 GB, 10.58 GB, 11.28 GB, 11.99 GB, 12.71 GB, 13.40 GB, 14.11 GB, 14.81 GB, 15.51 GB, 16.21 GB, 16.91 GB, 17.61 GB, 18.30 GB, 19.00 GB, 19.70 GB, 20.38 GB, 21.09 GB, 21.79 GB, 22.42 GB, 23.08 GB, 23.78 GB, 24.50 GB, 25.21 GB, 25.92 GB, 26.64 GB, 27.35 GB ThreadUsage(2s000ms): 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33 - AverageThreadTokens: 33.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 27.99 GB (30057716615) - PerHostPeakMemUsage: 27.99 GB (30057716615) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 32s042ms - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 0.000ns - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 0 (0) Fragment Instance Lifecycle Timings: - OpenTime: 0.000ns - ExecTreeOpenTime: 0.000ns - PrepareTime: 59.047ms - ExecTreePrepareTime: 39.770ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 3.53K (3529) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.38 GB (173281017856) - PeakMemoryUsage: 27.57 GB (29603397632) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns HdfsTableSink:(Total: 173.420us, non-child: 173.420us, % non-child: 100.00%) - BytesWritten: 0 - CompressTimer: 0.000ns - EncodeTimer: 0.000ns - FilesCreated: 0 (0) - HdfsWriteTimer: 0.000ns - PartitionsCreated: 0 (0) - PeakMemoryUsage: 0 - RowsInserted: 0 (0) CodeGen:(Total: 58.768ms, non-child: 58.768ms, % non-child: 100.00%) - CodegenTime: 1.169ms - CompileTime: 7.152ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1995628) - NumFunctions: 33 (33) - NumInstructions: 621 (621) - OptimizationTime: 33.520ms - PeakMemoryUsage: 310.50 KB (317952) - PrepareTime: 17.415ms SORT_NODE (id=1):(Total: 39.769ms, non-child: 0.000ns, % non-child: 0.00%) ExecOption: Codegen Enabled - InMemorySortTime: 0.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 27.57 GB (29603563726) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 3s723ms, non-child: 3s723ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 4:315/2.28 GB 9:315/2.28 GB 6:326/2.35 GB 5:314/2.26 GB 0:331/2.39 GB 13:290/2.10 GB 3:304/2.20 GB 11:303/2.19 GB 1:327/2.37 GB 10:305/2.21 GB 8:319/2.30 GB 7:305/2.21 GB 2:315/2.27 GB 12:336/2.43 GB ExecOption: PARQUET Codegen Enabled BytesRead(2s000ms): 300.17 MB, 596.59 MB, 906.14 MB, 1.23 GB, 1.55 GB, 1.89 GB, 2.21 GB, 2.52 GB, 2.87 GB, 3.18 GB, 3.52 GB, 3.85 GB, 4.16 GB, 4.51 GB, 4.81 GB, 5.15 GB, 5.48 GB, 5.79 GB, 6.13 GB, 6.44 GB, 6.77 GB, 7.10 GB, 7.41 GB, 7.75 GB, 8.07 GB, 8.38 GB, 8.72 GB, 9.03 GB, 9.35 GB, 9.68 GB, 9.99 GB, 10.30 GB, 10.61 GB, 10.92 GB, 11.26 GB, 11.58 GB, 11.89 GB, 12.25 GB, 12.57 GB - FooterProcessingTime: (Avg: 6.807ms ; Min: 2.070ms ; Max: 207.396ms ; Number of samples: 1759) - AverageHdfsReadThreadConcurrency: 0.13 - AverageScannerThreadConcurrency: 32.00 - BytesRead: 12.88 GB (13826839274) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 16s559ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 0 (0) - NumRowGroups: 1.76K (1759) - NumScannerThreadsStarted: 32 (32) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 434.01 MB (455087230) - PerReadThreadRawHdfsThroughput: 1.08 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.14B (1138754152) - RowsReturned: 1.14B (1138527848) - RowsReturnedRate: 305.81 M/sec - ScanRangesComplete: 1.73K (1727) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 1m40s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 11s896ms - TotalReadThroughput: 166.92 MB/sec Instance 3b44783e08b26695:e9a6c65800000006 (host=vd0236.halxg.cloudera.com:22000):(Total: 55.810ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 2:313/2.26 GB 1:321/2.32 GB 4:316/2.28 GB 11:314/2.27 GB 6:324/2.34 GB 7:314/2.27 GB 8:300/2.17 GB 13:354/2.56 GB 3:309/2.24 GB 5:308/2.22 GB 12:313/2.26 GB 9:306/2.21 GB 10:315/2.28 GB 0:298/2.15 GB MemoryUsage(2s000ms): 562.94 MB, 1.34 GB, 2.03 GB, 2.73 GB, 3.43 GB, 4.13 GB, 4.82 GB, 5.54 GB, 6.26 GB, 6.97 GB, 7.68 GB, 8.36 GB, 9.09 GB, 9.80 GB, 10.51 GB, 11.19 GB, 11.88 GB, 12.55 GB, 13.24 GB, 13.92 GB, 14.61 GB, 15.29 GB, 15.96 GB, 16.64 GB, 17.31 GB, 17.96 GB, 18.60 GB, 19.24 GB, 19.84 GB, 20.53 GB, 21.22 GB, 21.80 GB, 22.47 GB, 23.12 GB, 23.76 GB, 24.40 GB, 25.07 GB, 25.75 GB, 26.43 GB ThreadUsage(2s000ms): 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33 - AverageThreadTokens: 33.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 27.09 GB (29084304971) - PerHostPeakMemUsage: 27.09 GB (29084304971) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 31s255ms - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 0.000ns - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 0 (0) Fragment Instance Lifecycle Timings: - OpenTime: 0.000ns - ExecTreeOpenTime: 0.000ns - PrepareTime: 55.740ms - ExecTreePrepareTime: 37.726ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 3.41K (3413) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.38 GB (173281017856) - PeakMemoryUsage: 26.66 GB (28630319104) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns HdfsTableSink:(Total: 174.923us, non-child: 174.923us, % non-child: 100.00%) - BytesWritten: 0 - CompressTimer: 0.000ns - EncodeTimer: 0.000ns - FilesCreated: 0 (0) - HdfsWriteTimer: 0.000ns - PartitionsCreated: 0 (0) - PeakMemoryUsage: 0 - RowsInserted: 0 (0) CodeGen:(Total: 55.978ms, non-child: 55.978ms, % non-child: 100.00%) - CodegenTime: 991.901us - CompileTime: 7.047ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1995628) - NumFunctions: 33 (33) - NumInstructions: 621 (621) - OptimizationTime: 31.987ms - PeakMemoryUsage: 310.50 KB (317952) - PrepareTime: 16.286ms SORT_NODE (id=1):(Total: 37.725ms, non-child: 0.000ns, % non-child: 0.00%) ExecOption: Codegen Enabled - InMemorySortTime: 0.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 26.66 GB (28630484998) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 3s660ms, non-child: 3s660ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:313/2.26 GB 1:321/2.32 GB 4:316/2.28 GB 11:314/2.27 GB 6:324/2.34 GB 7:314/2.27 GB 8:300/2.17 GB 13:354/2.56 GB 3:309/2.24 GB 5:308/2.22 GB 12:313/2.26 GB 9:306/2.21 GB 10:315/2.28 GB 0:298/2.15 GB ExecOption: PARQUET Codegen Enabled BytesRead(2s000ms): 198.03 MB, 569.41 MB, 875.19 MB, 1.20 GB, 1.52 GB, 1.82 GB, 2.17 GB, 2.48 GB, 2.82 GB, 3.16 GB, 3.46 GB, 3.80 GB, 4.12 GB, 4.46 GB, 4.78 GB, 5.09 GB, 5.43 GB, 5.73 GB, 6.04 GB, 6.37 GB, 6.68 GB, 6.99 GB, 7.31 GB, 7.62 GB, 7.92 GB, 8.23 GB, 8.53 GB, 8.83 GB, 9.10 GB, 9.42 GB, 9.73 GB, 10.01 GB, 10.32 GB, 10.62 GB, 10.91 GB, 11.20 GB, 11.52 GB, 11.83 GB, 12.14 GB - FooterProcessingTime: (Avg: 7.111ms ; Min: 2.084ms ; Max: 223.082ms ; Number of samples: 1698) - AverageHdfsReadThreadConcurrency: 0.11 - AverageScannerThreadConcurrency: 32.00 - BytesRead: 12.43 GB (13349615640) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 16s254ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 0 (0) - NumRowGroups: 1.70K (1698) - NumScannerThreadsStarted: 32 (32) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 434.26 MB (455353482) - PerReadThreadRawHdfsThroughput: 1.12 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.10B (1101267193) - RowsReturned: 1.10B (1101040889) - RowsReturnedRate: 300.75 M/sec - ScanRangesComplete: 1.67K (1666) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 1m38s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 11s082ms - TotalReadThroughput: 160.87 MB/sec Instance 3b44783e08b26695:e9a6c65800000003 (host=vd0342.halxg.cloudera.com:22000):(Total: 71.225ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:331/2.39 GB 2:300/2.17 GB 7:322/2.33 GB 8:287/2.07 GB 12:309/2.23 GB 9:332/2.40 GB 3:315/2.28 GB 10:323/2.33 GB 11:329/2.38 GB 13:315/2.28 GB 0:311/2.25 GB 5:288/2.08 GB 4:306/2.21 GB 6:336/2.43 GB MemoryUsage(2s000ms): 517.62 MB, 1.30 GB, 2.01 GB, 2.73 GB, 3.44 GB, 4.14 GB, 4.84 GB, 5.55 GB, 6.26 GB, 6.99 GB, 7.72 GB, 8.45 GB, 9.16 GB, 9.87 GB, 10.57 GB, 11.26 GB, 11.95 GB, 12.61 GB, 13.32 GB, 14.00 GB, 14.65 GB, 15.34 GB, 16.05 GB, 16.79 GB, 17.51 GB, 18.22 GB, 18.96 GB, 19.69 GB, 20.40 GB, 21.12 GB, 21.84 GB, 22.58 GB, 23.32 GB, 24.05 GB, 24.79 GB, 25.52 GB, 26.27 GB, 26.98 GB, 27.69 GB ThreadUsage(2s000ms): 25, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33 - AverageThreadTokens: 32.80 - BloomFilterBytes: 0 - PeakMemoryUsage: 28.51 GB (30610284872) - PerHostPeakMemUsage: 28.51 GB (30610284872) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 21s350ms - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 0.000ns - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 0 (0) Fragment Instance Lifecycle Timings: - OpenTime: 0.000ns - ExecTreeOpenTime: 0.000ns - PrepareTime: 71.187ms - ExecTreePrepareTime: 35.555ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 3.60K (3595) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.38 GB (173281017856) - PeakMemoryUsage: 28.09 GB (30157045760) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns HdfsTableSink:(Total: 19.238ms, non-child: 19.238ms, % non-child: 100.00%) - BytesWritten: 0 - CompressTimer: 0.000ns - EncodeTimer: 0.000ns - FilesCreated: 0 (0) - HdfsWriteTimer: 0.000ns - PartitionsCreated: 0 (0) - PeakMemoryUsage: 0 - RowsInserted: 0 (0) CodeGen:(Total: 49.486ms, non-child: 49.486ms, % non-child: 100.00%) - CodegenTime: 901.863us - CompileTime: 6.579ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1995628) - NumFunctions: 33 (33) - NumInstructions: 621 (621) - OptimizationTime: 27.161ms - PeakMemoryUsage: 310.50 KB (317952) - PrepareTime: 15.159ms SORT_NODE (id=1):(Total: 35.555ms, non-child: 0.000ns, % non-child: 0.00%) ExecOption: Codegen Enabled - InMemorySortTime: 0.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 28.09 GB (30157146120) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 3s495ms, non-child: 3s495ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:331/2.39 GB 2:300/2.17 GB 7:322/2.33 GB 8:287/2.07 GB 12:309/2.23 GB 9:332/2.40 GB 3:315/2.28 GB 10:323/2.33 GB 11:329/2.38 GB 13:315/2.28 GB 0:311/2.25 GB 5:288/2.08 GB 4:306/2.21 GB 6:336/2.43 GB ExecOption: PARQUET Codegen Enabled BytesRead(2s000ms): 181.98 MB, 542.19 MB, 856.11 MB, 1.21 GB, 1.52 GB, 1.82 GB, 2.17 GB, 2.47 GB, 2.83 GB, 3.16 GB, 3.48 GB, 3.85 GB, 4.15 GB, 4.51 GB, 4.81 GB, 5.12 GB, 5.46 GB, 5.75 GB, 6.07 GB, 6.40 GB, 6.69 GB, 7.01 GB, 7.36 GB, 7.68 GB, 8.03 GB, 8.34 GB, 8.70 GB, 9.02 GB, 9.36 GB, 9.71 GB, 10.01 GB, 10.37 GB, 10.70 GB, 11.06 GB, 11.38 GB, 11.73 GB, 12.07 GB, 12.39 GB, 12.73 GB - FooterProcessingTime: (Avg: 4.663ms ; Min: 1.355ms ; Max: 148.713ms ; Number of samples: 1791) - AverageHdfsReadThreadConcurrency: 0.09 - AverageScannerThreadConcurrency: 32.00 - BytesRead: 13.11 GB (14080021835) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 9s081ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 0 (0) - NumRowGroups: 1.79K (1791) - NumScannerThreadsStarted: 32 (32) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 433.98 MB (455059771) - PerReadThreadRawHdfsThroughput: 1.89 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.16B (1159948787) - RowsReturned: 1.16B (1159743914) - RowsReturnedRate: 331.80 M/sec - ScanRangesComplete: 1.76K (1759) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 52s651ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 6s952ms - TotalReadThroughput: 168.90 MB/sec Instance 3b44783e08b26695:e9a6c65800000009 (host=vd0240.halxg.cloudera.com:22000):(Total: 78.398ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:329/2.38 GB 0:334/2.41 GB 13:316/2.29 GB 7:295/2.13 GB 10:302/2.18 GB 4:284/2.05 GB 12:325/2.35 GB 2:335/2.42 GB 1:313/2.26 GB 6:330/2.38 GB 8:312/2.26 GB 11:293/2.11 GB 9:290/2.09 GB 3:348/2.52 GB MemoryUsage(2s000ms): 500.65 MB, 1.22 GB, 1.92 GB, 2.62 GB, 3.31 GB, 4.00 GB, 4.70 GB, 5.37 GB, 6.08 GB, 6.78 GB, 7.51 GB, 8.23 GB, 8.95 GB, 9.67 GB, 10.39 GB, 11.10 GB, 11.82 GB, 12.54 GB, 13.26 GB, 13.98 GB, 14.68 GB, 15.37 GB, 16.07 GB, 16.75 GB, 17.44 GB, 18.12 GB, 18.81 GB, 19.51 GB, 20.19 GB, 20.87 GB, 21.55 GB, 22.24 GB, 22.86 GB, 23.50 GB, 24.19 GB, 24.82 GB, 25.44 GB, 26.15 GB, 26.82 GB ThreadUsage(2s000ms): 25, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33 - AverageThreadTokens: 32.80 - BloomFilterBytes: 0 - PeakMemoryUsage: 27.60 GB (29637804616) - PerHostPeakMemUsage: 27.60 GB (29637804616) - RowsProduced: 0 (0) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 31s167ms - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 0.000ns - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 0 (0) Fragment Instance Lifecycle Timings: - OpenTime: 0.000ns - ExecTreeOpenTime: 0.000ns - PrepareTime: 78.342ms - ExecTreePrepareTime: 59.357ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 3.48K (3479) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 161.38 GB (173281017856) - PeakMemoryUsage: 27.18 GB (29183967232) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns HdfsTableSink:(Total: 213.641us, non-child: 213.641us, % non-child: 100.00%) - BytesWritten: 0 - CompressTimer: 0.000ns - EncodeTimer: 0.000ns - FilesCreated: 0 (0) - HdfsWriteTimer: 0.000ns - PartitionsCreated: 0 (0) - PeakMemoryUsage: 0 - RowsInserted: 0 (0) CodeGen:(Total: 59.349ms, non-child: 59.349ms, % non-child: 100.00%) - CodegenTime: 1.197ms - CompileTime: 7.409ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.90 MB (1995628) - NumFunctions: 33 (33) - NumInstructions: 621 (621) - OptimizationTime: 33.999ms - PeakMemoryUsage: 310.50 KB (317952) - PrepareTime: 17.380ms SORT_NODE (id=1):(Total: 59.355ms, non-child: 0.000ns, % non-child: 0.00%) ExecOption: Codegen Enabled - InMemorySortTime: 0.000ns - InitialRunsCreated: 1 (1) - PeakMemoryUsage: 27.18 GB (29184198670) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SortDataSize: 0 - SpilledRuns: 0 (0) - TotalMergesPerformed: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 3s768ms, non-child: 3s768ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 5:329/2.38 GB 0:334/2.41 GB 13:316/2.29 GB 7:295/2.13 GB 10:302/2.18 GB 4:284/2.05 GB 12:325/2.35 GB 2:335/2.42 GB 1:313/2.26 GB 6:330/2.38 GB 8:312/2.26 GB 11:293/2.11 GB 9:290/2.09 GB 3:348/2.52 GB ExecOption: PARQUET Codegen Enabled BytesRead(2s000ms): 180.11 MB, 529.90 MB, 839.08 MB, 1.13 GB, 1.47 GB, 1.77 GB, 2.09 GB, 2.41 GB, 2.72 GB, 3.07 GB, 3.40 GB, 3.72 GB, 4.07 GB, 4.39 GB, 4.73 GB, 5.06 GB, 5.39 GB, 5.72 GB, 6.06 GB, 6.40 GB, 6.71 GB, 7.04 GB, 7.36 GB, 7.66 GB, 7.99 GB, 8.31 GB, 8.62 GB, 8.94 GB, 9.25 GB, 9.58 GB, 9.90 GB, 10.21 GB, 10.49 GB, 10.79 GB, 11.12 GB, 11.41 GB, 11.69 GB, 12.02 GB, 12.33 GB - FooterProcessingTime: (Avg: 6.864ms ; Min: 2.074ms ; Max: 225.640ms ; Number of samples: 1733) - AverageHdfsReadThreadConcurrency: 0.14 - AverageScannerThreadConcurrency: 32.00 - BytesRead: 12.68 GB (13612738265) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 16s388ms - MaxCompressedTextFileLength: 0 - NumColumns: 4 (4) - NumDisksAccessed: 0 (0) - NumRowGroups: 1.73K (1733) - NumScannerThreadsStarted: 32 (32) - NumScannersWithNoReads: 0 (0) - PeakMemoryUsage: 434.20 MB (455292074) - PerReadThreadRawHdfsThroughput: 1.09 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.12B (1122630185) - RowsReturned: 1.12B (1122404905) - RowsReturnedRate: 297.87 M/sec - ScanRangesComplete: 1.70K (1701) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 1m39s - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 11s580ms - TotalReadThroughput: 163.30 MB/sec