Query (id=544466204e0abc59:f772f133ae0a68b4) Summary Session ID: e7415326ab8305f2:86c99f3d774066a5 Session Type: BEESWAX Start Time: 2015-05-15 22:56:56.105685000 End Time: 2015-05-29 16:31:40.122493000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.2.0-cdh5 RELEASE (build 2ffd73a4255cefd521362ffe1cfb37463f67f75c) User: hodesi01 Connected User: hodesi01 Delegated User: Network Address: ::ffff:172.29.46.14:36677 Default Db: default Sql Statement: select STRAIGHT_JOIN count(*) from flatbam_snappy bam JOIN [BROADCAST] reference_genome ref ON ref.contig = concat('chr', bam.contig__contigname) where ref.contig = 'chr20' and bam.contig__contigname = '20' and ref.`position` between bam.`start` and bam.`end` Coordinator: demeter-csmau08-10.demeter.hpc.mssm.edu:22000 Plan: ---------------- Estimated Per-Host Requirements: Memory=1.14GB VCores=2 F02:PLAN FRAGMENT [UNPARTITIONED] 06:AGGREGATE [FINALIZE] | output: count:merge(*) | hosts=91 per-host-mem=unavailable | tuple-ids=2 row-size=8B cardinality=1 | 05:EXCHANGE [UNPARTITIONED] hosts=91 per-host-mem=unavailable tuple-ids=2 row-size=8B cardinality=1 F00:PLAN FRAGMENT [RANDOM] DATASTREAM SINK [FRAGMENT=F02, EXCHANGE=05, UNPARTITIONED] 03:AGGREGATE | output: count(*) | hosts=91 per-host-mem=10.00MB | tuple-ids=2 row-size=8B cardinality=1 | 02:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: concat('chr', bam.contig__contigname) = ref.contig | other predicates: ref.position >= bam.start, ref.position <= bam.`end` | hosts=91 per-host-mem=896.42MB | tuple-ids=0,1 row-size=58B cardinality=10079481 | |--04:EXCHANGE [BROADCAST] | hosts=91 per-host-mem=0B | tuple-ids=1 row-size=25B cardinality=34859198 | 00:SCAN HDFS [default.flatbam_snappy bam, RANDOM] partitions=1/1 files=470 size=105.58GB predicates: bam.contig__contigname = '20' table stats: 856755914 rows total column stats: all hosts=91 per-host-mem=264.00MB tuple-ids=0 row-size=33B cardinality=10079481 F01:PLAN FRAGMENT [RANDOM] DATASTREAM SINK [FRAGMENT=F00, EXCHANGE=04, BROADCAST] 01:SCAN HDFS [default.reference_genome ref, RANDOM] partitions=1/1 files=91 size=12.63GB predicates: ref.contig = 'chr20' table stats: 3137327831 rows total column stats: all hosts=91 per-host-mem=176.00MB tuple-ids=1 row-size=25B cardinality=34859198 ---------------- Estimated Per-Host Mem: 1227276333 Estimated Per-Host VCores: 2 Admission result: Admitted immediately Request Pool: root.hodesi01 ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail --------------------------------------------------------------------------------------------------------------------------- 06:AGGREGATE 1 385.156ms 385.156ms 1 1 60.00 KB -1.00 B FINALIZE 05:EXCHANGE 1 1.823ms 1.823ms 90 1 0 -1.00 B UNPARTITIONED 03:AGGREGATE 90 716.263ms 1s944ms 90 1 37.47 MB 10.00 MB 02:HASH JOIN 90 89h23m 329h34m 1.76B 10.08M 4.17 GB 896.42 MB INNER JOIN, BROADCAST |--04:EXCHANGE 90 3s586ms 5s811ms 63.03M 34.86M 0 0 BROADCAST | 01:SCAN HDFS 76 2s994ms 7s163ms 63.03M 34.86M 199.63 MB 176.00 MB default.reference_genome ref 00:SCAN HDFS 90 159.914ms 402.754ms 17.38M 10.08M 263.78 MB 264.00 MB default.flatbam_snappy bam Planner Timeline Analysis finished: 5259924 Equivalence classes computed: 5474070 Single node plan created: 11394953 Distributed plan created: 13723149 Lineage info computed: 15178580 Planning finished: 19756500 Query Timeline Start execution: 108584 Planning finished: 27440946 Submit for admission: 44349315 Completed admission: 44520994 Ready to start remote fragments: 433019604 Remote fragments started: 925038857 Rows available: 1186483696772157 First row fetched: 1186483982129314 Unregister query: 1186484016821409 ImpalaServer - AsyncTotalTime: 0 - ClientFetchWaitTimer: 287860070 - InactiveTotalTime: 0 - RowMaterializationTimer: 12860 - TotalTime: 0 Execution Profile 544466204e0abc59:f772f133ae0a68b4 Fragment start latencies: count: 166, last: 0.017383ns, min: 0.008589ns, max: 0.437014ns, mean: 0.207376ns, stddev: 0.179489ns Per Node Peak Memory Usage: demeter-csmau08-19.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmaz08-13.demeter.hpc.mssm.edu:22000(4.39 GB) demeter-csmaz10-8.demeter.hpc.mssm.edu:22000(4.34 GB) demeter-csmaz10-5.demeter.hpc.mssm.edu:22000(4.32 GB) demeter-csmaz08-20.demeter.hpc.mssm.edu:22000(4.39 GB) demeter-csmau10-14.demeter.hpc.mssm.edu:22000(4.42 GB) demeter-csmaz11-11.demeter.hpc.mssm.edu:22000(4.34 GB) demeter-csmaz11-4.demeter.hpc.mssm.edu:22000(4.37 GB) demeter-csmau10-3.demeter.hpc.mssm.edu:22000(4.36 GB) demeter-csmaz08-18.demeter.hpc.mssm.edu:22000(4.40 GB) demeter-csmau08-2.demeter.hpc.mssm.edu:22000(4.32 GB) demeter-csmau10-20.demeter.hpc.mssm.edu:22000(4.36 GB) demeter-csmau08-12.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmau08-15.demeter.hpc.mssm.edu:22000(4.44 GB) demeter-csmau08-18.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmaz10-13.demeter.hpc.mssm.edu:22000(4.39 GB) demeter-csmau08-13.demeter.hpc.mssm.edu:22000(4.36 GB) demeter-csmaz08-4.demeter.hpc.mssm.edu:22000(4.41 GB) demeter-csmaz08-1.demeter.hpc.mssm.edu:22000(4.33 GB) demeter-csmaz08-6.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmaz08-2.demeter.hpc.mssm.edu:22000(4.39 GB) demeter-csmaz10-3.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmau10-16.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmaz08-3.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmau08-5.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmaz11-18.demeter.hpc.mssm.edu:22000(4.30 GB) demeter-csmau08-11.demeter.hpc.mssm.edu:22000(4.40 GB) demeter-csmau10-5.demeter.hpc.mssm.edu:22000(4.40 GB) demeter-csmaz08-19.demeter.hpc.mssm.edu:22000(4.41 GB) demeter-csmaz10-6.demeter.hpc.mssm.edu:22000(4.43 GB) demeter-csmaz08-16.demeter.hpc.mssm.edu:22000(4.32 GB) demeter-csmaz08-8.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmaz10-15.demeter.hpc.mssm.edu:22000(4.43 GB) demeter-csmaz10-17.demeter.hpc.mssm.edu:22000(4.34 GB) demeter-csmaz11-6.demeter.hpc.mssm.edu:22000(4.39 GB) demeter-csmau10-8.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmaz11-20.demeter.hpc.mssm.edu:22000(4.36 GB) demeter-csmau08-3.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmaz11-14.demeter.hpc.mssm.edu:22000(4.37 GB) demeter-csmau08-7.demeter.hpc.mssm.edu:22000(4.41 GB) demeter-csmaz10-2.demeter.hpc.mssm.edu:22000(4.40 GB) demeter-csmaz10-19.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmaz11-5.demeter.hpc.mssm.edu:22000(4.39 GB) demeter-csmau10-15.demeter.hpc.mssm.edu:22000(4.32 GB) demeter-csmaz10-10.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmau10-1.demeter.hpc.mssm.edu:22000(4.34 GB) demeter-csmau08-1.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmau10-11.demeter.hpc.mssm.edu:22000(4.39 GB) demeter-csmaz10-20.demeter.hpc.mssm.edu:22000(4.37 GB) demeter-csmaz08-17.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmaz11-17.demeter.hpc.mssm.edu:22000(4.39 GB) demeter-csmau10-12.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmaz11-3.demeter.hpc.mssm.edu:22000(4.41 GB) demeter-csmau08-9.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmau10-19.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmaz08-14.demeter.hpc.mssm.edu:22000(4.34 GB) demeter-csmau10-18.demeter.hpc.mssm.edu:22000(4.43 GB) demeter-csmau08-14.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmaz11-15.demeter.hpc.mssm.edu:22000(4.39 GB) demeter-csmaz08-7.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmau10-13.demeter.hpc.mssm.edu:22000(4.33 GB) demeter-csmaz11-7.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmau10-6.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmau10-10.demeter.hpc.mssm.edu:22000(4.44 GB) demeter-csmaz08-5.demeter.hpc.mssm.edu:22000(4.39 GB) demeter-csmaz10-16.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmaz08-11.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmaz10-14.demeter.hpc.mssm.edu:22000(4.40 GB) demeter-csmaz08-10.demeter.hpc.mssm.edu:22000(4.39 GB) demeter-csmaz08-15.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmaz11-16.demeter.hpc.mssm.edu:22000(4.31 GB) demeter-csmau10-7.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmaz11-10.demeter.hpc.mssm.edu:22000(4.36 GB) demeter-csmau08-16.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmaz11-2.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmaz11-1.demeter.hpc.mssm.edu:22000(4.40 GB) demeter-csmaz10-1.demeter.hpc.mssm.edu:22000(4.34 GB) demeter-csmaz11-8.demeter.hpc.mssm.edu:22000(4.43 GB) demeter-csmau10-2.demeter.hpc.mssm.edu:22000(4.36 GB) demeter-csmaz08-12.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmau08-20.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmaz10-12.demeter.hpc.mssm.edu:22000(4.36 GB) demeter-csmaz11-12.demeter.hpc.mssm.edu:22000(4.41 GB) demeter-csmau08-17.demeter.hpc.mssm.edu:22000(4.41 GB) demeter-csmau08-8.demeter.hpc.mssm.edu:22000(4.35 GB) demeter-csmau10-17.demeter.hpc.mssm.edu:22000(4.38 GB) demeter-csmaz08-9.demeter.hpc.mssm.edu:22000(4.37 GB) demeter-csmau10-4.demeter.hpc.mssm.edu:22000(4.34 GB) demeter-csmaz10-4.demeter.hpc.mssm.edu:22000(4.36 GB) demeter-csmau08-10.demeter.hpc.mssm.edu:22000(4.38 GB) - AsyncTotalTime: 0 - FinalizationTimer: 0 - InactiveTotalTime: 0 - TotalTime: 1186483680933245 Coordinator Fragment F02 - AsyncTotalTime: 0 - AverageThreadTokens: 1.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 77824 - PerHostPeakMemUsage: 0 - PrepareTime: 384791202 - RowsProduced: 1 - TotalCpuTime: 1591035031 - TotalNetworkReceiveTime: 1186482343686950 - TotalNetworkSendTime: 0 - TotalStorageWaitTime: 0 - TotalTime: 1186482730157394 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 1161058 - CompileTime: 12955485 - InactiveTotalTime: 0 - LoadTime: 188889 - ModuleFileSize: 2309684 - OptimizationTime: 413085581 - PrepareTime: 380631624 - TotalTime: 809757808 AGGREGATION_NODE (id=6) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 141026 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 61440 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 1186482729754161 EXCHANGE_NODE (id=5) - AsyncTotalTime: 0 - BytesReceived: 1440 - ConvertRowBatchTime: 331675 - DeserializeRowBatchTimer: 850274 - FirstBatchArrivalWaitTime: 646333094984 - InactiveTotalTime: 1186482342773729 - PeakMemoryUsage: 0 - RowsReturned: 90 - RowsReturnedRate: 0 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 1186482344597416 Averaged Fragment F00 split sizes: min: 728.19 MB, max: 1.42 GB, avg: 1.17 GB, stddev: 171.89 MB completion times: min:10m46s max:329h34m mean: 89h23m stddev:92h59m execution rates: min:1.19 KB/sec max:2.25 MB/sec mean:811.08 KB/sec stddev:945.26 KB/sec num instances: 90 - AsyncTotalTime: 0 - AverageThreadTokens: 1.8946440606087025 - InactiveTotalTime: 0 - PeakMemoryUsage: 4628232516 - PerHostPeakMemUsage: 4695341745 - PrepareTime: 369174485 - RowsProduced: 1 - TotalCpuTime: 650430490446452 - TotalNetworkReceiveTime: 631388618686 - TotalNetworkSendTime: 477713 - TotalStorageWaitTime: 131464437 - TotalTime: 321781942016533 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 336484 - CompileTime: 121981647 - InactiveTotalTime: 0 - LoadTime: 309711 - ModuleFileSize: 2309684 - OptimizationTime: 545330771 - PrepareTime: 354613088 - TotalTime: 1024823855 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 43743 - OverallThroughput: 303017 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9921 - ThriftTransmitTime(*): 373536 - TotalTime: 53419 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) - AsyncTotalTime: 0 - BuildTime: 228474418 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 18079976 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 321781940565090 HASH_JOIN_NODE (id=2) - AsyncTotalTime: 0 - BuildPartitionTime: 7605400343 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2940914977 - GetNewBlockTime: 8794922 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 193097 - ProbeRowsPartitioned: 0 - ProbeTime: 321134750184369 - RowsReturned: 19580501 - RowsReturnedRate: 34 - SpilledPartitions: 0 - TotalTime: 321781224301765 - UnpinTime: 1958 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2312754470 - DeserializeRowBatchTimer: 4759976891 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99426 - SendersBlockedTimer: 288517517 - SendersBlockedTotalTimer(*): 1974333474 - TotalTime: 633890621043 HDFS_SCAN_NODE (id=0) - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 5.315655949115675E-8 - AverageScannerThreadConcurrency: 0.4613727660417687 - BytesRead: 68220339 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68220339 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68220339 - DecompressionTime: 281462041 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 205901081 - PerReadThreadRawHdfsThroughput: 1692102063 - RemoteScanRanges: 0 - RowsRead: 9519510 - RowsReturned: 193097 - RowsReturnedRate: 5564313 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 6 - ScannerThreadsTotalWallClockTime: 328633165411515 - MaterializeTupleTime(*): 314381528294534 - ScannerThreadsSysTime: 22482811 - ScannerThreadsUserTime: 1106629111 - ScannerThreadsVoluntaryContextSwitches: 185 - TotalRawHdfsReadTime(*): 48721645 - TotalReadThroughput: 28902 - TotalTime: 159914183 Averaged Fragment F01 split sizes: min: 123.96 MB, max: 305.41 MB, avg: 170.22 MB, stddev: 52.21 MB completion times: min:2s848ms max:10m42s mean: 14s670ms stddev:1m12s execution rates: min:446.88 KB/sec max:79.36 MB/sec mean:42.20 MB/sec stddev:18.20 MB/sec num instances: 76 - AsyncTotalTime: 0 - AverageThreadTokens: 2.1248829144154056 - InactiveTotalTime: 0 - PeakMemoryUsage: 62579876 - PerHostPeakMemUsage: 578484403 - PrepareTime: 154762 - RowsProduced: 829283 - TotalCpuTime: 20031209365 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 10118916334 - TotalStorageWaitTime: 37253880 - TotalTime: 14635319320 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 1172428816 - InactiveTotalTime: 0 - NetworkThroughput(*): 2595507 - OverallThroughput: 43485122 - PeakMemoryUsage: 491040 - SerializeBatchTime: 225343610 - ThriftTransmitTime(*): 82422563595 - TotalTime: 11624789472 - UncompressedRowBatchSize: 2463262541 HDFS_SCAN_NODE (id=1) - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.04212634380685961 - AverageScannerThreadConcurrency: 1.1378810030713875 - BytesRead: 165464660 - BytesReadDataNodeCache: 0 - BytesReadLocal: 165464660 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 165464660 - DecompressionTime: 69633896 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 62070612 - PerReadThreadRawHdfsThroughput: 1803485884 - RemoteScanRanges: 0 - RowsRead: 41280629 - RowsReturned: 829283 - RowsReturnedRate: 4454277 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 235 - ScannerThreadsTotalWallClockTime: 15507379880 - MaterializeTupleTime(*): 15381994074 - ScannerThreadsSysTime: 78316592 - ScannerThreadsUserTime: 5002962565 - ScannerThreadsVoluntaryContextSwitches: 741 - TotalRawHdfsReadTime(*): 154675820 - TotalReadThroughput: 41084848 - TotalTime: 2994719096 Fragment F00 Instance 544466204e0abc59:f772f133ae0a68de (host=demeter-csmau08-1.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 20:1/242.53 MB 8:1/242.72 MB 6:1/242.52 MB 2:1/243.04 MB 4:1/242.88 MB 13:1/234.31 MB - AsyncTotalTime: 0 - AverageThreadTokens: 3.3034124629080117 - InactiveTotalTime: 0 - PeakMemoryUsage: 4637829728 - PerHostPeakMemUsage: 4704938592 - PrepareTime: 385560835 - RowsProduced: 1 - TotalCpuTime: 3918813240202444 - TotalNetworkReceiveTime: 631727494321 - TotalNetworkSendTime: 457358 - TotalStorageWaitTime: 161136852 - TotalTime: 1186482623645339 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 343929 - CompileTime: 104719370 - InactiveTotalTime: 0 - LoadTime: 174301 - ModuleFileSize: 2309684 - OptimizationTime: 514864654 - PrepareTime: 371064713 - TotalTime: 993784708 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 38634 - OverallThroughput: 353857 - PeakMemoryUsage: 16384 - SerializeBatchTime: 6232 - ThriftTransmitTime(*): 414135 - TotalTime: 45216 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 997730201 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31664064 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 1186482622785897 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646166420986 - BuildPartitionTime: 7373354390 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2844551666 - GetNewBlockTime: 3488815 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 772096 - ProbeRowsPartitioned: 0 - ProbeTime: 1185834219182275 - RowsReturned: 78304768 - RowsReturnedRate: 65 - SpilledPartitions: 0 - TotalTime: 1186480677944979 - UnpinTime: 1102 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2258491338 - DeserializeRowBatchTimer: 5068269667 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630570332653 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99380 - SendersBlockedTimer: 255567114 - SendersBlockedTotalTimer(*): 1268478882 - TotalTime: 634183429882 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 20:1/242.53 MB 8:1/242.72 MB 6:1/242.52 MB 2:1/243.04 MB 4:1/242.88 MB 13:1/234.31 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 2.302869826400919 - BytesRead: 82483962 - BytesReadDataNodeCache: 0 - BytesReadLocal: 82483962 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 82483962 - DecompressionTime: 446760043 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 215616096 - PerReadThreadRawHdfsThroughput: 1605576962 - RemoteScanRanges: 0 - RowsRead: 11471699 - RowsReturned: 772096 - RowsReturnedRate: 18585006 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 22 - ScannerThreadsTotalWallClockTime: 2732315718258477 - MaterializeTupleTime(*): 2698926561465996 - ScannerThreadsSysTime: 25993000 - ScannerThreadsUserTime: 1885709000 - ScannerThreadsVoluntaryContextSwitches: 691 - TotalRawHdfsReadTime(*): 51373409 - TotalReadThroughput: 69 - TotalTime: 41544027 Instance 544466204e0abc59:f772f133ae0a6907 (host=demeter-csmau10-3.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 16:1/242.87 MB 13:1/242.88 MB 14:1/242.81 MB 22:1/233.24 MB 12:1/242.95 MB 18:1/62.00 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.5440170207229373 - InactiveTotalTime: 0 - PeakMemoryUsage: 4612569064 - PerHostPeakMemUsage: 4679677928 - PrepareTime: 370055303 - RowsProduced: 1 - TotalCpuTime: 2580863788526071 - TotalNetworkReceiveTime: 631883217538 - TotalNetworkSendTime: 551428 - TotalStorageWaitTime: 116482655 - TotalTime: 1014729573064574 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 352965 - CompileTime: 137263790 - InactiveTotalTime: 0 - LoadTime: 134199 - ModuleFileSize: 2309684 - OptimizationTime: 538839904 - PrepareTime: 356458735 - TotalTime: 1035346574 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 39817 - OverallThroughput: 349551 - PeakMemoryUsage: 16384 - SerializeBatchTime: 7012 - ThriftTransmitTime(*): 401838 - TotalTime: 45773 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 665685480 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31613240 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 1014729572175757 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646093787931 - BuildPartitionTime: 7197709056 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2896683091 - GetNewBlockTime: 7594271 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 601168 - ProbeRowsPartitioned: 0 - ProbeTime: 1014081839047362 - RowsReturned: 60956168 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 1014728176161536 - UnpinTime: 2261 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2239103108 - DeserializeRowBatchTimer: 4696128856 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 631014494882 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99362 - SendersBlockedTimer: 303826041 - SendersBlockedTotalTimer(*): 1847001995 - TotalTime: 634300829389 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 16:1/242.87 MB 13:1/242.88 MB 14:1/242.81 MB 22:1/233.24 MB 12:1/242.95 MB 18:1/62.00 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.5433817926974835 - BytesRead: 72626346 - BytesReadDataNodeCache: 0 - BytesReadLocal: 72626346 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 72626346 - DecompressionTime: 355558792 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 190355432 - PerReadThreadRawHdfsThroughput: 1732294902 - RemoteScanRanges: 0 - RowsRead: 10055710 - RowsReturned: 601168 - RowsReturnedRate: 18565997 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 8 - ScannerThreadsTotalWallClockTime: 1566119443933750 - MaterializeTupleTime(*): 1540208974684201 - ScannerThreadsSysTime: 17994000 - ScannerThreadsUserTime: 1489769000 - ScannerThreadsVoluntaryContextSwitches: 536 - TotalRawHdfsReadTime(*): 41924932 - TotalReadThroughput: 71 - TotalTime: 32380054 Instance 544466204e0abc59:f772f133ae0a68e6 (host=demeter-csmaz08-11.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 0:1/242.76 MB 23:1/242.41 MB 2:1/242.72 MB 21:1/242.92 MB 4:1/242.63 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.533494354117617 - InactiveTotalTime: 0 - PeakMemoryUsage: 4604079792 - PerHostPeakMemUsage: 4671188656 - PrepareTime: 361962307 - RowsProduced: 1 - TotalCpuTime: 2432358633012690 - TotalNetworkReceiveTime: 631224705456 - TotalNetworkSendTime: 532380 - TotalStorageWaitTime: 118403268 - TotalTime: 960323942028284 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 336490 - CompileTime: 117154413 - InactiveTotalTime: 0 - LoadTime: 167914 - ModuleFileSize: 2309684 - OptimizationTime: 544963468 - PrepareTime: 348621420 - TotalTime: 1013420057 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 38157 - OverallThroughput: 336000 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9245 - ThriftTransmitTime(*): 419315 - TotalTime: 47619 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 720915210 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31637736 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 960323941232516 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646164331985 - BuildPartitionTime: 7686220653 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2899824154 - GetNewBlockTime: 8897636 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 588800 - ProbeRowsPartitioned: 0 - ProbeTime: 959676026742035 - RowsReturned: 59694424 - RowsReturnedRate: 62 - SpilledPartitions: 0 - TotalTime: 960322458064346 - UnpinTime: 1885 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2374679982 - DeserializeRowBatchTimer: 5442236557 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 629340775695 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99439 - SendersBlockedTimer: 276581875 - SendersBlockedTotalTimer(*): 1929677650 - TotalTime: 633804898239 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/242.76 MB 23:1/242.41 MB 2:1/242.72 MB 21:1/242.92 MB 4:1/242.63 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:5.208e-05% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 1.5622990167410756E-6 - AverageScannerThreadConcurrency: 1.5328231211922216 - BytesRead: 68575383 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68575383 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68575383 - DecompressionTime: 338419989 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 181866160 - PerReadThreadRawHdfsThroughput: 1578119953 - RemoteScanRanges: 0 - RowsRead: 9553714 - RowsReturned: 588800 - RowsReturnedRate: 16904196 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 12 - ScannerThreadsTotalWallClockTime: 1472019206588121 - MaterializeTupleTime(*): 1447224281490569 - ScannerThreadsSysTime: 15995000 - ScannerThreadsUserTime: 1431779000 - ScannerThreadsVoluntaryContextSwitches: 503 - TotalRawHdfsReadTime(*): 43453847 - TotalReadThroughput: 71 - TotalTime: 34831588 Instance 544466204e0abc59:f772f133ae0a68d9 (host=demeter-csmaz10-12.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 2:1/242.79 MB 19:1/242.59 MB 0:1/242.72 MB 22:1/242.74 MB 21:1/242.78 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.510174530892114 - InactiveTotalTime: 0 - PeakMemoryUsage: 4617809472 - PerHostPeakMemUsage: 4684918336 - PrepareTime: 362773949 - RowsProduced: 1 - TotalCpuTime: 2404315236675890 - TotalNetworkReceiveTime: 629565950594 - TotalNetworkSendTime: 527240 - TotalStorageWaitTime: 106393596 - TotalTime: 958076052393397 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 333561 - CompileTime: 117088543 - InactiveTotalTime: 0 - LoadTime: 147239 - ModuleFileSize: 2309684 - OptimizationTime: 544333687 - PrepareTime: 349173724 - TotalTime: 1013373164 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 38771 - OverallThroughput: 334917 - PeakMemoryUsage: 16384 - SerializeBatchTime: 6638 - ThriftTransmitTime(*): 412675 - TotalTime: 47773 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 673246416 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31534936 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 958076051523325 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646197868874 - BuildPartitionTime: 8954301342 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2909341607 - GetNewBlockTime: 11897057 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 574464 - ProbeRowsPartitioned: 0 - ProbeTime: 957428187045558 - RowsReturned: 58237510 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 958074615440236 - UnpinTime: 1998 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2698492168 - DeserializeRowBatchTimer: 6447280499 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 626716827193 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99640 - SendersBlockedTimer: 274434891 - SendersBlockedTotalTimer(*): 1653838989 - TotalTime: 632528508906 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 2:1/242.79 MB 19:1/242.59 MB 0:1/242.72 MB 22:1/242.74 MB 21:1/242.78 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.5095017021947403 - BytesRead: 68929481 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68929481 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68929481 - DecompressionTime: 254802157 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 195595840 - PerReadThreadRawHdfsThroughput: 1834793086 - RemoteScanRanges: 0 - RowsRead: 9628021 - RowsReturned: 574464 - RowsReturnedRate: 17570822 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 1446221996235704 - MaterializeTupleTime(*): 1421106354236174 - ScannerThreadsSysTime: 18995000 - ScannerThreadsUserTime: 1025841000 - ScannerThreadsVoluntaryContextSwitches: 498 - TotalRawHdfsReadTime(*): 37567986 - TotalReadThroughput: 71 - TotalTime: 32694201 Instance 544466204e0abc59:f772f133ae0a68e0 (host=demeter-csmaz10-10.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 3:1/242.91 MB 8:1/242.86 MB 1:1/243.04 MB 22:1/242.95 MB 4:1/242.72 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.452880219264086 - InactiveTotalTime: 0 - PeakMemoryUsage: 4605694464 - PerHostPeakMemUsage: 4672803328 - PrepareTime: 370942758 - RowsProduced: 1 - TotalCpuTime: 2268995276192234 - TotalNetworkReceiveTime: 631748635424 - TotalNetworkSendTime: 495209 - TotalStorageWaitTime: 98819731 - TotalTime: 925286796103599 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 328304 - CompileTime: 135747428 - InactiveTotalTime: 0 - LoadTime: 143902 - ModuleFileSize: 2309684 - OptimizationTime: 499318807 - PrepareTime: 357251225 - TotalTime: 995078901 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 42649 - OverallThroughput: 361108 - PeakMemoryUsage: 16384 - SerializeBatchTime: 7020 - ThriftTransmitTime(*): 375155 - TotalTime: 44308 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 609884735 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31795480 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 925286795225924 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646021645166 - BuildPartitionTime: 7294187938 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2779002409 - GetNewBlockTime: 9280680 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 553984 - ProbeRowsPartitioned: 0 - ProbeTime: 924639250826952 - RowsReturned: 56157157 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 925285479000641 - UnpinTime: 1952 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2318671967 - DeserializeRowBatchTimer: 4535298775 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630882070447 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99369 - SendersBlockedTimer: 257245335 - SendersBlockedTotalTimer(*): 1776976329 - TotalTime: 634252618049 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 3:1/242.91 MB 8:1/242.86 MB 1:1/243.04 MB 22:1/242.95 MB 4:1/242.72 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.4521834819939823 - BytesRead: 69472184 - BytesReadDataNodeCache: 0 - BytesReadLocal: 69472184 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 69472184 - DecompressionTime: 327076062 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 183480832 - PerReadThreadRawHdfsThroughput: 1837594207 - RemoteScanRanges: 0 - RowsRead: 9655923 - RowsReturned: 553984 - RowsReturnedRate: 17345352 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 16 - ScannerThreadsTotalWallClockTime: 1343693669732467 - MaterializeTupleTime(*): 1318281203430955 - ScannerThreadsSysTime: 20994000 - ScannerThreadsUserTime: 1475773000 - ScannerThreadsVoluntaryContextSwitches: 483 - TotalRawHdfsReadTime(*): 37806053 - TotalReadThroughput: 75 - TotalTime: 31938469 Instance 544466204e0abc59:f772f133ae0a68c9 (host=demeter-csmaz08-7.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 5:1/242.41 MB 10:1/242.89 MB 16:1/163.63 MB 6:1/242.83 MB 4:1/242.71 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.5358142503825243 - InactiveTotalTime: 0 - PeakMemoryUsage: 4599574384 - PerHostPeakMemUsage: 4666683248 - PrepareTime: 363287871 - RowsProduced: 1 - TotalCpuTime: 2296027643051197 - TotalNetworkReceiveTime: 631907309140 - TotalNetworkSendTime: 497649 - TotalStorageWaitTime: 141990454 - TotalTime: 905684659092803 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 325699 - CompileTime: 102430983 - InactiveTotalTime: 0 - LoadTime: 150493 - ModuleFileSize: 2309684 - OptimizationTime: 508883686 - PrepareTime: 349887506 - TotalTime: 963871251 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 41100 - OverallThroughput: 281670 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9845 - ThriftTransmitTime(*): 389286 - TotalTime: 56804 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 668261612 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31741032 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 905684658232792 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646147700191 - BuildPartitionTime: 7220268023 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2798476580 - GetNewBlockTime: 6230991 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 550912 - ProbeRowsPartitioned: 0 - ProbeTime: 905036842178668 - RowsReturned: 55836507 - RowsReturnedRate: 61 - SpilledPartitions: 0 - TotalTime: 905683235124323 - UnpinTime: 1282 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2258953042 - DeserializeRowBatchTimer: 4471480730 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 631052644996 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99353 - SendersBlockedTimer: 221233703 - SendersBlockedTotalTimer(*): 1516273014 - TotalTime: 634358733289 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 5:1/242.41 MB 10:1/242.89 MB 16:1/163.63 MB 6:1/242.83 MB 4:1/242.71 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.535102526511669 - BytesRead: 64272165 - BytesReadDataNodeCache: 0 - BytesReadLocal: 64272165 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 64272165 - DecompressionTime: 259120112 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 177360752 - PerReadThreadRawHdfsThroughput: 1477121362 - RemoteScanRanges: 0 - RowsRead: 8950484 - RowsReturned: 550912 - RowsReturnedRate: 18188672 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 16 - ScannerThreadsTotalWallClockTime: 1390328273122158 - MaterializeTupleTime(*): 1365241245150366 - ScannerThreadsSysTime: 14995000 - ScannerThreadsUserTime: 1085832000 - ScannerThreadsVoluntaryContextSwitches: 469 - TotalRawHdfsReadTime(*): 43511770 - TotalReadThroughput: 70 - TotalTime: 30288741 Instance 544466204e0abc59:f772f133ae0a68d1 (host=demeter-csmaz11-3.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 2:1/242.81 MB 13:1/234.01 MB 22:1/242.66 MB 6:1/242.70 MB 14:1/234.70 MB 3:1/242.97 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.5365702609694734 - InactiveTotalTime: 0 - PeakMemoryUsage: 4666995448 - PerHostPeakMemUsage: 4734104312 - PrepareTime: 364614685 - RowsProduced: 1 - TotalCpuTime: 2291490866088677 - TotalNetworkReceiveTime: 631946161972 - TotalNetworkSendTime: 538823 - TotalStorageWaitTime: 146438402 - TotalTime: 903628709545485 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 323981 - CompileTime: 116261109 - InactiveTotalTime: 0 - LoadTime: 151192 - ModuleFileSize: 2309684 - OptimizationTime: 516059408 - PrepareTime: 350849014 - TotalTime: 985941997 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 37744 - OverallThroughput: 302063 - PeakMemoryUsage: 16384 - SerializeBatchTime: 7538 - ThriftTransmitTime(*): 423908 - TotalTime: 52969 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 623180970 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31456808 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 903628708663747 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646460754485 - BuildPartitionTime: 7158922886 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3146370279 - GetNewBlockTime: 7961321 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 539648 - ProbeRowsPartitioned: 0 - ProbeTime: 902980650128906 - RowsReturned: 54754600 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 903627341264149 - UnpinTime: 868 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2258568700 - DeserializeRowBatchTimer: 4514080928 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 631083481361 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99349 - SendersBlockedTimer: 244159073 - SendersBlockedTotalTimer(*): 1675142189 - TotalTime: 634384453091 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 2:1/242.81 MB 13:1/234.01 MB 22:1/242.66 MB 6:1/242.70 MB 14:1/234.70 MB 3:1/242.97 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.5358569154717867 - BytesRead: 82151197 - BytesReadDataNodeCache: 0 - BytesReadLocal: 82151197 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 82151197 - DecompressionTime: 304305315 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 244781816 - PerReadThreadRawHdfsThroughput: 1846957071 - RemoteScanRanges: 0 - RowsRead: 11416872 - RowsReturned: 539648 - RowsReturnedRate: 17233701 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 9 - ScannerThreadsTotalWallClockTime: 1387847155023277 - MaterializeTupleTime(*): 1352156882859024 - ScannerThreadsSysTime: 16994000 - ScannerThreadsUserTime: 1251807000 - ScannerThreadsVoluntaryContextSwitches: 472 - TotalRawHdfsReadTime(*): 44479213 - TotalReadThroughput: 90 - TotalTime: 31313528 Instance 544466204e0abc59:f772f133ae0a68ff (host=demeter-csmau08-13.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 17:1/242.95 MB 10:1/234.66 MB 2:1/242.83 MB 7:1/66.41 MB 16:1/242.86 MB 6:1/242.97 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.530074927192966 - InactiveTotalTime: 0 - PeakMemoryUsage: 4619371672 - PerHostPeakMemUsage: 4686480536 - PrepareTime: 366790991 - RowsProduced: 1 - TotalCpuTime: 2261256363451456 - TotalNetworkReceiveTime: 631923092873 - TotalNetworkSendTime: 520187 - TotalStorageWaitTime: 127969686 - TotalTime: 893995125397995 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 336325 - CompileTime: 115244477 - InactiveTotalTime: 0 - LoadTime: 156000 - ModuleFileSize: 2309684 - OptimizationTime: 508390487 - PrepareTime: 353288925 - TotalTime: 979562330 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 39685 - OverallThroughput: 331262 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9041 - ThriftTransmitTime(*): 403166 - TotalTime: 48300 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 627720527 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31790600 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 893995124549294 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646225679252 - BuildPartitionTime: 7207539983 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2860228375 - GetNewBlockTime: 7127126 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 539644 - ProbeRowsPartitioned: 0 - ProbeTime: 893347309830679 - RowsReturned: 54744855 - RowsReturnedRate: 61 - SpilledPartitions: 0 - TotalTime: 893993776670353 - UnpinTime: 880 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2247606567 - DeserializeRowBatchTimer: 4862483401 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630883472774 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99353 - SendersBlockedTimer: 245360794 - SendersBlockedTotalTimer(*): 1050591189 - TotalTime: 634358009622 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 17:1/242.95 MB 10:1/234.66 MB 2:1/242.83 MB 7:1/66.41 MB 16:1/242.86 MB 6:1/242.97 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.5293538823002908 - BytesRead: 72924193 - BytesReadDataNodeCache: 0 - BytesReadLocal: 72924193 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 72924193 - DecompressionTime: 260148240 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 197158040 - PerReadThreadRawHdfsThroughput: 1728618412 - RemoteScanRanges: 0 - RowsRead: 10126996 - RowsReturned: 539644 - RowsReturnedRate: 13304999 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 15 - ScannerThreadsTotalWallClockTime: 1367246438750104 - MaterializeTupleTime(*): 1332493009860604 - ScannerThreadsSysTime: 14995000 - ScannerThreadsUserTime: 1129825000 - ScannerThreadsVoluntaryContextSwitches: 476 - TotalRawHdfsReadTime(*): 42186403 - TotalReadThroughput: 81 - TotalTime: 40559489 Instance 544466204e0abc59:f772f133ae0a68e5 (host=demeter-csmaz10-16.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 8:1/242.80 MB 10:1/242.93 MB 13:1/243.10 MB 9:1/242.82 MB 20:1/242.88 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.4542354420935064 - InactiveTotalTime: 0 - PeakMemoryUsage: 4607148296 - PerHostPeakMemUsage: 4674257160 - PrepareTime: 377158913 - RowsProduced: 1 - TotalCpuTime: 2124713396750277 - TotalNetworkReceiveTime: 631649699385 - TotalNetworkSendTime: 493094 - TotalStorageWaitTime: 115260314 - TotalTime: 865985968069963 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 339860 - CompileTime: 136605818 - InactiveTotalTime: 0 - LoadTime: 154967 - ModuleFileSize: 2309684 - OptimizationTime: 501794673 - PrepareTime: 363557140 - TotalTime: 1004588229 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 40540 - OverallThroughput: 326404 - PeakMemoryUsage: 16384 - SerializeBatchTime: 10380 - ThriftTransmitTime(*): 394665 - TotalTime: 49019 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 634063888 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31693216 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 865985967207600 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646299727623 - BuildPartitionTime: 7450967573 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2994621557 - GetNewBlockTime: 8461797 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 515362 - ProbeRowsPartitioned: 0 - ProbeTime: 865338094024892 - RowsReturned: 52235142 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 865984601676260 - UnpinTime: 2193 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2258525818 - DeserializeRowBatchTimer: 4634582224 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630754344667 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99395 - SendersBlockedTimer: 267484843 - SendersBlockedTotalTimer(*): 1324563673 - TotalTime: 634090686452 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 8:1/242.80 MB 10:1/242.93 MB 13:1/243.10 MB 9:1/242.82 MB 20:1/242.88 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.4534904134904134 - BytesRead: 69294556 - BytesReadDataNodeCache: 0 - BytesReadLocal: 69294556 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 69294556 - DecompressionTime: 315361490 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 184934664 - PerReadThreadRawHdfsThroughput: 1613433407 - RemoteScanRanges: 0 - RowsRead: 9666858 - RowsReturned: 515362 - RowsReturnedRate: 13111142 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 11 - ScannerThreadsTotalWallClockTime: 1258712254451283 - MaterializeTupleTime(*): 1233131477516494 - ScannerThreadsSysTime: 13996000 - ScannerThreadsUserTime: 1324796000 - ScannerThreadsVoluntaryContextSwitches: 433 - TotalRawHdfsReadTime(*): 42948507 - TotalReadThroughput: 80 - TotalTime: 39307176 Instance 544466204e0abc59:f772f133ae0a68fc (host=demeter-csmaz08-6.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 12:1/242.89 MB 19:1/242.81 MB 21:1/237.62 MB 8:1/242.99 MB 14:1/242.85 MB 18:1/62.39 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.4616973784334495 - InactiveTotalTime: 0 - PeakMemoryUsage: 4632146184 - PerHostPeakMemUsage: 4699255048 - PrepareTime: 368568806 - RowsProduced: 1 - TotalCpuTime: 2088029758271723 - TotalNetworkReceiveTime: 631581746512 - TotalNetworkSendTime: 586455 - TotalStorageWaitTime: 146080090 - TotalTime: 848461882784772 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 330566 - CompileTime: 136362534 - InactiveTotalTime: 0 - LoadTime: 169923 - ModuleFileSize: 2309684 - OptimizationTime: 504562071 - PrepareTime: 355158153 - TotalTime: 998778564 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 33957 - OverallThroughput: 338302 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9232 - ThriftTransmitTime(*): 471183 - TotalTime: 47295 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 604267704 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31531096 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 848461881886135 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646892002714 - BuildPartitionTime: 7403841808 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3601888397 - GetNewBlockTime: 8661778 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 504757 - ProbeRowsPartitioned: 0 - ProbeTime: 847813458155887 - RowsReturned: 51110923 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 848460565805124 - UnpinTime: 2183 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2368438328 - DeserializeRowBatchTimer: 4430517265 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630742507547 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99388 - SendersBlockedTimer: 266153608 - SendersBlockedTotalTimer(*): 1584707564 - TotalTime: 634130669591 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 12:1/242.89 MB 19:1/242.81 MB 21:1/237.62 MB 8:1/242.99 MB 14:1/242.85 MB 18:1/62.39 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.4609369796514045 - BytesRead: 71782018 - BytesReadDataNodeCache: 0 - BytesReadLocal: 71782018 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 71782018 - DecompressionTime: 260981031 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 209932552 - PerReadThreadRawHdfsThroughput: 1391798161 - RemoteScanRanges: 0 - RowsRead: 10047736 - RowsReturned: 504757 - RowsReturnedRate: 10868172 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 8 - ScannerThreadsTotalWallClockTime: 1239552069218449 - MaterializeTupleTime(*): 1214244112953352 - ScannerThreadsSysTime: 17994000 - ScannerThreadsUserTime: 1090832000 - ScannerThreadsVoluntaryContextSwitches: 443 - TotalRawHdfsReadTime(*): 51575020 - TotalReadThroughput: 84 - TotalTime: 46443596 Instance 544466204e0abc59:f772f133ae0a68d4 (host=demeter-csmaz08-17.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 10:1/242.78 MB 20:1/235.47 MB 12:1/242.82 MB 8:1/243.03 MB 23:1/239.59 MB 15:1/242.82 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.447373944996539 - InactiveTotalTime: 0 - PeakMemoryUsage: 4639127008 - PerHostPeakMemUsage: 4706235872 - PrepareTime: 362148079 - RowsProduced: 1 - TotalCpuTime: 2075135259286803 - TotalNetworkReceiveTime: 631929259888 - TotalNetworkSendTime: 492211 - TotalStorageWaitTime: 154153053 - TotalTime: 848158956326604 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 321866 - CompileTime: 136976263 - InactiveTotalTime: 0 - LoadTime: 118478 - ModuleFileSize: 2309684 - OptimizationTime: 504300818 - PrepareTime: 348921803 - TotalTime: 992801834 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 41877 - OverallThroughput: 319501 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9955 - ThriftTransmitTime(*): 382068 - TotalTime: 50078 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 661066139 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31506672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 848158955394719 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646390664480 - BuildPartitionTime: 7203763044 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3028514968 - GetNewBlockTime: 8175513 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 507120 - ProbeRowsPartitioned: 0 - ProbeTime: 847510945318856 - RowsReturned: 51447641 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 848157579721268 - UnpinTime: 2328 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2244613266 - DeserializeRowBatchTimer: 4640557308 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 631029816947 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99355 - SendersBlockedTimer: 252794565 - SendersBlockedTotalTimer(*): 1736858563 - TotalTime: 634346017206 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 10:1/242.78 MB 20:1/235.47 MB 12:1/242.82 MB 8:1/243.03 MB 23:1/239.59 MB 15:1/242.82 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.4466132535832128 - BytesRead: 82737343 - BytesReadDataNodeCache: 0 - BytesReadLocal: 82737343 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 82737343 - DecompressionTime: 429243328 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 216913376 - PerReadThreadRawHdfsThroughput: 1624919097 - RemoteScanRanges: 0 - RowsRead: 11466773 - RowsReturned: 507120 - RowsReturnedRate: 10917041 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 11 - ScannerThreadsTotalWallClockTime: 1226961353375718 - MaterializeTupleTime(*): 1191434148346715 - ScannerThreadsSysTime: 16994000 - ScannerThreadsUserTime: 1740732000 - ScannerThreadsVoluntaryContextSwitches: 435 - TotalRawHdfsReadTime(*): 50917823 - TotalReadThroughput: 97 - TotalTime: 46452144 Instance 544466204e0abc59:f772f133ae0a68f1 (host=demeter-csmaz08-16.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 12:1/242.84 MB 4:1/242.45 MB 13:1/242.82 MB 17:1/243.04 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.4217519292545693 - InactiveTotalTime: 0 - PeakMemoryUsage: 4568853800 - PerHostPeakMemUsage: 4635962664 - PrepareTime: 373046197 - RowsProduced: 1 - TotalCpuTime: 2019061648525489 - TotalNetworkReceiveTime: 631445466159 - TotalNetworkSendTime: 468163 - TotalStorageWaitTime: 84697634 - TotalTime: 833975728938596 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 322393 - CompileTime: 114259212 - InactiveTotalTime: 0 - LoadTime: 160436 - ModuleFileSize: 2309684 - OptimizationTime: 501891840 - PrepareTime: 359394844 - TotalTime: 978290081 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 43257 - OverallThroughput: 308404 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9273 - ThriftTransmitTime(*): 369880 - TotalTime: 51880 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 595958923 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31718800 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 833975727959468 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646161458700 - BuildPartitionTime: 7611664957 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2846011999 - GetNewBlockTime: 6840140 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 495781 - ProbeRowsPartitioned: 0 - ProbeTime: 833328056448817 - RowsReturned: 50251553 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 833974440209403 - UnpinTime: 1024 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2292637855 - DeserializeRowBatchTimer: 4648178501 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630446467828 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99420 - SendersBlockedTimer: 243221543 - SendersBlockedTotalTimer(*): 1667438116 - TotalTime: 633926320065 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 12:1/242.84 MB 4:1/242.45 MB 13:1/242.82 MB 17:1/243.04 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:12 ExecOption: Codegen enabled: 0 out of 4 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.4209788733703483 - BytesRead: 55380516 - BytesReadDataNodeCache: 0 - BytesReadLocal: 55380516 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 55380516 - DecompressionTime: 186711219 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 4 - PeakMemoryUsage: 146640168 - PerReadThreadRawHdfsThroughput: 1735146106 - RemoteScanRanges: 0 - RowsRead: 7701887 - RowsReturned: 495781 - RowsReturnedRate: 16506178 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 10 - ScannerThreadsTotalWallClockTime: 1185070671145897 - MaterializeTupleTime(*): 1168000527483656 - ScannerThreadsSysTime: 8996000 - ScannerThreadsUserTime: 868866000 - ScannerThreadsVoluntaryContextSwitches: 398 - TotalRawHdfsReadTime(*): 31916918 - TotalReadThroughput: 66 - TotalTime: 30036087 Instance 544466204e0abc59:f772f133ae0a6905 (host=demeter-csmau08-2.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 13:1/242.91 MB 9:1/242.96 MB 12:1/242.83 MB 16:1/242.95 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.4666847393565483 - InactiveTotalTime: 0 - PeakMemoryUsage: 4576121112 - PerHostPeakMemUsage: 4643229976 - PrepareTime: 367765460 - RowsProduced: 1 - TotalCpuTime: 2051651378490822 - TotalNetworkReceiveTime: 631807860547 - TotalNetworkSendTime: 571952 - TotalStorageWaitTime: 86075217 - TotalTime: 831997621804608 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 335530 - CompileTime: 135710917 - InactiveTotalTime: 0 - LoadTime: 180861 - ModuleFileSize: 2309684 - OptimizationTime: 503412615 - PrepareTime: 354148505 - TotalTime: 995937904 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 34467 - OverallThroughput: 349337 - PeakMemoryUsage: 16384 - SerializeBatchTime: 6605 - ThriftTransmitTime(*): 464208 - TotalTime: 45801 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 547912155 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31718528 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 831997620976909 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646211028042 - BuildPartitionTime: 7288153904 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2900360884 - GetNewBlockTime: 7513502 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 494482 - ProbeRowsPartitioned: 0 - ProbeTime: 831350002515301 - RowsReturned: 50161115 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 831996413914961 - UnpinTime: 2120 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2278709024 - DeserializeRowBatchTimer: 4707020320 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630936708841 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99368 - SendersBlockedTimer: 264088112 - SendersBlockedTotalTimer(*): 1567219317 - TotalTime: 634260867034 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 13:1/242.91 MB 9:1/242.96 MB 12:1/242.83 MB 16:1/242.95 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:12 ExecOption: Codegen enabled: 0 out of 4 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.4659098969134132 - BytesRead: 55451794 - BytesReadDataNodeCache: 0 - BytesReadLocal: 55451794 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 55451794 - DecompressionTime: 227377365 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 4 - PeakMemoryUsage: 153907480 - PerReadThreadRawHdfsThroughput: 1814452588 - RemoteScanRanges: 0 - RowsRead: 7705247 - RowsReturned: 494482 - RowsReturnedRate: 16306285 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 10 - ScannerThreadsTotalWallClockTime: 1219638799364900 - MaterializeTupleTime(*): 1202136224726831 - ScannerThreadsSysTime: 19994000 - ScannerThreadsUserTime: 956854000 - ScannerThreadsVoluntaryContextSwitches: 401 - TotalRawHdfsReadTime(*): 30561170 - TotalReadThroughput: 66 - TotalTime: 30324626 Instance 544466204e0abc59:f772f133ae0a68f7 (host=demeter-csmau08-5.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 12:1/242.37 MB 16:1/234.34 MB 15:1/242.79 MB 13:1/234.70 MB 2:1/242.80 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.4589133229359263 - InactiveTotalTime: 0 - PeakMemoryUsage: 4603882152 - PerHostPeakMemUsage: 4670991016 - PrepareTime: 367114881 - RowsProduced: 1 - TotalCpuTime: 1937427794469995 - TotalNetworkReceiveTime: 629871114666 - TotalNetworkSendTime: 515132 - TotalStorageWaitTime: 141832628 - TotalTime: 788174853542135 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 330084 - CompileTime: 136936132 - InactiveTotalTime: 0 - LoadTime: 164702 - ModuleFileSize: 2309684 - OptimizationTime: 518565762 - PrepareTime: 353427998 - TotalTime: 1011759150 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 37723 - OverallThroughput: 314911 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9643 - ThriftTransmitTime(*): 424136 - TotalTime: 50808 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 590911856 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31352608 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 788174852712359 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646298693686 - BuildPartitionTime: 8699728612 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3023043787 - GetNewBlockTime: 10400753 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 479232 - ProbeRowsPartitioned: 0 - ProbeTime: 787527047781973 - RowsReturned: 48601805 - RowsReturnedRate: 61 - SpilledPartitions: 0 - TotalTime: 788173561690879 - UnpinTime: 1982 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2627811135 - DeserializeRowBatchTimer: 6173116227 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 627280787770 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99605 - SendersBlockedTimer: 278348507 - SendersBlockedTotalTimer(*): 1420887965 - TotalTime: 632749447939 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 12:1/242.37 MB 16:1/234.34 MB 15:1/242.79 MB 13:1/234.70 MB 2:1/242.80 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.458094759497269 - BytesRead: 67200654 - BytesReadDataNodeCache: 0 - BytesReadLocal: 67200654 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 67200654 - DecompressionTime: 302327280 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 181668520 - PerReadThreadRawHdfsThroughput: 1160858833 - RemoteScanRanges: 0 - RowsRead: 9527836 - RowsReturned: 479232 - RowsReturnedRate: 11809178 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 9 - ScannerThreadsTotalWallClockTime: 1149235998598194 - MaterializeTupleTime(*): 1124669988864456 - ScannerThreadsSysTime: 12995000 - ScannerThreadsUserTime: 1276803000 - ScannerThreadsVoluntaryContextSwitches: 400 - TotalRawHdfsReadTime(*): 57888739 - TotalReadThroughput: 85 - TotalTime: 40581315 Instance 544466204e0abc59:f772f133ae0a68f2 (host=demeter-csmaz08-19.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 19:1/62.41 MB 7:1/242.78 MB 21:1/233.76 MB 22:1/235.44 MB 0:1/234.40 MB 1:1/164.56 MB 15:1/242.69 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.149064781541534 - InactiveTotalTime: 0 - PeakMemoryUsage: 4673079440 - PerHostPeakMemUsage: 4740188304 - PrepareTime: 359929206 - RowsProduced: 1 - TotalCpuTime: 1650620904513828 - TotalNetworkReceiveTime: 631642068434 - TotalNetworkSendTime: 491260 - TotalStorageWaitTime: 154991491 - TotalTime: 768356520702740 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 323189 - CompileTime: 105782666 - InactiveTotalTime: 0 - LoadTime: 120509 - ModuleFileSize: 2309684 - OptimizationTime: 504050291 - PrepareTime: 346449821 - TotalTime: 958999594 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 42169 - OverallThroughput: 333667 - PeakMemoryUsage: 16384 - SerializeBatchTime: 8875 - ThriftTransmitTime(*): 379422 - TotalTime: 47952 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 526062712 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 39291608 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 768356519727988 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646173940269 - BuildPartitionTime: 7404489873 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2845908161 - GetNewBlockTime: 6019160 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 452148 - ProbeRowsPartitioned: 0 - ProbeTime: 767708975671706 - RowsReturned: 45836968 - RowsReturnedRate: 59 - SpilledPartitions: 0 - TotalTime: 768355354200532 - UnpinTime: 1948 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2329956120 - DeserializeRowBatchTimer: 4373349625 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630867583192 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99384 - SendersBlockedTimer: 224075597 - SendersBlockedTotalTimer(*): 1535342742 - TotalTime: 634158895060 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 19:1/62.41 MB 7:1/242.78 MB 21:1/233.76 MB 22:1/235.44 MB 0:1/234.40 MB 1:1/164.56 MB 15:1/242.69 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:21 ExecOption: Codegen enabled: 0 out of 7 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.148225337921617 - BytesRead: 81162377 - BytesReadDataNodeCache: 0 - BytesReadLocal: 81162377 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 81162377 - DecompressionTime: 277046110 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 7 - PeakMemoryUsage: 250865808 - PerReadThreadRawHdfsThroughput: 1800511737 - RemoteScanRanges: 0 - RowsRead: 11241245 - RowsReturned: 452148 - RowsReturnedRate: 15698918 - ScanRangesComplete: 7 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 882249396022511 - MaterializeTupleTime(*): 833032904325036 - ScannerThreadsSysTime: 22993000 - ScannerThreadsUserTime: 1139823000 - ScannerThreadsVoluntaryContextSwitches: 411 - TotalRawHdfsReadTime(*): 45077394 - TotalReadThroughput: 105 - TotalTime: 28801219 Instance 544466204e0abc59:f772f133ae0a68cc (host=demeter-csmaz11-15.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 18:1/7.18 MB 10:1/242.74 MB 14:1/243.04 MB 6:1/242.82 MB 12:1/243.06 MB 17:1/234.34 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.6533050949452304 - InactiveTotalTime: 0 - PeakMemoryUsage: 4643009856 - PerHostPeakMemUsage: 4710118720 - PrepareTime: 384724198 - RowsProduced: 1 - TotalCpuTime: 954310736541027 - TotalNetworkReceiveTime: 631656422527 - TotalNetworkSendTime: 525856 - TotalStorageWaitTime: 125492717 - TotalTime: 577592552793214 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 320889 - CompileTime: 138522274 - InactiveTotalTime: 0 - LoadTime: 201090 - ModuleFileSize: 2309684 - OptimizationTime: 508998642 - PrepareTime: 371397009 - TotalTime: 1021573091 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 37686 - OverallThroughput: 298892 - PeakMemoryUsage: 16384 - SerializeBatchTime: 10035 - ThriftTransmitTime(*): 424550 - TotalTime: 53531 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 411018491 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31634648 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 577592551923632 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646127195225 - BuildPartitionTime: 7389571978 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2886117417 - GetNewBlockTime: 7449677 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 340992 - ProbeRowsPartitioned: 0 - ProbeTime: 576945212567814 - RowsReturned: 34546611 - RowsReturnedRate: 59 - SpilledPartitions: 0 - TotalTime: 577591517929670 - UnpinTime: 2009 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2283656938 - DeserializeRowBatchTimer: 4453214882 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630833175675 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99392 - SendersBlockedTimer: 280409365 - SendersBlockedTotalTimer(*): 1676028767 - TotalTime: 634110298427 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 18:1/7.18 MB 10:1/242.74 MB 14:1/243.04 MB 6:1/242.82 MB 12:1/243.06 MB 17:1/234.34 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.6521875408575607 - BytesRead: 69606015 - BytesReadDataNodeCache: 0 - BytesReadLocal: 69606015 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 69606015 - DecompressionTime: 221991495 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 220796224 - PerReadThreadRawHdfsThroughput: 1848133684 - RemoteScanRanges: 0 - RowsRead: 9624338 - RowsReturned: 340992 - RowsReturnedRate: 8389962 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 376703189745730 - MaterializeTupleTime(*): 351016647232643 - ScannerThreadsSysTime: 19994000 - ScannerThreadsUserTime: 860866000 - ScannerThreadsVoluntaryContextSwitches: 277 - TotalRawHdfsReadTime(*): 37662868 - TotalReadThroughput: 120 - TotalTime: 40642850 Instance 544466204e0abc59:f772f133ae0a68d2 (host=demeter-csmau10-12.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 13:1/235.68 MB 23:1/243.00 MB 1:1/242.93 MB 7:1/243.06 MB 20:1/242.92 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.6591624637560065 - InactiveTotalTime: 0 - PeakMemoryUsage: 4637211416 - PerHostPeakMemUsage: 4704320280 - PrepareTime: 380769486 - RowsProduced: 1 - TotalCpuTime: 932863962780108 - TotalNetworkReceiveTime: 630522118661 - TotalNetworkSendTime: 570723 - TotalStorageWaitTime: 143935764 - TotalTime: 562627547915197 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 339804 - CompileTime: 152204848 - InactiveTotalTime: 0 - LoadTime: 172913 - ModuleFileSize: 2309684 - OptimizationTime: 581656936 - PrepareTime: 367278279 - TotalTime: 1103813625 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 35189 - OverallThroughput: 276305 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9882 - ThriftTransmitTime(*): 454683 - TotalTime: 57907 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 425892852 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31768984 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 562627529738036 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646873351702 - BuildPartitionTime: 8191329866 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3496578243 - GetNewBlockTime: 10201368 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 348160 - ProbeRowsPartitioned: 0 - ProbeTime: 561979445369977 - RowsReturned: 35317281 - RowsReturnedRate: 62 - SpilledPartitions: 0 - TotalTime: 562626498615317 - UnpinTime: 1931 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2425525359 - DeserializeRowBatchTimer: 5772172803 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 628176683069 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99538 - SendersBlockedTimer: 382238168 - SendersBlockedTotalTimer(*): 3274739821 - TotalTime: 633176892219 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 13:1/235.68 MB 23:1/243.00 MB 1:1/242.93 MB 7:1/243.06 MB 20:1/242.92 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.6580143145139785 - BytesRead: 69091140 - BytesReadDataNodeCache: 0 - BytesReadLocal: 69091140 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 69091140 - DecompressionTime: 461034138 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 214997784 - PerReadThreadRawHdfsThroughput: 1509370757 - RemoteScanRanges: 0 - RowsRead: 9594641 - RowsReturned: 348160 - RowsReturnedRate: 8056611 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 6 - ScannerThreadsTotalWallClockTime: 370219472615116 - MaterializeTupleTime(*): 345616860931879 - ScannerThreadsSysTime: 36992000 - ScannerThreadsUserTime: 1800725000 - ScannerThreadsVoluntaryContextSwitches: 271 - TotalRawHdfsReadTime(*): 45774797 - TotalReadThroughput: 122 - TotalTime: 43214197 Instance 544466204e0abc59:f772f133ae0a68ec (host=demeter-csmau10-8.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 11:1/242.72 MB 13:1/242.57 MB 9:1/242.81 MB 10:1/242.79 MB 19:1/163.17 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0591582431711553 - InactiveTotalTime: 0 - PeakMemoryUsage: 4606913520 - PerHostPeakMemUsage: 4674022384 - PrepareTime: 371570183 - RowsProduced: 1 - TotalCpuTime: 1128678795824850 - TotalNetworkReceiveTime: 631730759799 - TotalNetworkSendTime: 524255 - TotalStorageWaitTime: 101953909 - TotalTime: 548432176165253 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 341739 - CompileTime: 81843309 - InactiveTotalTime: 0 - LoadTime: 159012 - ModuleFileSize: 2309684 - OptimizationTime: 535924186 - PrepareTime: 357723259 - TotalTime: 978310806 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 38828 - OverallThroughput: 343686 - PeakMemoryUsage: 16384 - SerializeBatchTime: 6110 - ThriftTransmitTime(*): 412067 - TotalTime: 46554 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 330548478 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31494280 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 548432175228243 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646190286043 - BuildPartitionTime: 7325943201 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2882582000 - GetNewBlockTime: 6578665 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 331059 - ProbeRowsPartitioned: 0 - ProbeTime: 547784982496652 - RowsReturned: 33562911 - RowsReturnedRate: 61 - SpilledPartitions: 0 - TotalTime: 548431307433439 - UnpinTime: 1100 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2288632709 - DeserializeRowBatchTimer: 4797236587 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630769065291 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99376 - SendersBlockedTimer: 242147366 - SendersBlockedTotalTimer(*): 1419936367 - TotalTime: 634212347702 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 11:1/242.72 MB 13:1/242.57 MB 9:1/242.81 MB 10:1/242.79 MB 19:1/163.17 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.057982023156334 - BytesRead: 64918400 - BytesReadDataNodeCache: 0 - BytesReadLocal: 64918400 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 64918400 - DecompressionTime: 324737502 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 184699888 - PerReadThreadRawHdfsThroughput: 1694367951 - RemoteScanRanges: 0 - RowsRead: 8967788 - RowsReturned: 331059 - RowsReturnedRate: 12400905 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 12 - ScannerThreadsTotalWallClockTime: 580231643150664 - MaterializeTupleTime(*): 555121211141925 - ScannerThreadsSysTime: 16995000 - ScannerThreadsUserTime: 1312798000 - ScannerThreadsVoluntaryContextSwitches: 244 - TotalRawHdfsReadTime(*): 38314228 - TotalReadThroughput: 118 - TotalTime: 26696356 Instance 544466204e0abc59:f772f133ae0a68fb (host=demeter-csmaz08-2.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 9:1/242.87 MB 17:1/2.68 MB 7:1/242.84 MB 15:1/242.85 MB 5:1/242.72 MB 13:1/242.71 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.6331969384213507 - InactiveTotalTime: 0 - PeakMemoryUsage: 4643707576 - PerHostPeakMemUsage: 4710816440 - PrepareTime: 387989688 - RowsProduced: 1 - TotalCpuTime: 893612604160393 - TotalNetworkReceiveTime: 631571556708 - TotalNetworkSendTime: 535251 - TotalStorageWaitTime: 142256872 - TotalTime: 547541990305173 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 333361 - CompileTime: 136193103 - InactiveTotalTime: 0 - LoadTime: 146088 - ModuleFileSize: 2309684 - OptimizationTime: 498051268 - PrepareTime: 373978647 - TotalTime: 1011058701 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 37731 - OverallThroughput: 353528 - PeakMemoryUsage: 16384 - SerializeBatchTime: 7220 - ThriftTransmitTime(*): 424047 - TotalTime: 45258 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 331892785 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31737816 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 547541989454867 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646034272562 - BuildPartitionTime: 7449660618 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2822641268 - GetNewBlockTime: 9091371 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 321536 - ProbeRowsPartitioned: 0 - ProbeTime: 546894933398352 - RowsReturned: 32616088 - RowsReturnedRate: 59 - SpilledPartitions: 0 - TotalTime: 547541112708747 - UnpinTime: 2595 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2295432798 - DeserializeRowBatchTimer: 4381173426 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630766787231 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99401 - SendersBlockedTimer: 278994568 - SendersBlockedTotalTimer(*): 2224025186 - TotalTime: 634047298720 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 9:1/242.87 MB 17:1/2.68 MB 7:1/242.84 MB 15:1/242.85 MB 5:1/242.72 MB 13:1/242.71 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.6320189467492588 - BytesRead: 69627555 - BytesReadDataNodeCache: 0 - BytesReadLocal: 69627555 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 69627555 - DecompressionTime: 250636247 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 221493944 - PerReadThreadRawHdfsThroughput: 1718001081 - RemoteScanRanges: 0 - RowsRead: 9650328 - RowsReturned: 321536 - RowsReturnedRate: 9171191 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 13 - ScannerThreadsTotalWallClockTime: 346055658116914 - MaterializeTupleTime(*): 320346234668599 - ScannerThreadsSysTime: 17995000 - ScannerThreadsUserTime: 974847000 - ScannerThreadsVoluntaryContextSwitches: 265 - TotalRawHdfsReadTime(*): 40528237 - TotalReadThroughput: 127 - TotalTime: 35059350 Instance 544466204e0abc59:f772f133ae0a68ca (host=demeter-csmau08-10.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 5:1/242.37 MB 12:1/235.36 MB 3:1/242.76 MB 7:1/242.51 MB 1:1/243.04 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.619430905229461 - InactiveTotalTime: 0 - PeakMemoryUsage: 4632444440 - PerHostPeakMemUsage: 4699586072 - PrepareTime: 373232122 - RowsProduced: 1 - TotalCpuTime: 861601322432681 - TotalNetworkReceiveTime: 628559036749 - TotalNetworkSendTime: 545624 - TotalStorageWaitTime: 121868297 - TotalTime: 532430205229154 CodeGen - AsyncTotalTime: 0 - CodegenTime: 376974 - CompileTime: 137505292 - InactiveTotalTime: 0 - LoadTime: 136236 - ModuleFileSize: 2309684 - OptimizationTime: 650451397 - PrepareTime: 359115543 - TotalTime: 1149754488 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 31923 - OverallThroughput: 316061 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9830 - ThriftTransmitTime(*): 501205 - TotalTime: 50623 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 414083788 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31869464 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 532430203900796 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646969160436 - BuildPartitionTime: 9673691959 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3338759220 - GetNewBlockTime: 36778057 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 318464 - ProbeRowsPartitioned: 0 - ProbeTime: 531782032404130 - RowsReturned: 32300323 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 532429180004184 - UnpinTime: 2264 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2793813871 - DeserializeRowBatchTimer: 6410087475 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 625872533334 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99784 - SendersBlockedTimer: 410942894 - SendersBlockedTotalTimer(*): 2691928620 - TotalTime: 631619418454 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 5:1/242.37 MB 12:1/235.36 MB 3:1/242.76 MB 7:1/242.51 MB 1:1/243.04 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.6182165900651981 - BytesRead: 66943009 - BytesReadDataNodeCache: 0 - BytesReadLocal: 66943009 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 66943009 - DecompressionTime: 527413493 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 210230808 - PerReadThreadRawHdfsThroughput: 1289510087 - RemoteScanRanges: 0 - RowsRead: 9531636 - RowsReturned: 318464 - RowsReturnedRate: 8354424 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 1 - ScannerThreadsTotalWallClockTime: 329152039982716 - MaterializeTupleTime(*): 303742409801869 - ScannerThreadsSysTime: 142975000 - ScannerThreadsUserTime: 1668743000 - ScannerThreadsVoluntaryContextSwitches: 240 - TotalRawHdfsReadTime(*): 51913521 - TotalReadThroughput: 125 - TotalTime: 38119201 Instance 544466204e0abc59:f772f133ae0a68df (host=demeter-csmau10-15.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 9:1/60.80 MB 0:1/242.83 MB 1:1/242.75 MB 8:1/62.21 MB 3:1/242.89 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.6009590302719827 - InactiveTotalTime: 0 - PeakMemoryUsage: 4569824256 - PerHostPeakMemUsage: 4636933120 - PrepareTime: 363725212 - RowsProduced: 1 - TotalCpuTime: 849583693432145 - TotalNetworkReceiveTime: 631787332632 - TotalNetworkSendTime: 514071 - TotalStorageWaitTime: 104335594 - TotalTime: 531063621283423 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 347734 - CompileTime: 136668585 - InactiveTotalTime: 0 - LoadTime: 158534 - ModuleFileSize: 2309684 - OptimizationTime: 506878050 - PrepareTime: 350297511 - TotalTime: 996591676 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 40161 - OverallThroughput: 340947 - PeakMemoryUsage: 16384 - SerializeBatchTime: 8914 - ThriftTransmitTime(*): 398390 - TotalTime: 46928 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 383817325 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31577888 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 531063620478247 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646511536968 - BuildPartitionTime: 7276427479 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3087326178 - GetNewBlockTime: 11471825 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 319488 - ProbeRowsPartitioned: 0 - ProbeTime: 530415974122339 - RowsReturned: 32363965 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 531062653842889 - UnpinTime: 2173 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2304613333 - DeserializeRowBatchTimer: 4632662183 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630842857832 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99365 - SendersBlockedTimer: 257756578 - SendersBlockedTotalTimer(*): 1269912517 - TotalTime: 634282000954 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 9:1/60.80 MB 0:1/242.83 MB 1:1/242.75 MB 8:1/62.21 MB 3:1/242.89 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5997425379318658 - BytesRead: 48866284 - BytesReadDataNodeCache: 0 - BytesReadLocal: 48866284 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 48866284 - DecompressionTime: 166671184 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 147610624 - PerReadThreadRawHdfsThroughput: 1621239393 - RemoteScanRanges: 0 - RowsRead: 6727325 - RowsReturned: 319488 - RowsReturnedRate: 8416418 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 318504807581751 - MaterializeTupleTime(*): 308335894351904 - ScannerThreadsSysTime: 9996000 - ScannerThreadsUserTime: 664895000 - ScannerThreadsVoluntaryContextSwitches: 240 - TotalRawHdfsReadTime(*): 30141313 - TotalReadThroughput: 92 - TotalTime: 37960090 Instance 544466204e0abc59:f772f133ae0a68fd (host=demeter-csmaz08-1.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 16:1/242.75 MB 20:1/242.64 MB 21:1/235.25 MB 9:1/242.69 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5792188030862244 - InactiveTotalTime: 0 - PeakMemoryUsage: 4584365888 - PerHostPeakMemUsage: 4651474752 - PrepareTime: 359369409 - RowsProduced: 1 - TotalCpuTime: 815298103585338 - TotalNetworkReceiveTime: 631580759944 - TotalNetworkSendTime: 493448 - TotalStorageWaitTime: 104902391 - TotalTime: 516668414219106 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 324658 - CompileTime: 135766231 - InactiveTotalTime: 0 - LoadTime: 206456 - ModuleFileSize: 2309684 - OptimizationTime: 501389605 - PrepareTime: 346064915 - TotalTime: 985930791 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 42300 - OverallThroughput: 330886 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9142 - ThriftTransmitTime(*): 378243 - TotalTime: 48355 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 369577194 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31243120 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 516668413384316 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646369033467 - BuildPartitionTime: 7505113243 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2855361896 - GetNewBlockTime: 5813177 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 312320 - ProbeRowsPartitioned: 0 - ProbeTime: 516020984395288 - RowsReturned: 31624270 - RowsReturnedRate: 61 - SpilledPartitions: 0 - TotalTime: 516667496179704 - UnpinTime: 2230 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2283384285 - DeserializeRowBatchTimer: 4949463175 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630523331496 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99400 - SendersBlockedTimer: 251945269 - SendersBlockedTotalTimer(*): 1487342937 - TotalTime: 634057780333 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 16:1/242.75 MB 20:1/242.64 MB 21:1/235.25 MB 9:1/242.69 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:12 ExecOption: Codegen enabled: 0 out of 4 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5779683575241626 - BytesRead: 54769443 - BytesReadDataNodeCache: 0 - BytesReadLocal: 54769443 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 54769443 - DecompressionTime: 194946845 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 3 - NumScannerThreadsStarted: 4 - PeakMemoryUsage: 162152256 - PerReadThreadRawHdfsThroughput: 1734894441 - RemoteScanRanges: 0 - RowsRead: 7578094 - RowsReturned: 312320 - RowsReturnedRate: 9018535 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 298614367218458 - MaterializeTupleTime(*): 282623306953830 - ScannerThreadsSysTime: 17996000 - ScannerThreadsUserTime: 830872000 - ScannerThreadsVoluntaryContextSwitches: 219 - TotalRawHdfsReadTime(*): 31569323 - TotalReadThroughput: 106 - TotalTime: 34630898 Instance 544466204e0abc59:f772f133ae0a6900 (host=demeter-csmaz10-13.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 13:1/242.81 MB 16:1/242.91 MB 10:1/242.81 MB 21:1/63.04 MB 12:1/242.53 MB 17:1/242.78 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.599566061937333 - InactiveTotalTime: 0 - PeakMemoryUsage: 4642443328 - PerHostPeakMemUsage: 4709552192 - PrepareTime: 392740846 - RowsProduced: 1 - TotalCpuTime: 802012063804954 - TotalNetworkReceiveTime: 631613958028 - TotalNetworkSendTime: 425240 - TotalStorageWaitTime: 119113413 - TotalTime: 501789009646122 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 334199 - CompileTime: 102280949 - InactiveTotalTime: 0 - LoadTime: 186850 - ModuleFileSize: 2309684 - OptimizationTime: 502574233 - PrepareTime: 378882392 - TotalTime: 986643215 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 51305 - OverallThroughput: 363537 - PeakMemoryUsage: 16384 - SerializeBatchTime: 5674 - ThriftTransmitTime(*): 311855 - TotalTime: 44012 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 295781673 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31461624 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 501789008790380 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646122331419 - BuildPartitionTime: 7429253149 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2832541465 - GetNewBlockTime: 7701881 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 297732 - ProbeRowsPartitioned: 0 - ProbeTime: 501141913790952 - RowsReturned: 30192866 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 501788163288870 - UnpinTime: 2008 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2291110897 - DeserializeRowBatchTimer: 4628800638 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630684318374 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99394 - SendersBlockedTimer: 254053838 - SendersBlockedTotalTimer(*): 1498328355 - TotalTime: 634092823594 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 13:1/242.81 MB 16:1/242.91 MB 10:1/242.81 MB 21:1/63.04 MB 12:1/242.53 MB 17:1/242.78 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5982795926116765 - BytesRead: 73042035 - BytesReadDataNodeCache: 0 - BytesReadLocal: 73042035 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 73042035 - DecompressionTime: 282068704 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 220229696 - PerReadThreadRawHdfsThroughput: 1878436911 - RemoteScanRanges: 0 - RowsRead: 10079204 - RowsReturned: 297732 - RowsReturnedRate: 10764896 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 300208058979283 - MaterializeTupleTime(*): 274888772645540 - ScannerThreadsSysTime: 9997000 - ScannerThreadsUserTime: 1119827000 - ScannerThreadsVoluntaryContextSwitches: 238 - TotalRawHdfsReadTime(*): 38884476 - TotalReadThroughput: 145 - TotalTime: 27657675 Instance 544466204e0abc59:f772f133ae0a68c2 (host=demeter-csmaz08-10.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 20:1/242.80 MB 23:1/242.68 MB 1:1/242.42 MB 0:1/242.83 MB 22:1/242.88 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5837577992459468 - InactiveTotalTime: 0 - PeakMemoryUsage: 4647559016 - PerHostPeakMemUsage: 4714667880 - PrepareTime: 363868385 - RowsProduced: 1 - TotalCpuTime: 784634164536601 - TotalNetworkReceiveTime: 631681279749 - TotalNetworkSendTime: 424442 - TotalStorageWaitTime: 125272775 - TotalTime: 495823048193599 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 331714 - CompileTime: 105933178 - InactiveTotalTime: 0 - LoadTime: 156352 - ModuleFileSize: 2309684 - OptimizationTime: 512310222 - PrepareTime: 350275060 - TotalTime: 971339704 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 45881 - OverallThroughput: 354641 - PeakMemoryUsage: 16384 - SerializeBatchTime: 8107 - ThriftTransmitTime(*): 348721 - TotalTime: 45116 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 312272130 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31663152 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 495823047382039 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646254726326 - BuildPartitionTime: 7361559955 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2934215064 - GetNewBlockTime: 6987232 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 286720 - ProbeRowsPartitioned: 0 - ProbeTime: 495175809592432 - RowsReturned: 29094078 - RowsReturnedRate: 58 - SpilledPartitions: 0 - TotalTime: 495822185772590 - UnpinTime: 810 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2310661087 - DeserializeRowBatchTimer: 4442707819 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630840238360 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99380 - SendersBlockedTimer: 227459005 - SendersBlockedTotalTimer(*): 1555092919 - TotalTime: 634181361829 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 20:1/242.80 MB 23:1/242.68 MB 1:1/242.42 MB 0:1/242.83 MB 22:1/242.88 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5824558218349069 - BytesRead: 68425182 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68425182 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68425182 - DecompressionTime: 225391649 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 225345384 - PerReadThreadRawHdfsThroughput: 1311018770 - RemoteScanRanges: 0 - RowsRead: 9596301 - RowsReturned: 286720 - RowsReturnedRate: 10698636 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 10 - ScannerThreadsTotalWallClockTime: 288796049044651 - MaterializeTupleTime(*): 262887966291525 - ScannerThreadsSysTime: 17995000 - ScannerThreadsUserTime: 878863000 - ScannerThreadsVoluntaryContextSwitches: 209 - TotalRawHdfsReadTime(*): 52192374 - TotalReadThroughput: 138 - TotalTime: 26799678 Instance 544466204e0abc59:f772f133ae0a68e2 (host=demeter-csmaz11-5.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 9:1/242.32 MB 8:1/242.90 MB 22:1/242.68 MB 20:1/242.73 MB 21:1/242.77 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5715618793500823 - InactiveTotalTime: 0 - PeakMemoryUsage: 4648414440 - PerHostPeakMemUsage: 4715523304 - PrepareTime: 376265021 - RowsProduced: 1 - TotalCpuTime: 754022212923460 - TotalNetworkReceiveTime: 631977021770 - TotalNetworkSendTime: 461915 - TotalStorageWaitTime: 116810592 - TotalTime: 480197549939834 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 346247 - CompileTime: 54562325 - InactiveTotalTime: 0 - LoadTime: 180309 - ModuleFileSize: 2309684 - OptimizationTime: 497178229 - PrepareTime: 362242774 - TotalTime: 916762213 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 45958 - OverallThroughput: 337453 - PeakMemoryUsage: 16384 - SerializeBatchTime: 6135 - ThriftTransmitTime(*): 348139 - TotalTime: 47414 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 307972574 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31615872 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 480197549085244 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646167916710 - BuildPartitionTime: 7207684852 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2785014417 - GetNewBlockTime: 7530062 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 283648 - ProbeRowsPartitioned: 0 - ProbeTime: 479550410906591 - RowsReturned: 28785258 - RowsReturnedRate: 59 - SpilledPartitions: 0 - TotalTime: 480196710648128 - UnpinTime: 869 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2257943394 - DeserializeRowBatchTimer: 4694975359 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630990113662 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99344 - SendersBlockedTimer: 178684036 - SendersBlockedTotalTimer(*): 1219654984 - TotalTime: 634411215901 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 9:1/242.32 MB 8:1/242.90 MB 22:1/242.68 MB 20:1/242.73 MB 21:1/242.77 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5702174902440269 - BytesRead: 69323506 - BytesReadDataNodeCache: 0 - BytesReadLocal: 69323506 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 69323506 - DecompressionTime: 208840321 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 3 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 226200808 - PerReadThreadRawHdfsThroughput: 2310854168 - RemoteScanRanges: 0 - RowsRead: 9642929 - RowsReturned: 283648 - RowsReturnedRate: 9906363 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 2 - ScannerThreadsTotalWallClockTime: 273810036185321 - MaterializeTupleTime(*): 248526295662633 - ScannerThreadsSysTime: 12995000 - ScannerThreadsUserTime: 797877000 - ScannerThreadsVoluntaryContextSwitches: 206 - TotalRawHdfsReadTime(*): 29999083 - TotalReadThroughput: 144 - TotalTime: 28632909 Instance 544466204e0abc59:f772f133ae0a68d7 (host=demeter-csmaz11-8.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 16:1/242.65 MB 12:1/242.45 MB 8:1/242.74 MB 13:1/243.00 MB 10:1/242.65 MB 7:1/242.75 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5968776826051267 - InactiveTotalTime: 0 - PeakMemoryUsage: 4687660584 - PerHostPeakMemUsage: 4754769448 - PrepareTime: 363770467 - RowsProduced: 1 - TotalCpuTime: 764085537257667 - TotalNetworkReceiveTime: 631747131675 - TotalNetworkSendTime: 425313 - TotalStorageWaitTime: 138066428 - TotalTime: 478882728908111 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 344611 - CompileTime: 136373009 - InactiveTotalTime: 0 - LoadTime: 166606 - ModuleFileSize: 2309684 - OptimizationTime: 508810514 - PrepareTime: 350097339 - TotalTime: 997994757 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 47666 - OverallThroughput: 372526 - PeakMemoryUsage: 16384 - SerializeBatchTime: 6958 - ThriftTransmitTime(*): 335669 - TotalTime: 42950 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 291564193 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31617832 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 478882728012231 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646046654231 - BuildPartitionTime: 7294627957 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2798007064 - GetNewBlockTime: 8044615 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 280576 - ProbeRowsPartitioned: 0 - ProbeTime: 478235683569674 - RowsReturned: 28439272 - RowsReturnedRate: 59 - SpilledPartitions: 0 - TotalTime: 478881892255107 - UnpinTime: 2001 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2298210169 - DeserializeRowBatchTimer: 4756593223 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630750633361 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99373 - SendersBlockedTimer: 258931078 - SendersBlockedTotalTimer(*): 1788949309 - TotalTime: 634228859925 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 16:1/242.65 MB 12:1/242.45 MB 8:1/242.74 MB 13:1/243.00 MB 10:1/242.65 MB 7:1/242.75 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5955296609284471 - BytesRead: 82130995 - BytesReadDataNodeCache: 0 - BytesReadLocal: 82130995 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 82130995 - DecompressionTime: 284719843 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 265446952 - PerReadThreadRawHdfsThroughput: 1494932281 - RemoteScanRanges: 0 - RowsRead: 11414907 - RowsReturned: 280576 - RowsReturnedRate: 7427440 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 9 - ScannerThreadsTotalWallClockTime: 285188000315874 - MaterializeTupleTime(*): 249208476110516 - ScannerThreadsSysTime: 16994000 - ScannerThreadsUserTime: 1151822000 - ScannerThreadsVoluntaryContextSwitches: 224 - TotalRawHdfsReadTime(*): 54939609 - TotalReadThroughput: 171 - TotalTime: 37775598 Instance 544466204e0abc59:f772f133ae0a68cb (host=demeter-csmau10-18.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 20:1/242.46 MB 6:1/234.10 MB 2:1/242.90 MB 19:1/242.95 MB 0:1/242.68 MB 1:1/243.02 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.6021603289420723 - InactiveTotalTime: 0 - PeakMemoryUsage: 4690112496 - PerHostPeakMemUsage: 4757221360 - PrepareTime: 363917677 - RowsProduced: 1 - TotalCpuTime: 765781994312277 - TotalNetworkReceiveTime: 631349306353 - TotalNetworkSendTime: 367774 - TotalStorageWaitTime: 142845060 - TotalTime: 478361525980492 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 364958 - CompileTime: 125526411 - InactiveTotalTime: 0 - LoadTime: 136757 - ModuleFileSize: 2309684 - OptimizationTime: 496153565 - PrepareTime: 350234012 - TotalTime: 974821063 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 55546 - OverallThroughput: 345207 - PeakMemoryUsage: 16384 - SerializeBatchTime: 6331 - ThriftTransmitTime(*): 288048 - TotalTime: 46349 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 319993699 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31574760 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 478361525112238 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646059682300 - BuildPartitionTime: 7668389764 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2785313883 - GetNewBlockTime: 5264125 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 284672 - ProbeRowsPartitioned: 0 - ProbeTime: 477714469199870 - RowsReturned: 28839692 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 478360682340389 - UnpinTime: 2044 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2325155067 - DeserializeRowBatchTimer: 4565564327 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630488485884 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99431 - SendersBlockedTimer: 233477545 - SendersBlockedTotalTimer(*): 1597415655 - TotalTime: 633859905632 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 20:1/242.46 MB 6:1/234.10 MB 2:1/242.90 MB 19:1/242.95 MB 0:1/242.68 MB 1:1/243.02 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.600811902324993 - BytesRead: 82517784 - BytesReadDataNodeCache: 0 - BytesReadLocal: 82517784 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 82517784 - DecompressionTime: 271474739 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 267898864 - PerReadThreadRawHdfsThroughput: 1529693233 - RemoteScanRanges: 0 - RowsRead: 11495375 - RowsReturned: 284672 - RowsReturnedRate: 7986544 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 12 - ScannerThreadsTotalWallClockTime: 287405278122541 - MaterializeTupleTime(*): 252077653914182 - ScannerThreadsSysTime: 35992000 - ScannerThreadsUserTime: 992845000 - ScannerThreadsVoluntaryContextSwitches: 224 - TotalRawHdfsReadTime(*): 53944008 - TotalReadThroughput: 172 - TotalTime: 35643951 Instance 544466204e0abc59:f772f133ae0a690f (host=demeter-csmau08-19.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 0:1/242.84 MB 1:1/133.25 MB 21:1/242.93 MB 7:1/235.29 MB 18:1/242.69 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5656304932434417 - InactiveTotalTime: 0 - PeakMemoryUsage: 4603827552 - PerHostPeakMemUsage: 4670936416 - PrepareTime: 361662615 - RowsProduced: 1 - TotalCpuTime: 745905421822361 - TotalNetworkReceiveTime: 631625913415 - TotalNetworkSendTime: 426535 - TotalStorageWaitTime: 97303682 - TotalTime: 476826136582225 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 325293 - CompileTime: 137229430 - InactiveTotalTime: 0 - LoadTime: 161320 - ModuleFileSize: 2309684 - OptimizationTime: 511661519 - PrepareTime: 348298941 - TotalTime: 999783688 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 50091 - OverallThroughput: 334196 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9617 - ThriftTransmitTime(*): 319416 - TotalTime: 47876 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 367598153 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31636520 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 476826135811076 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646158623388 - BuildPartitionTime: 7392061736 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2880168768 - GetNewBlockTime: 7066728 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 278528 - ProbeRowsPartitioned: 0 - ProbeTime: 476178881362279 - RowsReturned: 28267279 - RowsReturnedRate: 59 - SpilledPartitions: 0 - TotalTime: 476825200329718 - UnpinTime: 1998 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2309062419 - DeserializeRowBatchTimer: 4584691526 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630723313716 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99391 - SendersBlockedTimer: 270606557 - SendersBlockedTotalTimer(*): 2147372387 - TotalTime: 634112217732 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/242.84 MB 1:1/133.25 MB 21:1/242.93 MB 7:1/235.29 MB 18:1/242.69 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5642765894516919 - BytesRead: 62732649 - BytesReadDataNodeCache: 0 - BytesReadLocal: 62732649 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 62732649 - DecompressionTime: 285405864 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 181613920 - PerReadThreadRawHdfsThroughput: 2005897627 - RemoteScanRanges: 0 - RowsRead: 8710724 - RowsReturned: 278528 - RowsReturnedRate: 8067533 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 10 - ScannerThreadsTotalWallClockTime: 269064199550115 - MaterializeTupleTime(*): 242482695008626 - ScannerThreadsSysTime: 10995000 - ScannerThreadsUserTime: 1155821000 - ScannerThreadsVoluntaryContextSwitches: 201 - TotalRawHdfsReadTime(*): 31274103 - TotalReadThroughput: 131 - TotalTime: 34524554 Instance 544466204e0abc59:f772f133ae0a68dc (host=demeter-csmau08-8.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 0:1/242.43 MB 18:1/242.92 MB 2:1/242.92 MB 21:1/242.38 MB 20:1/242.98 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5671444071560143 - InactiveTotalTime: 0 - PeakMemoryUsage: 4601313592 - PerHostPeakMemUsage: 4668422456 - PrepareTime: 368981360 - RowsProduced: 1 - TotalCpuTime: 744815060508245 - TotalNetworkReceiveTime: 631813049698 - TotalNetworkSendTime: 478021 - TotalStorageWaitTime: 704392442 - TotalTime: 475671223184224 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 333036 - CompileTime: 136765686 - InactiveTotalTime: 0 - LoadTime: 140276 - ModuleFileSize: 2309684 - OptimizationTime: 503913367 - PrepareTime: 355584462 - TotalTime: 998993848 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 43278 - OverallThroughput: 306677 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9301 - ThriftTransmitTime(*): 369701 - TotalTime: 52172 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 384568669 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31536536 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 475671222412708 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646225733046 - BuildPartitionTime: 7266189647 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2931197417 - GetNewBlockTime: 7515498 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 279552 - ProbeRowsPartitioned: 0 - ProbeTime: 475023867886884 - RowsReturned: 28368603 - RowsReturnedRate: 59 - SpilledPartitions: 0 - TotalTime: 475670268492043 - UnpinTime: 2014 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2277690834 - DeserializeRowBatchTimer: 4662026466 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630922100155 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99367 - SendersBlockedTimer: 259810026 - SendersBlockedTotalTimer(*): 1279710152 - TotalTime: 634267700281 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/242.43 MB 18:1/242.92 MB 2:1/242.92 MB 21:1/242.38 MB 20:1/242.98 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5657861812431793 - BytesRead: 66626220 - BytesReadDataNodeCache: 0 - BytesReadLocal: 66626220 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 66626220 - DecompressionTime: 722723821 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 179099960 - PerReadThreadRawHdfsThroughput: 213097724 - RemoteScanRanges: 0 - RowsRead: 9518518 - RowsReturned: 279552 - RowsReturnedRate: 5163613 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 14 - ScannerThreadsTotalWallClockTime: 269129487315472 - MaterializeTupleTime(*): 243437280027858 - ScannerThreadsSysTime: 15995000 - ScannerThreadsUserTime: 3178514000 - ScannerThreadsVoluntaryContextSwitches: 220 - TotalRawHdfsReadTime(*): 312655709 - TotalReadThroughput: 140 - TotalTime: 54138831 Instance 544466204e0abc59:f772f133ae0a68b9 (host=demeter-csmaz08-12.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 13:1/242.82 MB 4:1/242.85 MB 3:1/242.87 MB 5:1/242.93 MB 7:1/65.03 MB 9:1/235.53 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5834068750718284 - InactiveTotalTime: 0 - PeakMemoryUsage: 4639484232 - PerHostPeakMemUsage: 4706593096 - PrepareTime: 377098384 - RowsProduced: 1 - TotalCpuTime: 750414157248959 - TotalNetworkReceiveTime: 631730665885 - TotalNetworkSendTime: 446169 - TotalStorageWaitTime: 135733863 - TotalTime: 474323439795939 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 352155 - CompileTime: 117982255 - InactiveTotalTime: 0 - LoadTime: 133306 - ModuleFileSize: 2309684 - OptimizationTime: 501163856 - PrepareTime: 363080770 - TotalTime: 985006363 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 50916 - OverallThroughput: 323421 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9163 - ThriftTransmitTime(*): 314241 - TotalTime: 49471 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 319770528 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31514648 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 474323438873186 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646059216535 - BuildPartitionTime: 7365585259 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2751588653 - GetNewBlockTime: 27296523 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 276480 - ProbeRowsPartitioned: 0 - ProbeTime: 473676383029625 - RowsReturned: 28058969 - RowsReturnedRate: 59 - SpilledPartitions: 0 - TotalTime: 474322568461204 - UnpinTime: 1608 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2290741623 - DeserializeRowBatchTimer: 4573627970 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630811645073 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99377 - SendersBlockedTimer: 239383387 - SendersBlockedTotalTimer(*): 1642596550 - TotalTime: 634202781575 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 13:1/242.82 MB 4:1/242.85 MB 3:1/242.87 MB 5:1/242.93 MB 7:1/65.03 MB 9:1/235.53 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5820458669455811 - BytesRead: 72975612 - BytesReadDataNodeCache: 0 - BytesReadLocal: 72975612 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 72975612 - DecompressionTime: 292326447 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 217270600 - PerReadThreadRawHdfsThroughput: 1615847380 - RemoteScanRanges: 0 - RowsRead: 10073288 - RowsReturned: 276480 - RowsReturnedRate: 9446985 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 276075904550559 - MaterializeTupleTime(*): 240126179642022 - ScannerThreadsSysTime: 23994000 - ScannerThreadsUserTime: 1145823000 - ScannerThreadsVoluntaryContextSwitches: 213 - TotalRawHdfsReadTime(*): 45162441 - TotalReadThroughput: 153 - TotalTime: 29266478 Instance 544466204e0abc59:f772f133ae0a68eb (host=demeter-csmaz11-20.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 23:1/242.35 MB 20:1/243.04 MB 16:1/242.84 MB 13:1/242.71 MB 18:1/242.77 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5727011762734908 - InactiveTotalTime: 0 - PeakMemoryUsage: 4615312368 - PerHostPeakMemUsage: 4682421232 - PrepareTime: 432290966 - RowsProduced: 1 - TotalCpuTime: 731734848318215 - TotalNetworkReceiveTime: 629373163229 - TotalNetworkSendTime: 444706 - TotalStorageWaitTime: 703198312 - TotalTime: 465672817810892 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 382412 - CompileTime: 149308180 - InactiveTotalTime: 0 - LoadTime: 6696113 - ModuleFileSize: 2309684 - OptimizationTime: 510866922 - PrepareTime: 352497402 - TotalTime: 1022513094 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 54803 - OverallThroughput: 318991 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9129 - ThriftTransmitTime(*): 291952 - TotalTime: 50158 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 348123857 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31889848 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 465672816519610 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646135181743 - BuildPartitionTime: 9093428940 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2926173502 - GetNewBlockTime: 15146761 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 284672 - ProbeRowsPartitioned: 0 - ProbeTime: 465025408338879 - RowsReturned: 28864881 - RowsReturnedRate: 61 - SpilledPartitions: 0 - TotalTime: 465671975922034 - UnpinTime: 2094 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2699381796 - DeserializeRowBatchTimer: 6245732390 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 626665970742 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99672 - SendersBlockedTimer: 348100715 - SendersBlockedTotalTimer(*): 1813108729 - TotalTime: 632326156167 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 23:1/242.35 MB 20:1/243.04 MB 16:1/242.84 MB 13:1/242.71 MB 18:1/242.77 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0.0001074% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 3.2217913374622378E-6 - AverageScannerThreadConcurrency: 0.5713148882199496 - BytesRead: 68638117 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68638117 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68638117 - DecompressionTime: 575877654 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 193098736 - PerReadThreadRawHdfsThroughput: 131768953 - RemoteScanRanges: 0 - RowsRead: 9591205 - RowsReturned: 284672 - RowsReturnedRate: 1055125 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 25 - ScannerThreadsTotalWallClockTime: 266045310329907 - MaterializeTupleTime(*): 242973693672844 - ScannerThreadsSysTime: 107982000 - ScannerThreadsUserTime: 2106677000 - ScannerThreadsVoluntaryContextSwitches: 313 - TotalRawHdfsReadTime(*): 520897487 - TotalReadThroughput: 147 - TotalTime: 269799064 Instance 544466204e0abc59:f772f133ae0a68cf (host=demeter-csmau10-19.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 13:1/235.35 MB 12:1/233.89 MB 2:1/242.75 MB 11:1/235.33 MB 5:1/242.84 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5557505172182942 - InactiveTotalTime: 0 - PeakMemoryUsage: 4639001968 - PerHostPeakMemUsage: 4706110832 - PrepareTime: 367389655 - RowsProduced: 1 - TotalCpuTime: 719915693545843 - TotalNetworkReceiveTime: 631730747262 - TotalNetworkSendTime: 297373 - TotalStorageWaitTime: 99049639 - TotalTime: 463149766186099 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 325648 - CompileTime: 137315413 - InactiveTotalTime: 0 - LoadTime: 170996 - ModuleFileSize: 2309684 - OptimizationTime: 507996560 - PrepareTime: 353928686 - TotalTime: 1001980623 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 63740 - OverallThroughput: 331915 - PeakMemoryUsage: 16384 - SerializeBatchTime: 6243 - ThriftTransmitTime(*): 251016 - TotalTime: 48205 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 335352140 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31306488 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 463149765344034 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 645949114870 - BuildPartitionTime: 7326485731 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2757562076 - GetNewBlockTime: 6866040 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 274915 - ProbeRowsPartitioned: 0 - ProbeTime: 462502795339661 - RowsReturned: 27896521 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 463148883884595 - UnpinTime: 2172 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2275243808 - DeserializeRowBatchTimer: 4789839154 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630724637348 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99378 - SendersBlockedTimer: 259087165 - SendersBlockedTotalTimer(*): 1021423110 - TotalTime: 634195484021 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 13:1/235.35 MB 12:1/233.89 MB 2:1/242.75 MB 11:1/235.33 MB 5:1/242.84 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.554356628722846 - BytesRead: 68059722 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68059722 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68059722 - DecompressionTime: 245450141 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 216788336 - PerReadThreadRawHdfsThroughput: 1973322976 - RemoteScanRanges: 0 - RowsRead: 9435206 - RowsReturned: 274915 - RowsReturnedRate: 8184690 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 8 - ScannerThreadsTotalWallClockTime: 256751161968505 - MaterializeTupleTime(*): 231596106695196 - ScannerThreadsSysTime: 13995000 - ScannerThreadsUserTime: 962852000 - ScannerThreadsVoluntaryContextSwitches: 199 - TotalRawHdfsReadTime(*): 34489905 - TotalReadThroughput: 146 - TotalTime: 33588929 Instance 544466204e0abc59:f772f133ae0a68bf (host=demeter-csmaz11-2.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 21:1/237.65 MB 13:1/242.53 MB 22:1/80.92 MB 17:1/242.58 MB 16:1/242.98 MB 15:1/242.74 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5634847260803635 - InactiveTotalTime: 0 - PeakMemoryUsage: 4639255856 - PerHostPeakMemUsage: 4706364720 - PrepareTime: 369352026 - RowsProduced: 1 - TotalCpuTime: 723332598933059 - TotalNetworkReceiveTime: 631677078519 - TotalNetworkSendTime: 418501 - TotalStorageWaitTime: 124388316 - TotalTime: 463043710124162 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 324583 - CompileTime: 137851643 - InactiveTotalTime: 0 - LoadTime: 145176 - ModuleFileSize: 2309684 - OptimizationTime: 501314828 - PrepareTime: 355778186 - TotalTime: 997633917 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 52526 - OverallThroughput: 340621 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9302 - ThriftTransmitTime(*): 304610 - TotalTime: 46973 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 344584636 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31556560 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 463043709280185 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646402341073 - BuildPartitionTime: 7410893440 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2859483555 - GetNewBlockTime: 8879526 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 278238 - ProbeRowsPartitioned: 0 - ProbeTime: 462396277658916 - RowsReturned: 28232526 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 463042808781774 - UnpinTime: 2154 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2283001551 - DeserializeRowBatchTimer: 4369428416 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630889057269 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99387 - SendersBlockedTimer: 252835243 - SendersBlockedTotalTimer(*): 1755041965 - TotalTime: 634141077733 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 21:1/237.65 MB 13:1/242.53 MB 22:1/80.92 MB 17:1/242.58 MB 16:1/242.98 MB 15:1/242.74 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.562089449070624 - BytesRead: 73294218 - BytesReadDataNodeCache: 0 - BytesReadLocal: 73294218 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 73294218 - DecompressionTime: 313282714 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 217042224 - PerReadThreadRawHdfsThroughput: 1646297455 - RemoteScanRanges: 0 - RowsRead: 10221584 - RowsReturned: 278238 - RowsReturnedRate: 7792982 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 2 - ScannerThreadsTotalWallClockTime: 260273647728429 - MaterializeTupleTime(*): 235244944466763 - ScannerThreadsSysTime: 37991000 - ScannerThreadsUserTime: 1142823000 - ScannerThreadsVoluntaryContextSwitches: 222 - TotalRawHdfsReadTime(*): 44520641 - TotalReadThroughput: 158 - TotalTime: 35703659 Instance 544466204e0abc59:f772f133ae0a68d5 (host=demeter-csmaz10-20.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 20:1/242.66 MB 12:1/242.72 MB 22:1/242.73 MB 8:1/242.49 MB 14:1/242.90 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5724160073336497 - InactiveTotalTime: 0 - PeakMemoryUsage: 4629057832 - PerHostPeakMemUsage: 4696166696 - PrepareTime: 367469738 - RowsProduced: 1 - TotalCpuTime: 726809935494116 - TotalNetworkReceiveTime: 631648473467 - TotalNetworkSendTime: 446673 - TotalStorageWaitTime: 134580021 - TotalTime: 462624562776918 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 323058 - CompileTime: 136486562 - InactiveTotalTime: 0 - LoadTime: 132561 - ModuleFileSize: 2309684 - OptimizationTime: 503644999 - PrepareTime: 354038420 - TotalTime: 996763058 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 48459 - OverallThroughput: 317756 - PeakMemoryUsage: 16384 - SerializeBatchTime: 8830 - ThriftTransmitTime(*): 330174 - TotalTime: 50353 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 371202817 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31488472 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 462624561993284 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646227079351 - BuildPartitionTime: 7422667540 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2920343495 - GetNewBlockTime: 5387022 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 278528 - ProbeRowsPartitioned: 0 - ProbeTime: 461977248812974 - RowsReturned: 28263590 - RowsReturnedRate: 61 - SpilledPartitions: 0 - TotalTime: 462623613169634 - UnpinTime: 1968 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2269381057 - DeserializeRowBatchTimer: 4935210090 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630584831318 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99391 - SendersBlockedTimer: 257472926 - SendersBlockedTotalTimer(*): 1519048082 - TotalTime: 634113177259 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 20:1/242.66 MB 12:1/242.72 MB 22:1/242.73 MB 8:1/242.49 MB 14:1/242.90 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5710194811301887 - BytesRead: 68082372 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68082372 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68082372 - DecompressionTime: 271275083 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 206844200 - PerReadThreadRawHdfsThroughput: 1595114377 - RemoteScanRanges: 0 - RowsRead: 9602741 - RowsReturned: 278528 - RowsReturnedRate: 7484188 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 264170287546081 - MaterializeTupleTime(*): 239080738866043 - ScannerThreadsSysTime: 16994000 - ScannerThreadsUserTime: 1071835000 - ScannerThreadsVoluntaryContextSwitches: 202 - TotalRawHdfsReadTime(*): 42681812 - TotalReadThroughput: 147 - TotalTime: 37215526 Instance 544466204e0abc59:f772f133ae0a6901 (host=demeter-csmau08-18.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 8:1/242.59 MB 7:1/242.40 MB 10:1/243.14 MB 9:1/243.00 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.547765449068264 - InactiveTotalTime: 0 - PeakMemoryUsage: 4603138416 - PerHostPeakMemUsage: 4670247280 - PrepareTime: 367304851 - RowsProduced: 1 - TotalCpuTime: 698786380991257 - TotalNetworkReceiveTime: 628888916109 - TotalNetworkSendTime: 479809 - TotalStorageWaitTime: 103773848 - TotalTime: 451882501151985 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 338752 - CompileTime: 133860267 - InactiveTotalTime: 0 - LoadTime: 171636 - ModuleFileSize: 2309684 - OptimizationTime: 3496140511 - PrepareTime: 353776767 - TotalTime: 3986514287 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 44088 - OverallThroughput: 310221 - PeakMemoryUsage: 16384 - SerializeBatchTime: 8659 - ThriftTransmitTime(*): 362909 - TotalTime: 51576 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 323609078 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31771416 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 451882498743525 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 643485093880 - BuildPartitionTime: 7246134244 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3056358431 - GetNewBlockTime: 7192333 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 270336 - ProbeRowsPartitioned: 0 - ProbeTime: 451238019420469 - RowsReturned: 27432302 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 451881646051649 - UnpinTime: 894 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2233859475 - DeserializeRowBatchTimer: 4745040040 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 627918948702 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99835 - SendersBlockedTimer: 3258395137 - SendersBlockedTotalTimer(*): 34201746562 - TotalTime: 631292303293 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 8:1/242.59 MB 7:1/242.40 MB 10:1/243.14 MB 9:1/243.00 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:12 ExecOption: Codegen enabled: 0 out of 4 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5463459550628946 - BytesRead: 55149443 - BytesReadDataNodeCache: 0 - BytesReadLocal: 55149443 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 55149443 - DecompressionTime: 328274151 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 4 - PeakMemoryUsage: 180924784 - PerReadThreadRawHdfsThroughput: 1327561340 - RemoteScanRanges: 0 - RowsRead: 7708574 - RowsReturned: 270336 - RowsReturnedRate: 7497617 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 2 - ScannerThreadsTotalWallClockTime: 246885753857648 - MaterializeTupleTime(*): 229909899729206 - ScannerThreadsSysTime: 27994000 - ScannerThreadsUserTime: 1283802000 - ScannerThreadsVoluntaryContextSwitches: 178 - TotalRawHdfsReadTime(*): 41541917 - TotalReadThroughput: 122 - TotalTime: 36056256 Instance 544466204e0abc59:f772f133ae0a68e8 (host=demeter-csmau08-3.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 1:1/242.86 MB 17:1/242.79 MB 20:1/242.96 MB 16:1/242.98 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.579697744753085 - InactiveTotalTime: 0 - PeakMemoryUsage: 4604318800 - PerHostPeakMemUsage: 4671427664 - PrepareTime: 361037548 - RowsProduced: 1 - TotalCpuTime: 690021712941136 - TotalNetworkReceiveTime: 631256570791 - TotalNetworkSendTime: 526811 - TotalStorageWaitTime: 82919090 - TotalTime: 437205573979955 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 327216 - CompileTime: 138415992 - InactiveTotalTime: 0 - LoadTime: 160704 - ModuleFileSize: 2309684 - OptimizationTime: 519860064 - PrepareTime: 347736509 - TotalTime: 1008675033 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 38609 - OverallThroughput: 328825 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9127 - ThriftTransmitTime(*): 414410 - TotalTime: 48658 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 352310843 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31689072 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 437205573153216 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646158061387 - BuildPartitionTime: 7665537794 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2888532281 - GetNewBlockTime: 7270110 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 286559 - ProbeRowsPartitioned: 0 - ProbeTime: 436558388671293 - RowsReturned: 29075201 - RowsReturnedRate: 66 - SpilledPartitions: 0 - TotalTime: 437204680466388 - UnpinTime: 2192 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2376212678 - DeserializeRowBatchTimer: 5076251296 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630104199688 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99434 - SendersBlockedTimer: 272580894 - SendersBlockedTotalTimer(*): 1634610129 - TotalTime: 633839821691 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 1:1/242.86 MB 17:1/242.79 MB 20:1/242.96 MB 16:1/242.98 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:12 ExecOption: Codegen enabled: 0 out of 4 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.578221192231757 - BytesRead: 55627559 - BytesReadDataNodeCache: 0 - BytesReadLocal: 55627559 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 55627559 - DecompressionTime: 444034296 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 4 - PeakMemoryUsage: 182105168 - PerReadThreadRawHdfsThroughput: 1553921515 - RemoteScanRanges: 0 - RowsRead: 7737039 - RowsReturned: 286559 - RowsReturnedRate: 7850245 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 252800661041622 - MaterializeTupleTime(*): 236597996331454 - ScannerThreadsSysTime: 92984000 - ScannerThreadsUserTime: 1471774000 - ScannerThreadsVoluntaryContextSwitches: 191 - TotalRawHdfsReadTime(*): 35798178 - TotalReadThroughput: 127 - TotalTime: 36503191 Instance 544466204e0abc59:f772f133ae0a68dd (host=demeter-csmau10-17.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 13:1/242.83 MB 10:1/243.00 MB 14:1/242.85 MB 15:1/242.84 MB 17:1/242.97 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5582555770554636 - InactiveTotalTime: 0 - PeakMemoryUsage: 4636438704 - PerHostPeakMemUsage: 4703547568 - PrepareTime: 365407966 - RowsProduced: 1 - TotalCpuTime: 669522932264365 - TotalNetworkReceiveTime: 631894569979 - TotalNetworkSendTime: 456466 - TotalStorageWaitTime: 114592502 - TotalTime: 430068349100335 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 328868 - CompileTime: 136851609 - InactiveTotalTime: 0 - LoadTime: 190396 - ModuleFileSize: 2309684 - OptimizationTime: 507549935 - PrepareTime: 351783892 - TotalTime: 999041013 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 45460 - OverallThroughput: 316881 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9439 - ThriftTransmitTime(*): 351955 - TotalTime: 50492 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 321232506 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31576680 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 430068348162186 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646688486962 - BuildPartitionTime: 7198171875 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3235105007 - GetNewBlockTime: 7053061 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 260096 - ProbeRowsPartitioned: 0 - ProbeTime: 429420674705776 - RowsReturned: 26365246 - RowsReturnedRate: 61 - SpilledPartitions: 0 - TotalTime: 430067495173359 - UnpinTime: 2135 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2242540331 - DeserializeRowBatchTimer: 4864508815 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630867565621 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99359 - SendersBlockedTimer: 259542547 - SendersBlockedTotalTimer(*): 1283650132 - TotalTime: 634319009827 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 13:1/242.83 MB 10:1/243.00 MB 14:1/242.85 MB 15:1/242.84 MB 17:1/242.97 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5567532996104425 - BytesRead: 69332116 - BytesReadDataNodeCache: 0 - BytesReadLocal: 69332116 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 69332116 - DecompressionTime: 436697244 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 214225072 - PerReadThreadRawHdfsThroughput: 1545287568 - RemoteScanRanges: 0 - RowsRead: 9599434 - RowsReturned: 260096 - RowsReturnedRate: 6724319 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 12 - ScannerThreadsTotalWallClockTime: 239439258590421 - MaterializeTupleTime(*): 214368580806958 - ScannerThreadsSysTime: 31993000 - ScannerThreadsUserTime: 1705739000 - ScannerThreadsVoluntaryContextSwitches: 185 - TotalRawHdfsReadTime(*): 44866805 - TotalReadThroughput: 161 - TotalTime: 38679897 Instance 544466204e0abc59:f772f133ae0a68c8 (host=demeter-csmau10-13.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 12:1/242.52 MB 18:1/240.18 MB 6:1/242.79 MB 7:1/242.90 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5097997348043357 - InactiveTotalTime: 0 - PeakMemoryUsage: 4587549648 - PerHostPeakMemUsage: 4654658512 - PrepareTime: 368342604 - RowsProduced: 1 - TotalCpuTime: 639410270703322 - TotalNetworkReceiveTime: 631839210523 - TotalNetworkSendTime: 391958 - TotalStorageWaitTime: 99024041 - TotalTime: 423923948964609 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 341215 - CompileTime: 83340721 - InactiveTotalTime: 0 - LoadTime: 178107 - ModuleFileSize: 2309684 - OptimizationTime: 502996855 - PrepareTime: 354176128 - TotalTime: 943309504 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 57967 - OverallThroughput: 330694 - PeakMemoryUsage: 16384 - SerializeBatchTime: 8260 - ThriftTransmitTime(*): 276018 - TotalTime: 48383 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 300407697 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31289816 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 423923948132882 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646271765040 - BuildPartitionTime: 7296814103 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2900780045 - GetNewBlockTime: 6741805 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 248832 - ProbeRowsPartitioned: 0 - ProbeTime: 423276714955593 - RowsReturned: 25229615 - RowsReturnedRate: 59 - SpilledPartitions: 0 - TotalTime: 423923118271938 - UnpinTime: 1039 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2269401575 - DeserializeRowBatchTimer: 5311859493 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630626157765 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99362 - SendersBlockedTimer: 200593990 - SendersBlockedTotalTimer(*): 972371345 - TotalTime: 634298508017 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 12:1/242.52 MB 18:1/240.18 MB 6:1/242.79 MB 7:1/242.90 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:12 ExecOption: Codegen enabled: 0 out of 4 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5082767667759068 - BytesRead: 54397515 - BytesReadDataNodeCache: 0 - BytesReadLocal: 54397515 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 54397515 - DecompressionTime: 216283915 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 3 - NumScannerThreadsStarted: 4 - PeakMemoryUsage: 165336016 - PerReadThreadRawHdfsThroughput: 1734039185 - RemoteScanRanges: 0 - RowsRead: 7616030 - RowsReturned: 248832 - RowsReturnedRate: 8833928 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 215471400882371 - MaterializeTupleTime(*): 198457829513408 - ScannerThreadsSysTime: 10997000 - ScannerThreadsUserTime: 864866000 - ScannerThreadsVoluntaryContextSwitches: 157 - TotalRawHdfsReadTime(*): 31370407 - TotalReadThroughput: 128 - TotalTime: 28167761 Instance 544466204e0abc59:f772f133ae0a68e3 (host=demeter-csmaz10-19.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 22:1/242.86 MB 19:1/242.94 MB 2:1/165.46 MB 20:1/242.79 MB 3:1/163.35 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5296745419219202 - InactiveTotalTime: 0 - PeakMemoryUsage: 4602756456 - PerHostPeakMemUsage: 4669865320 - PrepareTime: 366321142 - RowsProduced: 1 - TotalCpuTime: 637608215542300 - TotalNetworkReceiveTime: 631946155289 - TotalNetworkSendTime: 356017 - TotalStorageWaitTime: 99768356 - TotalTime: 417238761864457 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 339182 - CompileTime: 73841219 - InactiveTotalTime: 0 - LoadTime: 161786 - ModuleFileSize: 2309684 - OptimizationTime: 507702557 - PrepareTime: 352725471 - TotalTime: 937083189 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 50657 - OverallThroughput: 244992 - PeakMemoryUsage: 16384 - SerializeBatchTime: 7914 - ThriftTransmitTime(*): 315844 - TotalTime: 65308 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 276454895 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31569720 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 417238760949363 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646125052575 - BuildPartitionTime: 7228803940 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2798151180 - GetNewBlockTime: 7429203 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 245760 - ProbeRowsPartitioned: 0 - ProbeTime: 416591694102871 - RowsReturned: 24934768 - RowsReturnedRate: 59 - SpilledPartitions: 0 - TotalTime: 417237952474752 - UnpinTime: 818 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2228033001 - DeserializeRowBatchTimer: 4731004135 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630985846338 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99355 - SendersBlockedTimer: 200119134 - SendersBlockedTotalTimer(*): 1558893060 - TotalTime: 634345011483 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 22:1/242.86 MB 19:1/242.94 MB 2:1/165.46 MB 20:1/242.79 MB 3:1/163.35 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.528127206193463 - BytesRead: 60649794 - BytesReadDataNodeCache: 0 - BytesReadLocal: 60649794 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 60649794 - DecompressionTime: 247687997 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 180542824 - PerReadThreadRawHdfsThroughput: 1665447869 - RemoteScanRanges: 0 - RowsRead: 8403599 - RowsReturned: 245760 - RowsReturnedRate: 7503251 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 6 - ScannerThreadsTotalWallClockTime: 220354792233637 - MaterializeTupleTime(*): 194449931143510 - ScannerThreadsSysTime: 17994000 - ScannerThreadsUserTime: 957853000 - ScannerThreadsVoluntaryContextSwitches: 174 - TotalRawHdfsReadTime(*): 36416507 - TotalReadThroughput: 145 - TotalTime: 32753802 Instance 544466204e0abc59:f772f133ae0a6902 (host=demeter-csmau08-15.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 8:1/243.01 MB 3:1/242.66 MB 0:1/242.74 MB 22:1/242.71 MB 4:1/242.82 MB 7:1/234.52 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5636128196827452 - InactiveTotalTime: 0 - PeakMemoryUsage: 4697017928 - PerHostPeakMemUsage: 4764126792 - PrepareTime: 364190511 - RowsProduced: 1 - TotalCpuTime: 649144786234969 - TotalNetworkReceiveTime: 631569704004 - TotalNetworkSendTime: 396630 - TotalStorageWaitTime: 131707299 - TotalTime: 415560439233939 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 323375 - CompileTime: 136948206 - InactiveTotalTime: 0 - LoadTime: 173298 - ModuleFileSize: 2309684 - OptimizationTime: 509177060 - PrepareTime: 350589303 - TotalTime: 999535060 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 55977 - OverallThroughput: 323173 - PeakMemoryUsage: 16384 - SerializeBatchTime: 10057 - ThriftTransmitTime(*): 285827 - TotalTime: 49509 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 284159382 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31713504 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 415560438331899 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646040959201 - BuildPartitionTime: 7484336238 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2818313693 - GetNewBlockTime: 5139446 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 244784 - ProbeRowsPartitioned: 0 - ProbeTime: 414913434079876 - RowsReturned: 24839677 - RowsReturnedRate: 59 - SpilledPartitions: 0 - TotalTime: 415559625897659 - UnpinTime: 2182 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2300356045 - DeserializeRowBatchTimer: 4487265263 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630745472035 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99401 - SendersBlockedTimer: 266335992 - SendersBlockedTotalTimer(*): 1071719227 - TotalTime: 634049085686 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 8:1/243.01 MB 3:1/242.66 MB 0:1/242.74 MB 22:1/242.71 MB 4:1/242.82 MB 7:1/234.52 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5620593088060616 - BytesRead: 82748284 - BytesReadDataNodeCache: 0 - BytesReadLocal: 82748284 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 82748284 - DecompressionTime: 280993252 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 274804296 - PerReadThreadRawHdfsThroughput: 1729333487 - RemoteScanRanges: 0 - RowsRead: 11513508 - RowsReturned: 244784 - RowsReturnedRate: 7029672 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 6 - ScannerThreadsTotalWallClockTime: 233569360047248 - MaterializeTupleTime(*): 197857915316330 - ScannerThreadsSysTime: 41992000 - ScannerThreadsUserTime: 1008844000 - ScannerThreadsVoluntaryContextSwitches: 200 - TotalRawHdfsReadTime(*): 47849813 - TotalReadThroughput: 199 - TotalTime: 34821535 Instance 544466204e0abc59:f772f133ae0a68ce (host=demeter-csmaz08-14.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 11:1/242.66 MB 6:1/242.79 MB 10:1/242.88 MB 13:1/242.91 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.4920940929972124 - InactiveTotalTime: 0 - PeakMemoryUsage: 4589417608 - PerHostPeakMemUsage: 4656526472 - PrepareTime: 365270784 - RowsProduced: 1 - TotalCpuTime: 619055310440566 - TotalNetworkReceiveTime: 631739513999 - TotalNetworkSendTime: 330863 - TotalStorageWaitTime: 76510984 - TotalTime: 415314718182572 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 326142 - CompileTime: 135963635 - InactiveTotalTime: 0 - LoadTime: 133619 - ModuleFileSize: 2309684 - OptimizationTime: 506644925 - PrepareTime: 351675690 - TotalTime: 996930385 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 60740 - OverallThroughput: 238995 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9379 - ThriftTransmitTime(*): 263416 - TotalTime: 66947 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 241851687 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31373960 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 415314717315934 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646212510808 - BuildPartitionTime: 7379992334 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2886616302 - GetNewBlockTime: 5844588 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 239616 - ProbeRowsPartitioned: 0 - ProbeTime: 414667636394249 - RowsReturned: 24289635 - RowsReturnedRate: 58 - SpilledPartitions: 0 - TotalTime: 415313965421218 - UnpinTime: 2583 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2261535510 - DeserializeRowBatchTimer: 4528160895 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630858162663 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99380 - SendersBlockedTimer: 256212576 - SendersBlockedTotalTimer(*): 1772200324 - TotalTime: 634181296347 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 11:1/242.66 MB 6:1/242.79 MB 10:1/242.88 MB 13:1/242.91 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:12 ExecOption: Codegen enabled: 0 out of 4 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.49053829656825854 - BytesRead: 55424313 - BytesReadDataNodeCache: 0 - BytesReadLocal: 55424313 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 55424313 - DecompressionTime: 234831510 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 4 - PeakMemoryUsage: 167203976 - PerReadThreadRawHdfsThroughput: 2078273142 - RemoteScanRanges: 0 - RowsRead: 7662247 - RowsReturned: 239616 - RowsReturnedRate: 8646834 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 6 - ScannerThreadsTotalWallClockTime: 203725552449912 - MaterializeTupleTime(*): 186095391129358 - ScannerThreadsSysTime: 10996000 - ScannerThreadsUserTime: 920858000 - ScannerThreadsVoluntaryContextSwitches: 152 - TotalRawHdfsReadTime(*): 26668445 - TotalReadThroughput: 133 - TotalTime: 27711414 Instance 544466204e0abc59:f772f133ae0a68ea (host=demeter-csmau08-7.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 3:1/242.93 MB 8:1/242.83 MB 9:1/242.65 MB 1:1/242.83 MB 0:1/242.96 MB 22:1/242.84 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5457774533953714 - InactiveTotalTime: 0 - PeakMemoryUsage: 4672180128 - PerHostPeakMemUsage: 4739288992 - PrepareTime: 367180455 - RowsProduced: 1 - TotalCpuTime: 634870793166096 - TotalNetworkReceiveTime: 631991422333 - TotalNetworkSendTime: 427417 - TotalStorageWaitTime: 154891479 - TotalTime: 411125192555952 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 335539 - CompileTime: 137749478 - InactiveTotalTime: 0 - LoadTime: 177187 - ModuleFileSize: 2309684 - OptimizationTime: 517137951 - PrepareTime: 353367478 - TotalTime: 1011068736 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 49823 - OverallThroughput: 318293 - PeakMemoryUsage: 16384 - SerializeBatchTime: 6587 - ThriftTransmitTime(*): 321134 - TotalTime: 50268 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 253321289 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31677128 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 411125191705909 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 645937704360 - BuildPartitionTime: 7115992813 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2774343442 - GetNewBlockTime: 38242342 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 241918 - ProbeRowsPartitioned: 0 - ProbeTime: 410478350135046 - RowsReturned: 24535978 - RowsReturnedRate: 59 - SpilledPartitions: 0 - TotalTime: 411124415335837 - UnpinTime: 1933 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2220481487 - DeserializeRowBatchTimer: 4583917883 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 631107516839 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99347 - SendersBlockedTimer: 275514534 - SendersBlockedTotalTimer(*): 1386399976 - TotalTime: 634395680083 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 3:1/242.93 MB 8:1/242.83 MB 9:1/242.65 MB 1:1/242.83 MB 0:1/242.96 MB 22:1/242.84 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5442071446209765 - BytesRead: 83314372 - BytesReadDataNodeCache: 0 - BytesReadLocal: 83314372 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 83314372 - DecompressionTime: 292620446 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 249966496 - PerReadThreadRawHdfsThroughput: 1335350406 - RemoteScanRanges: 0 - RowsRead: 11579963 - RowsReturned: 241918 - RowsReturnedRate: 7149351 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 12 - ScannerThreadsTotalWallClockTime: 223731153290387 - MaterializeTupleTime(*): 188370745558740 - ScannerThreadsSysTime: 16994000 - ScannerThreadsUserTime: 1188817000 - ScannerThreadsVoluntaryContextSwitches: 186 - TotalRawHdfsReadTime(*): 62391393 - TotalReadThroughput: 202 - TotalTime: 33837755 Instance 544466204e0abc59:f772f133ae0a68da (host=demeter-csmaz11-12.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 0:1/242.84 MB 12:1/173.69 MB 4:1/242.59 MB 1:1/242.42 MB 15:1/243.00 MB 6:1/242.78 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.552243852775121 - InactiveTotalTime: 0 - PeakMemoryUsage: 4664699648 - PerHostPeakMemUsage: 4731808512 - PrepareTime: 382807251 - RowsProduced: 1 - TotalCpuTime: 628414089313639 - TotalNetworkReceiveTime: 631845301726 - TotalNetworkSendTime: 447556 - TotalStorageWaitTime: 134305978 - TotalTime: 405250142676525 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 328083 - CompileTime: 110890008 - InactiveTotalTime: 0 - LoadTime: 7069400 - ModuleFileSize: 2309684 - OptimizationTime: 495509768 - PrepareTime: 362207288 - TotalTime: 978221034 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 46979 - OverallThroughput: 284393 - PeakMemoryUsage: 16384 - SerializeBatchTime: 11414 - ThriftTransmitTime(*): 340574 - TotalTime: 56260 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 306678308 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31554008 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 405250141820108 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646184954186 - BuildPartitionTime: 7273648000 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2868870603 - GetNewBlockTime: 7238527 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 241664 - ProbeRowsPartitioned: 0 - ProbeTime: 404602975303110 - RowsReturned: 24503714 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 405249289767278 - UnpinTime: 1813 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2250705286 - DeserializeRowBatchTimer: 4750668263 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630830279291 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99366 - SendersBlockedTimer: 239114023 - SendersBlockedTotalTimer(*): 1402258470 - TotalTime: 634274449542 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/242.84 MB 12:1/173.69 MB 4:1/242.59 MB 1:1/242.42 MB 15:1/243.00 MB 6:1/242.78 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5506507898702122 - BytesRead: 78700305 - BytesReadDataNodeCache: 0 - BytesReadLocal: 78700305 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 78700305 - DecompressionTime: 261416242 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 242486016 - PerReadThreadRawHdfsThroughput: 1736399532 - RemoteScanRanges: 0 - RowsRead: 10959013 - RowsReturned: 241664 - RowsReturnedRate: 7925675 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 12 - ScannerThreadsTotalWallClockTime: 223149133753477 - MaterializeTupleTime(*): 188380944792565 - ScannerThreadsSysTime: 10996000 - ScannerThreadsUserTime: 1061837000 - ScannerThreadsVoluntaryContextSwitches: 183 - TotalRawHdfsReadTime(*): 45323846 - TotalReadThroughput: 194 - TotalTime: 30491280 Instance 544466204e0abc59:f772f133ae0a68ee (host=demeter-csmaz11-6.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 11:1/243.00 MB 17:1/242.83 MB 8:1/243.00 MB 14:1/242.66 MB 10:1/242.93 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.5375766435614469 - InactiveTotalTime: 0 - PeakMemoryUsage: 4644658000 - PerHostPeakMemUsage: 4711766864 - PrepareTime: 371083412 - RowsProduced: 1 - TotalCpuTime: 622154616984632 - TotalNetworkReceiveTime: 631208538683 - TotalNetworkSendTime: 318862 - TotalStorageWaitTime: 131438829 - TotalTime: 405043735086893 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 355541 - CompileTime: 138251837 - InactiveTotalTime: 0 - LoadTime: 179649 - ModuleFileSize: 2309684 - OptimizationTime: 583044189 - PrepareTime: 357449594 - TotalTime: 1081563311 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 63622 - OverallThroughput: 298401 - PeakMemoryUsage: 16384 - SerializeBatchTime: 8636 - ThriftTransmitTime(*): 251482 - TotalTime: 53619 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 308033714 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31656104 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 405043734193626 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 645925609587 - BuildPartitionTime: 7618117642 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2786290974 - GetNewBlockTime: 9496852 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 247918 - ProbeRowsPartitioned: 0 - ProbeTime: 404396837402900 - RowsReturned: 25129732 - RowsReturnedRate: 62 - SpilledPartitions: 0 - TotalTime: 405042894846165 - UnpinTime: 2228 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2340585134 - DeserializeRowBatchTimer: 5922754375 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 628616746993 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99444 - SendersBlockedTimer: 347309223 - SendersBlockedTotalTimer(*): 2526392364 - TotalTime: 633775766651 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 11:1/243.00 MB 17:1/242.83 MB 8:1/243.00 MB 14:1/242.66 MB 10:1/242.93 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5359827438894897 - BytesRead: 69286768 - BytesReadDataNodeCache: 0 - BytesReadLocal: 69286768 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 69286768 - DecompressionTime: 230580694 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 222444368 - PerReadThreadRawHdfsThroughput: 1367572492 - RemoteScanRanges: 0 - RowsRead: 9632346 - RowsReturned: 247918 - RowsReturnedRate: 6771787 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 10 - ScannerThreadsTotalWallClockTime: 217095573990893 - MaterializeTupleTime(*): 192398308516964 - ScannerThreadsSysTime: 15994000 - ScannerThreadsUserTime: 897862000 - ScannerThreadsVoluntaryContextSwitches: 171 - TotalRawHdfsReadTime(*): 50664055 - TotalReadThroughput: 171 - TotalTime: 36610421 Instance 544466204e0abc59:f772f133ae0a68c5 (host=demeter-csmau10-10.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 0:1/242.80 MB 5:1/242.91 MB 2:1/242.84 MB 14:1/961.42 KB 16:1/235.63 MB 1:1/242.88 MB 10:1/242.90 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.558967401738062 - InactiveTotalTime: 0 - PeakMemoryUsage: 4698941488 - PerHostPeakMemUsage: 4766050352 - PrepareTime: 241946108 - RowsProduced: 1 - TotalCpuTime: 608514358186485 - TotalNetworkReceiveTime: 632440297104 - TotalNetworkSendTime: 353687 - TotalStorageWaitTime: 123343326 - TotalTime: 390737654362189 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 341673 - CompileTime: 136520242 - InactiveTotalTime: 0 - LoadTime: 92882 - ModuleFileSize: 2309684 - OptimizationTime: 500896678 - PrepareTime: 228340939 - TotalTime: 868391389 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 65256 - OverallThroughput: 332730 - PeakMemoryUsage: 16384 - SerializeBatchTime: 8105 - ThriftTransmitTime(*): 245185 - TotalTime: 48087 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 275933016 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31666432 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 390737653725107 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646599470783 - BuildPartitionTime: 6968166247 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3296002076 - GetNewBlockTime: 5159651 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 242688 - ProbeRowsPartitioned: 0 - ProbeTime: 390090280198387 - RowsReturned: 24602574 - RowsReturnedRate: 62 - SpilledPartitions: 0 - TotalTime: 390736999278298 - UnpinTime: 2080 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2087009438 - DeserializeRowBatchTimer: 2367646563 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 632062650961 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99299 - SendersBlockedTimer: 124889645 - SendersBlockedTotalTimer(*): 715500133 - TotalTime: 634699999463 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/242.80 MB 5:1/242.91 MB 2:1/242.84 MB 14:1/961.42 KB 16:1/235.63 MB 1:1/242.88 MB 10:1/242.90 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:21 ExecOption: Codegen enabled: 0 out of 7 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.5573139577744558 - BytesRead: 83143911 - BytesReadDataNodeCache: 0 - BytesReadLocal: 83143911 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 83143911 - DecompressionTime: 255801915 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 7 - PeakMemoryUsage: 276596832 - PerReadThreadRawHdfsThroughput: 2341099793 - RemoteScanRanges: 0 - RowsRead: 11544828 - RowsReturned: 242688 - RowsReturnedRate: 8708905 - ScanRangesComplete: 7 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 217762029758216 - MaterializeTupleTime(*): 183097979241810 - ScannerThreadsSysTime: 26994000 - ScannerThreadsUserTime: 944853000 - ScannerThreadsVoluntaryContextSwitches: 197 - TotalRawHdfsReadTime(*): 35514894 - TotalReadThroughput: 212 - TotalTime: 27866648 Instance 544466204e0abc59:f772f133ae0a68f5 (host=demeter-csmau08-11.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 5:1/235.37 MB 9:1/165.18 MB 4:1/234.83 MB 7:1/235.49 MB 6:1/235.45 MB 2:1/242.83 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.4766289545508147 - InactiveTotalTime: 0 - PeakMemoryUsage: 4653490392 - PerHostPeakMemUsage: 4720599256 - PrepareTime: 360584979 - RowsProduced: 1 - TotalCpuTime: 519132660568444 - TotalNetworkReceiveTime: 631354899323 - TotalNetworkSendTime: 452103 - TotalStorageWaitTime: 131553870 - TotalTime: 351993591441620 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 335333 - CompileTime: 137726587 - InactiveTotalTime: 0 - LoadTime: 180957 - ModuleFileSize: 2309684 - OptimizationTime: 517541150 - PrepareTime: 347155788 - TotalTime: 1005100681 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 46787 - OverallThroughput: 330756 - PeakMemoryUsage: 16384 - SerializeBatchTime: 8410 - ThriftTransmitTime(*): 341972 - TotalTime: 48374 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 251236864 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31715256 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 351993590603149 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646646454000 - BuildPartitionTime: 7598514354 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3283016750 - GetNewBlockTime: 7673153 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 217767 - ProbeRowsPartitioned: 0 - ProbeTime: 351346065214328 - RowsReturned: 22059588 - RowsReturnedRate: 62 - SpilledPartitions: 0 - TotalTime: 351992840518139 - UnpinTime: 2382 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2376399551 - DeserializeRowBatchTimer: 4709458430 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630449681643 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99421 - SendersBlockedTimer: 272733008 - SendersBlockedTotalTimer(*): 2161615008 - TotalTime: 633919711227 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 5:1/235.37 MB 9:1/165.18 MB 4:1/234.83 MB 7:1/235.49 MB 6:1/235.45 MB 2:1/242.83 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.4747932410679147 - BytesRead: 77445956 - BytesReadDataNodeCache: 0 - BytesReadLocal: 77445956 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 77445956 - DecompressionTime: 437472441 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 231276760 - PerReadThreadRawHdfsThroughput: 1758823201 - RemoteScanRanges: 0 - RowsRead: 10706521 - RowsReturned: 217767 - RowsReturnedRate: 6099459 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 7 - ScannerThreadsTotalWallClockTime: 167123252897379 - MaterializeTupleTime(*): 142275522115091 - ScannerThreadsSysTime: 24994000 - ScannerThreadsUserTime: 1744730000 - ScannerThreadsVoluntaryContextSwitches: 168 - TotalRawHdfsReadTime(*): 44032826 - TotalReadThroughput: 220 - TotalTime: 35702671 Instance 544466204e0abc59:f772f133ae0a68c7 (host=demeter-csmaz11-7.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 2:1/163.44 MB 13:1/242.50 MB 22:1/242.83 MB 12:1/242.26 MB 23:1/242.89 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.2604159388460747 - InactiveTotalTime: 0 - PeakMemoryUsage: 4637751512 - PerHostPeakMemUsage: 4704860376 - PrepareTime: 371086425 - RowsProduced: 1 - TotalCpuTime: 333166745518900 - TotalNetworkReceiveTime: 631842548845 - TotalNetworkSendTime: 565002 - TotalStorageWaitTime: 97527571 - TotalTime: 264833288714640 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 361488 - CompileTime: 137962517 - InactiveTotalTime: 0 - LoadTime: 148825 - ModuleFileSize: 2309684 - OptimizationTime: 506785730 - PrepareTime: 357118694 - TotalTime: 1004682285 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 38121 - OverallThroughput: 248586 - PeakMemoryUsage: 16384 - SerializeBatchTime: 16677 - ThriftTransmitTime(*): 419707 - TotalTime: 64364 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 176836250 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31507064 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 264833287808815 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646251188293 - BuildPartitionTime: 7235841089 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2933780801 - GetNewBlockTime: 6717942 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 156075 - ProbeRowsPartitioned: 0 - ProbeTime: 264186315859093 - RowsReturned: 15840021 - RowsReturnedRate: 59 - SpilledPartitions: 0 - TotalTime: 264832656323050 - UnpinTime: 1954 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2283923111 - DeserializeRowBatchTimer: 4599891989 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630925800075 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99362 - SendersBlockedTimer: 261841909 - SendersBlockedTotalTimer(*): 1554384044 - TotalTime: 634301540224 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 2:1/163.44 MB 13:1/242.50 MB 22:1/242.83 MB 12:1/242.26 MB 23:1/242.89 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.257977101182709 - BytesRead: 63584385 - BytesReadDataNodeCache: 0 - BytesReadLocal: 63584385 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 63584385 - DecompressionTime: 208104028 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 215537880 - PerReadThreadRawHdfsThroughput: 1771089884 - RemoteScanRanges: 0 - RowsRead: 8950097 - RowsReturned: 156075 - RowsReturnedRate: 6642580 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 12 - ScannerThreadsTotalWallClockTime: 68318499965155 - MaterializeTupleTime(*): 42683527598508 - ScannerThreadsSysTime: 9996000 - ScannerThreadsUserTime: 857867000 - ScannerThreadsVoluntaryContextSwitches: 87 - TotalRawHdfsReadTime(*): 35901275 - TotalReadThroughput: 240 - TotalTime: 23496140 Instance 544466204e0abc59:f772f133ae0a68fe (host=demeter-csmaz08-4.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 10:1/164.39 MB 3:1/242.75 MB 1:1/242.83 MB 7:1/234.35 MB 6:1/242.39 MB 15:1/242.65 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.2633995624632666 - InactiveTotalTime: 0 - PeakMemoryUsage: 4671223856 - PerHostPeakMemUsage: 4738332720 - PrepareTime: 365007723 - RowsProduced: 1 - TotalCpuTime: 308977551279065 - TotalNetworkReceiveTime: 631692616907 - TotalNetworkSendTime: 546038 - TotalStorageWaitTime: 140087310 - TotalTime: 245061631613748 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 327450 - CompileTime: 54717179 - InactiveTotalTime: 0 - LoadTime: 140182 - ModuleFileSize: 2309684 - OptimizationTime: 498694721 - PrepareTime: 351259503 - TotalTime: 907413085 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 37542 - OverallThroughput: 311356 - PeakMemoryUsage: 16384 - SerializeBatchTime: 7761 - ThriftTransmitTime(*): 426179 - TotalTime: 51388 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 155710491 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 31637992 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 245061630727408 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646343059812 - BuildPartitionTime: 7241633593 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3095785522 - GetNewBlockTime: 8480438 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 145554 - ProbeRowsPartitioned: 0 - ProbeTime: 244414587271421 - RowsReturned: 14764996 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 245061028636510 - UnpinTime: 2088 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2275329970 - DeserializeRowBatchTimer: 4542938660 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630804215652 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99384 - SendersBlockedTimer: 174789675 - SendersBlockedTotalTimer(*): 1017321583 - TotalTime: 634160588296 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 10:1/164.39 MB 3:1/242.75 MB 1:1/242.83 MB 7:1/234.35 MB 6:1/242.39 MB 15:1/242.65 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.2607614052631042 - BytesRead: 77357911 - BytesReadDataNodeCache: 0 - BytesReadLocal: 77357911 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 77357911 - DecompressionTime: 284114394 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 249010224 - PerReadThreadRawHdfsThroughput: 1765652771 - RemoteScanRanges: 0 - RowsRead: 10833590 - RowsReturned: 145554 - RowsReturnedRate: 6197171 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 63900854589221 - MaterializeTupleTime(*): 27741546208394 - ScannerThreadsSysTime: 22993000 - ScannerThreadsUserTime: 1094832000 - ScannerThreadsVoluntaryContextSwitches: 94 - TotalRawHdfsReadTime(*): 43812641 - TotalReadThroughput: 315 - TotalTime: 23487168 Instance 544466204e0abc59:f772f133ae0a6909 (host=demeter-csmaz11-11.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 11:1/243.03 MB 15:1/242.79 MB 6:1/242.88 MB 13:1/63.21 MB 7:1/242.95 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.0041774714011342 - InactiveTotalTime: 0 - PeakMemoryUsage: 4593772672 - PerHostPeakMemUsage: 4660881536 - PrepareTime: 369759531 - RowsProduced: 1 - TotalCpuTime: 154687101190206 - TotalNetworkReceiveTime: 631817489611 - TotalNetworkSendTime: 514862 - TotalStorageWaitTime: 132994151 - TotalTime: 154671083887805 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 342381 - CompileTime: 137098524 - InactiveTotalTime: 0 - LoadTime: 191896 - ModuleFileSize: 2309684 - OptimizationTime: 508767707 - PrepareTime: 356150742 - TotalTime: 1004705142 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 39521 - OverallThroughput: 323369 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9153 - ThriftTransmitTime(*): 404843 - TotalTime: 49479 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 115941320 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 33265440 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 154671083098520 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646031513885 - BuildPartitionTime: 7258854949 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2812822456 - GetNewBlockTime: 6040523 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 93570 - ProbeRowsPartitioned: 0 - ProbeTime: 154024439934991 - RowsReturned: 9484177 - RowsReturnedRate: 61 - SpilledPartitions: 0 - TotalTime: 154670544335557 - UnpinTime: 2074 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2275093306 - DeserializeRowBatchTimer: 4466082488 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630988045597 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99368 - SendersBlockedTimer: 273305427 - SendersBlockedTotalTimer(*): 1634225207 - TotalTime: 634263133121 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 11:1/243.03 MB 15:1/242.79 MB 6:1/242.88 MB 13:1/63.21 MB 7:1/242.95 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 59245093 - BytesReadDataNodeCache: 0 - BytesReadLocal: 59245093 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 59245093 - DecompressionTime: 286505406 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 171559040 - PerReadThreadRawHdfsThroughput: 1226939023 - RemoteScanRanges: 0 - RowsRead: 8201965 - RowsReturned: 93570 - RowsReturnedRate: 3351067 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 1289297218 - MaterializeTupleTime(*): 834297092 - ScannerThreadsSysTime: 18994000 - ScannerThreadsUserTime: 1129826000 - ScannerThreadsVoluntaryContextSwitches: 56 - TotalRawHdfsReadTime(*): 48286909 - TotalReadThroughput: 383 - TotalTime: 27922449 Instance 544466204e0abc59:f772f133ae0a6903 (host=demeter-csmau08-12.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 3:1/242.83 MB 7:1/242.75 MB 4:1/242.87 MB 11:1/234.66 MB 2:1/242.76 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.0046076073480146 - InactiveTotalTime: 0 - PeakMemoryUsage: 4640846928 - PerHostPeakMemUsage: 4707955792 - PrepareTime: 366944952 - RowsProduced: 1 - TotalCpuTime: 140797337495537 - TotalNetworkReceiveTime: 631680631216 - TotalNetworkSendTime: 509710 - TotalStorageWaitTime: 99788901 - TotalTime: 140781087243817 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 330727 - CompileTime: 137735254 - InactiveTotalTime: 0 - LoadTime: 131852 - ModuleFileSize: 2309684 - OptimizationTime: 509145385 - PrepareTime: 353469689 - TotalTime: 1003021978 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 40013 - OverallThroughput: 324688 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9679 - ThriftTransmitTime(*): 399864 - TotalTime: 49278 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 117256497 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 34469944 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 140781086465130 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646260262476 - BuildPartitionTime: 7359895837 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2964412343 - GetNewBlockTime: 27847025 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 84404 - ProbeRowsPartitioned: 0 - ProbeTime: 140134114497390 - RowsReturned: 8555720 - RowsReturnedRate: 60 - SpilledPartitions: 0 - TotalTime: 140780547035230 - UnpinTime: 2284 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2286505803 - DeserializeRowBatchTimer: 4616878594 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630760764610 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99385 - SendersBlockedTimer: 270418920 - SendersBlockedTotalTimer(*): 1343328377 - TotalTime: 634153398461 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 3:1/242.83 MB 7:1/242.75 MB 4:1/242.87 MB 11:1/234.66 MB 2:1/242.76 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.7762683444113268E-5 - BytesRead: 68816655 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68816655 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68816655 - DecompressionTime: 250315271 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 200334520 - PerReadThreadRawHdfsThroughput: 1921515779 - RemoteScanRanges: 0 - RowsRead: 9555948 - RowsReturned: 84404 - RowsReturnedRate: 637812 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 1122491078 - MaterializeTupleTime(*): 746163701 - ScannerThreadsSysTime: 7996000 - ScannerThreadsUserTime: 1007844000 - ScannerThreadsVoluntaryContextSwitches: 56 - TotalRawHdfsReadTime(*): 35813734 - TotalReadThroughput: 488 - TotalTime: 132333622 Instance 544466204e0abc59:f772f133ae0a68e7 (host=demeter-csmaz10-14.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 17:1/242.64 MB 7:1/242.48 MB 14:1/242.75 MB 11:1/242.78 MB 15:1/243.01 MB 18:1/63.19 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.0059876096152338 - InactiveTotalTime: 0 - PeakMemoryUsage: 4652528704 - PerHostPeakMemUsage: 4719637568 - PrepareTime: 366184674 - RowsProduced: 1 - TotalCpuTime: 108425920570550 - TotalNetworkReceiveTime: 632030409456 - TotalNetworkSendTime: 481188 - TotalStorageWaitTime: 125193283 - TotalTime: 108410133676821 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 340942 - CompileTime: 66119515 - InactiveTotalTime: 0 - LoadTime: 153362 - ModuleFileSize: 2309684 - OptimizationTime: 503492886 - PrepareTime: 352790952 - TotalTime: 925098866 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 42328 - OverallThroughput: 314440 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9603 - ThriftTransmitTime(*): 377995 - TotalTime: 50884 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 81491068 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 33711240 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 108410132811779 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646211026362 - BuildPartitionTime: 7137901265 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2828235625 - GetNewBlockTime: 5716399 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 65396 - ProbeRowsPartitioned: 0 - ProbeTime: 107763369223005 - RowsReturned: 6636082 - RowsReturnedRate: 61 - SpilledPartitions: 0 - TotalTime: 108409649546069 - UnpinTime: 1093 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2244769708 - DeserializeRowBatchTimer: 4866128728 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 631029652176 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99336 - SendersBlockedTimer: 186573393 - SendersBlockedTotalTimer(*): 904867258 - TotalTime: 634464735136 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 17:1/242.64 MB 7:1/242.48 MB 14:1/242.75 MB 11:1/242.78 MB 15:1/243.01 MB 18:1/63.19 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 2.7677957735758538E-5 - BytesRead: 71897326 - BytesReadDataNodeCache: 0 - BytesReadLocal: 71897326 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 71897326 - DecompressionTime: 264193337 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 230315072 - PerReadThreadRawHdfsThroughput: 1575895598 - RemoteScanRanges: 0 - RowsRead: 10098796 - RowsReturned: 65396 - RowsReturnedRate: 2515803 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 1160851152 - MaterializeTupleTime(*): 741807553 - ScannerThreadsSysTime: 20993000 - ScannerThreadsUserTime: 1003845000 - ScannerThreadsVoluntaryContextSwitches: 75 - TotalRawHdfsReadTime(*): 45623153 - TotalReadThroughput: 663 - TotalTime: 25994084 Instance 544466204e0abc59:f772f133ae0a68b8 (host=demeter-csmau10-4.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 16:1/242.76 MB 11:1/242.94 MB 14:1/242.83 MB 19:1/62.18 MB 12:1/242.87 MB 20:1/62.23 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.997686969930609 - InactiveTotalTime: 0 - PeakMemoryUsage: 4595824912 - PerHostPeakMemUsage: 4662933776 - PrepareTime: 405026428 - RowsProduced: 1 - TotalCpuTime: 664791409500 - TotalNetworkReceiveTime: 630950297155 - TotalNetworkSendTime: 477415 - TotalStorageWaitTime: 243328234 - TotalTime: 647494692404 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 365473 - CompileTime: 136529995 - InactiveTotalTime: 0 - LoadTime: 173887 - ModuleFileSize: 2309684 - OptimizationTime: 500238502 - PrepareTime: 391479390 - TotalTime: 1030962169 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 45631 - OverallThroughput: 284621 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12004 - ThriftTransmitTime(*): 350632 - TotalTime: 56215 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 171892 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647493880529 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646683421780 - BuildPartitionTime: 8090642858 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3101966699 - GetNewBlockTime: 5094261 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1953 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647100225144 - UnpinTime: 2031 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2285596590 - DeserializeRowBatchTimer: 4497639792 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630116836916 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99500 - SendersBlockedTimer: 284085992 - SendersBlockedTotalTimer(*): 1985900270 - TotalTime: 633419768362 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 16:1/242.76 MB 11:1/242.94 MB 14:1/242.83 MB 19:1/62.18 MB 12:1/242.87 MB 20:1/62.23 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 7.727975270479134E-4 - BytesRead: 63165880 - BytesReadDataNodeCache: 0 - BytesReadLocal: 63165880 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 63165880 - DecompressionTime: 247349115 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 173811984 - PerReadThreadRawHdfsThroughput: 1756245574 - RemoteScanRanges: 0 - RowsRead: 8727019 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 1169225599 - MaterializeTupleTime(*): 650880010 - ScannerThreadsSysTime: 22994000 - ScannerThreadsUserTime: 894861000 - ScannerThreadsVoluntaryContextSwitches: 67 - TotalRawHdfsReadTime(*): 35966428 - TotalReadThroughput: 97478 - TotalTime: 402754564 Instance 544466204e0abc59:f772f133ae0a68bc (host=demeter-csmaz11-10.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 20:1/242.57 MB 21:1/242.79 MB 0:1/242.44 MB 16:1/242.70 MB 1:1/242.82 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0007716049382718 - InactiveTotalTime: 0 - PeakMemoryUsage: 4609756032 - PerHostPeakMemUsage: 4676864896 - PrepareTime: 363301303 - RowsProduced: 1 - TotalCpuTime: 665025367111 - TotalNetworkReceiveTime: 631134560176 - TotalNetworkSendTime: 469396 - TotalStorageWaitTime: 118954405 - TotalTime: 647339811304 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 321941 - CompileTime: 136721733 - InactiveTotalTime: 0 - LoadTime: 137152 - ModuleFileSize: 2309684 - OptimizationTime: 508469124 - PrepareTime: 349767891 - TotalTime: 997718114 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 41469 - OverallThroughput: 354688 - PeakMemoryUsage: 16384 - SerializeBatchTime: 7227 - ThriftTransmitTime(*): 385829 - TotalTime: 45110 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 137045 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647338915206 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646581564212 - BuildPartitionTime: 7937851352 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3166474711 - GetNewBlockTime: 6047507 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1063 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646987197050 - UnpinTime: 2215 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2282305967 - DeserializeRowBatchTimer: 4576019887 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630275027489 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99471 - SendersBlockedTimer: 252855093 - SendersBlockedTotalTimer(*): 1482852289 - TotalTime: 633602623131 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 20:1/242.57 MB 21:1/242.79 MB 0:1/242.44 MB 16:1/242.70 MB 1:1/242.82 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0030911901081916537 - BytesRead: 66193652 - BytesReadDataNodeCache: 0 - BytesReadLocal: 66193652 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 66193652 - DecompressionTime: 389301227 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 187743104 - PerReadThreadRawHdfsThroughput: 1414564595 - RemoteScanRanges: 0 - RowsRead: 9615078 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 7 - ScannerThreadsTotalWallClockTime: 1711344333 - MaterializeTupleTime(*): 1163355001 - ScannerThreadsSysTime: 18994000 - ScannerThreadsUserTime: 1568759000 - ScannerThreadsVoluntaryContextSwitches: 53 - TotalRawHdfsReadTime(*): 46794365 - TotalReadThroughput: 102229 - TotalTime: 392090688 Instance 544466204e0abc59:f772f133ae0a68c3 (host=demeter-csmau08-16.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 14:1/242.82 MB 18:1/243.06 MB 17:1/242.95 MB 20:1/242.81 MB 15:1/242.78 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9984555984555985 - InactiveTotalTime: 0 - PeakMemoryUsage: 4633890728 - PerHostPeakMemUsage: 4700999592 - PrepareTime: 365406601 - RowsProduced: 1 - TotalCpuTime: 664162208963 - TotalNetworkReceiveTime: 631508459032 - TotalNetworkSendTime: 435559 - TotalStorageWaitTime: 105562885 - TotalTime: 647323438541 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 331407 - CompileTime: 123202943 - InactiveTotalTime: 0 - LoadTime: 153017 - ModuleFileSize: 2309684 - OptimizationTime: 505931926 - PrepareTime: 351676847 - TotalTime: 983603971 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 46620 - OverallThroughput: 250905 - PeakMemoryUsage: 16384 - SerializeBatchTime: 10333 - ThriftTransmitTime(*): 343200 - TotalTime: 63769 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 228733 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647322547794 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646564473197 - BuildPartitionTime: 7605023993 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3186325348 - GetNewBlockTime: 3563757 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1507 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646968697487 - UnpinTime: 1081 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2247025706 - DeserializeRowBatchTimer: 4811128583 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630499348111 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99418 - SendersBlockedTimer: 239158848 - SendersBlockedTotalTimer(*): 1640597970 - TotalTime: 633939868676 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 14:1/242.82 MB 18:1/243.06 MB 17:1/242.95 MB 20:1/242.81 MB 15:1/242.78 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 69402293 - BytesReadDataNodeCache: 0 - BytesReadLocal: 69402293 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 69402293 - DecompressionTime: 308546784 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 211877800 - PerReadThreadRawHdfsThroughput: 2038819515 - RemoteScanRanges: 0 - RowsRead: 9642680 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 1258218009 - MaterializeTupleTime(*): 816798055 - ScannerThreadsSysTime: 50990000 - ScannerThreadsUserTime: 1093831000 - ScannerThreadsVoluntaryContextSwitches: 54 - TotalRawHdfsReadTime(*): 34040430 - TotalReadThroughput: 0 - TotalTime: 389369631 Instance 544466204e0abc59:f772f133ae0a68ed (host=demeter-csmaz10-17.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 17:1/242.35 MB 10:1/242.87 MB 11:1/243.01 MB 19:1/242.71 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9984555984555985 - InactiveTotalTime: 0 - PeakMemoryUsage: 4592156352 - PerHostPeakMemUsage: 4659265216 - PrepareTime: 373495122 - RowsProduced: 1 - TotalCpuTime: 664186896655 - TotalNetworkReceiveTime: 631114167057 - TotalNetworkSendTime: 556116 - TotalStorageWaitTime: 79763585 - TotalTime: 647304588976 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 335412 - CompileTime: 109138733 - InactiveTotalTime: 0 - LoadTime: 190007 - ModuleFileSize: 2309684 - OptimizationTime: 498694532 - PrepareTime: 359423441 - TotalTime: 969970919 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 35747 - OverallThroughput: 284692 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9594 - ThriftTransmitTime(*): 447588 - TotalTime: 56201 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 176902 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647303707861 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646545113813 - BuildPartitionTime: 8038769535 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3075054786 - GetNewBlockTime: 4441325 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1536 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646942319676 - UnpinTime: 1556 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2232433749 - DeserializeRowBatchTimer: 4581980714 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630195940742 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99483 - SendersBlockedTimer: 235520411 - SendersBlockedTotalTimer(*): 1844847070 - TotalTime: 633527516574 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 17:1/242.35 MB 10:1/242.87 MB 11:1/243.01 MB 19:1/242.71 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:12 ExecOption: Codegen enabled: 0 out of 4 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 53633173 - BytesReadDataNodeCache: 0 - BytesReadLocal: 53633173 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 53633173 - DecompressionTime: 219251407 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 4 - PeakMemoryUsage: 170143424 - PerReadThreadRawHdfsThroughput: 1838264346 - RemoteScanRanges: 0 - RowsRead: 7673930 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 2 - ScannerThreadsTotalWallClockTime: 922259193 - MaterializeTupleTime(*): 602784228 - ScannerThreadsSysTime: 13995000 - ScannerThreadsUserTime: 823872000 - ScannerThreadsVoluntaryContextSwitches: 45 - TotalRawHdfsReadTime(*): 29175985 - TotalReadThroughput: 0 - TotalTime: 383054364 Instance 544466204e0abc59:f772f133ae0a690e (host=demeter-csmaz08-13.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 12:1/237.35 MB 16:1/242.87 MB 7:1/242.66 MB 9:1/242.82 MB 10:1/242.97 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0015444015444017 - InactiveTotalTime: 0 - PeakMemoryUsage: 4646001424 - PerHostPeakMemUsage: 4713110288 - PrepareTime: 387274999 - RowsProduced: 1 - TotalCpuTime: 664680908623 - TotalNetworkReceiveTime: 631053891923 - TotalNetworkSendTime: 549354 - TotalStorageWaitTime: 110352606 - TotalTime: 647268776383 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 332336 - CompileTime: 125226086 - InactiveTotalTime: 0 - LoadTime: 191749 - ModuleFileSize: 2309684 - OptimizationTime: 496841102 - PrepareTime: 373508284 - TotalTime: 998383045 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 36063 - OverallThroughput: 302606 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9297 - ThriftTransmitTime(*): 443660 - TotalTime: 52874 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 169521 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647267877333 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646482926272 - BuildPartitionTime: 8057491388 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3092327960 - GetNewBlockTime: 6314479 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1908 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646892181114 - UnpinTime: 2028 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2248460689 - DeserializeRowBatchTimer: 4536964477 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630136117375 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99489 - SendersBlockedTimer: 268040270 - SendersBlockedTotalTimer(*): 1861879441 - TotalTime: 633488449264 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 12:1/237.35 MB 16:1/242.87 MB 7:1/242.66 MB 9:1/242.82 MB 10:1/242.97 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0038669760247486465 - BytesRead: 68804721 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68804721 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68804721 - DecompressionTime: 291334182 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 223988496 - PerReadThreadRawHdfsThroughput: 1716324561 - RemoteScanRanges: 0 - RowsRead: 9572594 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 14 - ScannerThreadsTotalWallClockTime: 1470308567 - MaterializeTupleTime(*): 1039918779 - ScannerThreadsSysTime: 12995000 - ScannerThreadsUserTime: 1339794000 - ScannerThreadsVoluntaryContextSwitches: 65 - TotalRawHdfsReadTime(*): 40088409 - TotalReadThroughput: 106262 - TotalTime: 395309977 Instance 544466204e0abc59:f772f133ae0a68e9 (host=demeter-csmaz11-14.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 11:1/242.60 MB 10:1/242.80 MB 0:1/242.82 MB 14:1/242.86 MB 9:1/243.05 MB 13:1/65.44 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9992283950617284 - InactiveTotalTime: 0 - PeakMemoryUsage: 4627414384 - PerHostPeakMemUsage: 4694523248 - PrepareTime: 376125018 - RowsProduced: 1 - TotalCpuTime: 664354347437 - TotalNetworkReceiveTime: 631150016231 - TotalNetworkSendTime: 411900 - TotalStorageWaitTime: 111895442 - TotalTime: 647259287720 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 353809 - CompileTime: 135801295 - InactiveTotalTime: 0 - LoadTime: 153116 - ModuleFileSize: 2309684 - OptimizationTime: 502247126 - PrepareTime: 362426882 - TotalTime: 1003337611 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 50704 - OverallThroughput: 272665 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9347 - ThriftTransmitTime(*): 315555 - TotalTime: 58680 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 175509 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647258388596 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646486312078 - BuildPartitionTime: 7954095650 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3146957393 - GetNewBlockTime: 6211281 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1407 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646893668036 - UnpinTime: 1996 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2254962731 - DeserializeRowBatchTimer: 4472553663 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630297373559 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99473 - SendersBlockedTimer: 267631091 - SendersBlockedTotalTimer(*): 1862028071 - TotalTime: 633593758170 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 11:1/242.60 MB 10:1/242.80 MB 0:1/242.82 MB 14:1/242.86 MB 9:1/243.05 MB 13:1/65.44 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0015455950540958269 - BytesRead: 73392362 - BytesReadDataNodeCache: 0 - BytesReadLocal: 73392362 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 73392362 - DecompressionTime: 294359358 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 205401456 - PerReadThreadRawHdfsThroughput: 1857232250 - RemoteScanRanges: 0 - RowsRead: 10119466 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 1231591025 - MaterializeTupleTime(*): 794536052 - ScannerThreadsSysTime: 38991000 - ScannerThreadsUserTime: 1069835000 - ScannerThreadsVoluntaryContextSwitches: 69 - TotalRawHdfsReadTime(*): 39517062 - TotalReadThroughput: 113347 - TotalTime: 393194920 Instance 544466204e0abc59:f772f133ae0a690b (host=demeter-csmaz08-20.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 21:1/242.91 MB 23:1/234.35 MB 11:1/242.86 MB 16:1/242.51 MB 0:1/164.06 MB 22:1/63.67 MB 19:1/242.36 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.002314814814815 - InactiveTotalTime: 0 - PeakMemoryUsage: 4650838976 - PerHostPeakMemUsage: 4717947840 - PrepareTime: 371695511 - RowsProduced: 1 - TotalCpuTime: 663713346430 - TotalNetworkReceiveTime: 631711471401 - TotalNetworkSendTime: 484274 - TotalStorageWaitTime: 169911225 - TotalTime: 647202910924 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 354742 - CompileTime: 136546236 - InactiveTotalTime: 0 - LoadTime: 141292 - ModuleFileSize: 2309684 - OptimizationTime: 498886881 - PrepareTime: 357980690 - TotalTime: 996106949 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 42740 - OverallThroughput: 270101 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12544 - ThriftTransmitTime(*): 374348 - TotalTime: 59237 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 217106 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647202003407 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646621969797 - BuildPartitionTime: 7372491181 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2989778033 - GetNewBlockTime: 7441234 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1796 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646841930932 - UnpinTime: 1986 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2261042130 - DeserializeRowBatchTimer: 4367111848 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630907600723 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99385 - SendersBlockedTimer: 265332369 - SendersBlockedTotalTimer(*): 1576819652 - TotalTime: 634149038877 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 21:1/242.91 MB 23:1/234.35 MB 11:1/242.86 MB 16:1/242.51 MB 0:1/164.06 MB 22:1/63.67 MB 19:1/242.36 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:21 ExecOption: Codegen enabled: 0 out of 7 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.00463678516228748 - BytesRead: 80634873 - BytesReadDataNodeCache: 0 - BytesReadLocal: 80634873 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 80634873 - DecompressionTime: 242842267 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 7 - PeakMemoryUsage: 228826048 - PerReadThreadRawHdfsThroughput: 1609649717 - RemoteScanRanges: 0 - RowsRead: 11384102 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 7 - ScannerThreadsInvoluntaryContextSwitches: 16 - ScannerThreadsTotalWallClockTime: 1133461666 - MaterializeTupleTime(*): 684812546 - ScannerThreadsSysTime: 13994000 - ScannerThreadsUserTime: 935854000 - ScannerThreadsVoluntaryContextSwitches: 95 - TotalRawHdfsReadTime(*): 50094671 - TotalReadThroughput: 124532 - TotalTime: 205146488 Instance 544466204e0abc59:f772f133ae0a68cd (host=demeter-csmau08-14.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 13:1/242.92 MB 15:1/242.92 MB 19:1/242.72 MB 22:1/63.11 MB 11:1/242.93 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.998456790123457 - InactiveTotalTime: 0 - PeakMemoryUsage: 4598882920 - PerHostPeakMemUsage: 4665991784 - PrepareTime: 367373037 - RowsProduced: 1 - TotalCpuTime: 663812329962 - TotalNetworkReceiveTime: 631334747936 - TotalNetworkSendTime: 458649 - TotalStorageWaitTime: 87905556 - TotalTime: 647190268167 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 330311 - CompileTime: 137669308 - InactiveTotalTime: 0 - LoadTime: 135525 - ModuleFileSize: 2309684 - OptimizationTime: 506715325 - PrepareTime: 353905900 - TotalTime: 1000904767 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 44766 - OverallThroughput: 255313 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12300 - ThriftTransmitTime(*): 357409 - TotalTime: 62668 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 202955 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647189346201 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646493674023 - BuildPartitionTime: 7684693638 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3142399597 - GetNewBlockTime: 6141643 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1854 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646833353264 - UnpinTime: 2010 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2313546855 - DeserializeRowBatchTimer: 4645312168 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630399727453 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99435 - SendersBlockedTimer: 259419781 - SendersBlockedTotalTimer(*): 1533570546 - TotalTime: 633835700770 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 13:1/242.92 MB 15:1/242.92 MB 19:1/242.72 MB 22:1/63.11 MB 11:1/242.93 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 7.727975270479134E-4 - BytesRead: 59402365 - BytesReadDataNodeCache: 0 - BytesReadLocal: 59402365 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 59402365 - DecompressionTime: 205791243 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 176869992 - PerReadThreadRawHdfsThroughput: 2092810955 - RemoteScanRanges: 0 - RowsRead: 8233733 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 2 - ScannerThreadsTotalWallClockTime: 905641854 - MaterializeTupleTime(*): 588304471 - ScannerThreadsSysTime: 4997000 - ScannerThreadsUserTime: 804875000 - ScannerThreadsVoluntaryContextSwitches: 59 - TotalRawHdfsReadTime(*): 28384009 - TotalReadThroughput: 91741 - TotalTime: 325762611 Instance 544466204e0abc59:f772f133ae0a690a (host=demeter-csmau10-14.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 8:1/242.94 MB 13:1/242.80 MB 7:1/242.69 MB 15:1/234.17 MB 6:1/242.72 MB 5:1/242.70 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.003088803088803 - InactiveTotalTime: 0 - PeakMemoryUsage: 4682831072 - PerHostPeakMemUsage: 4749939936 - PrepareTime: 369864937 - RowsProduced: 1 - TotalCpuTime: 664598855030 - TotalNetworkReceiveTime: 630838016967 - TotalNetworkSendTime: 507747 - TotalStorageWaitTime: 126403168 - TotalTime: 647184820197 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 330133 - CompileTime: 88129223 - InactiveTotalTime: 0 - LoadTime: 176747 - ModuleFileSize: 2309684 - OptimizationTime: 500786217 - PrepareTime: 356168039 - TotalTime: 947887737 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 42420 - OverallThroughput: 278891 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12733 - ThriftTransmitTime(*): 377172 - TotalTime: 57370 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 183241 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647184001969 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646566771289 - BuildPartitionTime: 8209242643 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2989697132 - GetNewBlockTime: 5520081 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1964 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646825584189 - UnpinTime: 2009 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2337679542 - DeserializeRowBatchTimer: 4644103450 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 629941737234 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99509 - SendersBlockedTimer: 216099336 - SendersBlockedTotalTimer(*): 1267848153 - TotalTime: 633362978594 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 8:1/242.94 MB 13:1/242.80 MB 7:1/242.69 MB 15:1/234.17 MB 6:1/242.72 MB 5:1/242.70 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.00463678516228748 - BytesRead: 82698912 - BytesReadDataNodeCache: 0 - BytesReadLocal: 82698912 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 82698912 - DecompressionTime: 282329949 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 260818144 - PerReadThreadRawHdfsThroughput: 1756150850 - RemoteScanRanges: 0 - RowsRead: 11440175 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 1221929585 - MaterializeTupleTime(*): 786905369 - ScannerThreadsSysTime: 41992000 - ScannerThreadsUserTime: 1044838000 - ScannerThreadsVoluntaryContextSwitches: 80 - TotalRawHdfsReadTime(*): 47091007 - TotalReadThroughput: 127720 - TotalTime: 244842881 Instance 544466204e0abc59:f772f133ae0a68bb (host=demeter-csmaz10-1.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 10:1/242.89 MB 4:1/242.95 MB 13:1/242.64 MB 6:1/242.97 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.998456790123457 - InactiveTotalTime: 0 - PeakMemoryUsage: 4592656464 - PerHostPeakMemUsage: 4659765328 - PrepareTime: 366950314 - RowsProduced: 1 - TotalCpuTime: 663463459310 - TotalNetworkReceiveTime: 631535133888 - TotalNetworkSendTime: 450636 - TotalStorageWaitTime: 79159101 - TotalTime: 647157048965 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 329656 - CompileTime: 109749521 - InactiveTotalTime: 0 - LoadTime: 165969 - ModuleFileSize: 2309684 - OptimizationTime: 499858682 - PrepareTime: 353383228 - TotalTime: 965668421 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 46501 - OverallThroughput: 280485 - PeakMemoryUsage: 16384 - SerializeBatchTime: 10224 - ThriftTransmitTime(*): 344076 - TotalTime: 57044 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 213085 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647156141212 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646392546596 - BuildPartitionTime: 7537353365 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2996403160 - GetNewBlockTime: 5264022 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1853 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646800632501 - UnpinTime: 1541 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2293423689 - DeserializeRowBatchTimer: 4692184075 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630572541321 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99406 - SendersBlockedTimer: 219840895 - SendersBlockedTotalTimer(*): 1287445976 - TotalTime: 634020004079 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 10:1/242.89 MB 4:1/242.95 MB 13:1/242.64 MB 6:1/242.97 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:12 ExecOption: Codegen enabled: 0 out of 4 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 7.727975270479134E-4 - BytesRead: 55433216 - BytesReadDataNodeCache: 0 - BytesReadLocal: 55433216 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 55433216 - DecompressionTime: 211405243 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 4 - PeakMemoryUsage: 170643536 - PerReadThreadRawHdfsThroughput: 2010780247 - RemoteScanRanges: 0 - RowsRead: 7696731 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 917625164 - MaterializeTupleTime(*): 604909226 - ScannerThreadsSysTime: 4996000 - ScannerThreadsUserTime: 828872000 - ScannerThreadsVoluntaryContextSwitches: 48 - TotalRawHdfsReadTime(*): 27568013 - TotalReadThroughput: 85611 - TotalTime: 394362919 Instance 544466204e0abc59:f772f133ae0a690c (host=demeter-csmaz10-5.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 19:1/242.92 MB 14:1/242.91 MB 0:1/242.88 MB 22:1/242.89 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0007722007722006 - InactiveTotalTime: 0 - PeakMemoryUsage: 4574693712 - PerHostPeakMemUsage: 4641802576 - PrepareTime: 371006719 - RowsProduced: 1 - TotalCpuTime: 663732119800 - TotalNetworkReceiveTime: 631376342614 - TotalNetworkSendTime: 469868 - TotalStorageWaitTime: 80400646 - TotalTime: 647112469097 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 321888 - CompileTime: 68546072 - InactiveTotalTime: 0 - LoadTime: 176617 - ModuleFileSize: 2309684 - OptimizationTime: 505866540 - PrepareTime: 357334792 - TotalTime: 934406427 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 46968 - OverallThroughput: 278867 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12037 - ThriftTransmitTime(*): 340652 - TotalTime: 57375 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 186563 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647111665692 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646344571799 - BuildPartitionTime: 7729572992 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2903891255 - GetNewBlockTime: 5595902 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1563 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646752304910 - UnpinTime: 12234 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2286504935 - DeserializeRowBatchTimer: 4505336188 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630535123992 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99434 - SendersBlockedTimer: 203976827 - SendersBlockedTotalTimer(*): 1387592291 - TotalTime: 633842177458 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 19:1/242.92 MB 14:1/242.91 MB 0:1/242.88 MB 22:1/242.89 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:12 ExecOption: Codegen enabled: 0 out of 4 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0030935808197989174 - BytesRead: 55717752 - BytesReadDataNodeCache: 0 - BytesReadLocal: 55717752 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 55717752 - DecompressionTime: 214862541 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 4 - PeakMemoryUsage: 152680784 - PerReadThreadRawHdfsThroughput: 1839420922 - RemoteScanRanges: 0 - RowsRead: 7726943 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 1 - ScannerThreadsTotalWallClockTime: 1156408271 - MaterializeTupleTime(*): 840157838 - ScannerThreadsSysTime: 12997000 - ScannerThreadsUserTime: 1057837000 - ScannerThreadsVoluntaryContextSwitches: 52 - TotalRawHdfsReadTime(*): 30290920 - TotalReadThroughput: 86117 - TotalTime: 393940885 Instance 544466204e0abc59:f772f133ae0a6908 (host=demeter-csmaz11-4.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 8:1/242.67 MB 23:1/165.08 MB 22:1/164.47 MB 3:1/243.04 MB 4:1/242.79 MB 13:1/242.41 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.998456790123457 - InactiveTotalTime: 0 - PeakMemoryUsage: 4627402864 - PerHostPeakMemUsage: 4694511728 - PrepareTime: 362146086 - RowsProduced: 1 - TotalCpuTime: 664401335209 - TotalNetworkReceiveTime: 630906254415 - TotalNetworkSendTime: 507134 - TotalStorageWaitTime: 120742202 - TotalTime: 647107619349 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 333393 - CompileTime: 136707230 - InactiveTotalTime: 0 - LoadTime: 152084 - ModuleFileSize: 2309684 - OptimizationTime: 506847901 - PrepareTime: 348789327 - TotalTime: 995007014 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 40185 - OverallThroughput: 305664 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9710 - ThriftTransmitTime(*): 398157 - TotalTime: 52345 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 234273 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647106744078 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646360771797 - BuildPartitionTime: 8168629613 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3068663822 - GetNewBlockTime: 3899475 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1817 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646755819290 - UnpinTime: 1873 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2271784452 - DeserializeRowBatchTimer: 4672888443 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 629978274092 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99510 - SendersBlockedTimer: 264541368 - SendersBlockedTotalTimer(*): 1830151495 - TotalTime: 633357896832 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 8:1/242.67 MB 23:1/165.08 MB 22:1/164.47 MB 3:1/243.04 MB 4:1/242.79 MB 13:1/242.41 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 7.727975270479134E-4 - BytesRead: 74528438 - BytesReadDataNodeCache: 0 - BytesReadLocal: 74528438 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 74528438 - DecompressionTime: 319169545 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 205389936 - PerReadThreadRawHdfsThroughput: 1659543809 - RemoteScanRanges: 0 - RowsRead: 10322580 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 1 - ScannerThreadsTotalWallClockTime: 1315273920 - MaterializeTupleTime(*): 838267868 - ScannerThreadsSysTime: 26993000 - ScannerThreadsUserTime: 1156821000 - ScannerThreadsVoluntaryContextSwitches: 70 - TotalRawHdfsReadTime(*): 44908991 - TotalReadThroughput: 115101 - TotalTime: 380496632 Instance 544466204e0abc59:f772f133ae0a68b7 (host=demeter-csmaz08-9.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 3:1/242.86 MB 10:1/65.19 MB 7:1/242.48 MB 4:1/242.34 MB 11:1/242.87 MB 9:1/242.77 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9992277992277991 - InactiveTotalTime: 0 - PeakMemoryUsage: 4625711544 - PerHostPeakMemUsage: 4692820408 - PrepareTime: 386075106 - RowsProduced: 1 - TotalCpuTime: 664208219462 - TotalNetworkReceiveTime: 630954204299 - TotalNetworkSendTime: 509702 - TotalStorageWaitTime: 124471290 - TotalTime: 647053969484 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 328145 - CompileTime: 136242811 - InactiveTotalTime: 0 - LoadTime: 148989 - ModuleFileSize: 2309684 - OptimizationTime: 501974307 - PrepareTime: 362240493 - TotalTime: 1003159353 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 38050 - OverallThroughput: 274127 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12370 - ThriftTransmitTime(*): 420494 - TotalTime: 58367 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 169606 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647020239311 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646243826806 - BuildPartitionTime: 8089464653 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3036266611 - GetNewBlockTime: 6814313 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 11094 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646655821273 - UnpinTime: 2242 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2275711186 - DeserializeRowBatchTimer: 4609800102 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630070987872 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99501 - SendersBlockedTimer: 299256597 - SendersBlockedTotalTimer(*): 1813731437 - TotalTime: 633414005221 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 3:1/242.86 MB 10:1/65.19 MB 7:1/242.48 MB 4:1/242.34 MB 11:1/242.87 MB 9:1/242.77 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0015467904098994587 - BytesRead: 69783459 - BytesReadDataNodeCache: 0 - BytesReadLocal: 69783459 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 69783459 - DecompressionTime: 292542964 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 203698616 - PerReadThreadRawHdfsThroughput: 1570425430 - RemoteScanRanges: 0 - RowsRead: 10146822 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 6 - ScannerThreadsTotalWallClockTime: 1349823707 - MaterializeTupleTime(*): 897691461 - ScannerThreadsSysTime: 4996000 - ScannerThreadsUserTime: 1208812000 - ScannerThreadsVoluntaryContextSwitches: 75 - TotalRawHdfsReadTime(*): 44436022 - TotalReadThroughput: 107773 - TotalTime: 387599275 Instance 544466204e0abc59:f772f133ae0a68d6 (host=demeter-csmau10-11.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 8:1/242.78 MB 6:1/242.89 MB 12:1/233.98 MB 2:1/242.42 MB 0:1/242.88 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9984555984555985 - InactiveTotalTime: 0 - PeakMemoryUsage: 4644426856 - PerHostPeakMemUsage: 4711535720 - PrepareTime: 364790146 - RowsProduced: 1 - TotalCpuTime: 663554903100 - TotalNetworkReceiveTime: 631437706212 - TotalNetworkSendTime: 495018 - TotalStorageWaitTime: 120245947 - TotalTime: 647051867126 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 341239 - CompileTime: 136856404 - InactiveTotalTime: 0 - LoadTime: 162546 - ModuleFileSize: 2309684 - OptimizationTime: 508152241 - PrepareTime: 351262165 - TotalTime: 999022198 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 41170 - OverallThroughput: 265732 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12540 - ThriftTransmitTime(*): 388626 - TotalTime: 60211 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 200238 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647050996824 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646322936979 - BuildPartitionTime: 7634755779 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 3023499062 - GetNewBlockTime: 5653512 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1610 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646697475251 - UnpinTime: 2222 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2279480535 - DeserializeRowBatchTimer: 4877897646 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630411113664 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99425 - SendersBlockedTimer: 259713044 - SendersBlockedTotalTimer(*): 1789168552 - TotalTime: 633896465136 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 8:1/242.78 MB 6:1/242.89 MB 12:1/233.98 MB 2:1/242.42 MB 0:1/242.88 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 68284840 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68284840 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68284840 - DecompressionTime: 244516857 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 222413928 - PerReadThreadRawHdfsThroughput: 1789794001 - RemoteScanRanges: 0 - RowsRead: 9561891 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 7 - ScannerThreadsTotalWallClockTime: 1091986901 - MaterializeTupleTime(*): 703303783 - ScannerThreadsSysTime: 13995000 - ScannerThreadsUserTime: 950853000 - ScannerThreadsVoluntaryContextSwitches: 60 - TotalRawHdfsReadTime(*): 38152346 - TotalReadThroughput: 0 - TotalTime: 360431774 Instance 544466204e0abc59:f772f133ae0a68f8 (host=demeter-csmaz08-3.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 6:1/242.69 MB 10:1/242.83 MB 11:1/238.71 MB 2:1/242.76 MB 5:1/242.88 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9984555984555985 - InactiveTotalTime: 0 - PeakMemoryUsage: 4639002224 - PerHostPeakMemUsage: 4706111088 - PrepareTime: 359686796 - RowsProduced: 1 - TotalCpuTime: 663216871528 - TotalNetworkReceiveTime: 631683316527 - TotalNetworkSendTime: 487981 - TotalStorageWaitTime: 107425717 - TotalTime: 647013258499 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 333151 - CompileTime: 115402489 - InactiveTotalTime: 0 - LoadTime: 183473 - ModuleFileSize: 2309684 - OptimizationTime: 504567547 - PrepareTime: 346358612 - TotalTime: 969046089 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 45551 - OverallThroughput: 285673 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12511 - ThriftTransmitTime(*): 351249 - TotalTime: 56008 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 202966 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 647012390964 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646272484036 - BuildPartitionTime: 7407073304 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2951177595 - GetNewBlockTime: 7103298 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1771 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646663893412 - UnpinTime: 2070 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2287092091 - DeserializeRowBatchTimer: 4407282132 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630871376762 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99385 - SendersBlockedTimer: 234680159 - SendersBlockedTotalTimer(*): 1611819151 - TotalTime: 634151054992 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 6:1/242.69 MB 10:1/242.83 MB 11:1/238.71 MB 2:1/242.76 MB 5:1/242.88 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 68931251 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68931251 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68931251 - DecompressionTime: 265330781 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 216989296 - PerReadThreadRawHdfsThroughput: 1823757067 - RemoteScanRanges: 0 - RowsRead: 9586911 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 6 - ScannerThreadsTotalWallClockTime: 1100838148 - MaterializeTupleTime(*): 705348938 - ScannerThreadsSysTime: 31993000 - ScannerThreadsUserTime: 954853000 - ScannerThreadsVoluntaryContextSwitches: 62 - TotalRawHdfsReadTime(*): 37796290 - TotalReadThroughput: 0 - TotalTime: 377473919 Instance 544466204e0abc59:f772f133ae0a68e1 (host=demeter-csmau10-1.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 15:1/242.88 MB 17:1/243.04 MB 10:1/242.94 MB 7:1/242.60 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9984555984555985 - InactiveTotalTime: 0 - PeakMemoryUsage: 4589580696 - PerHostPeakMemUsage: 4656689560 - PrepareTime: 370321252 - RowsProduced: 1 - TotalCpuTime: 665049397405 - TotalNetworkReceiveTime: 629787203387 - TotalNetworkSendTime: 442060 - TotalStorageWaitTime: 80828457 - TotalTime: 646993907877 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 342576 - CompileTime: 137058915 - InactiveTotalTime: 0 - LoadTime: 178360 - ModuleFileSize: 2309684 - OptimizationTime: 574406932 - PrepareTime: 356678289 - TotalTime: 1070998325 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 47670 - OverallThroughput: 274155 - PeakMemoryUsage: 16384 - SerializeBatchTime: 11717 - ThriftTransmitTime(*): 335638 - TotalTime: 58361 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 225673 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646993092000 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646243442511 - BuildPartitionTime: 8738897786 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2962700302 - GetNewBlockTime: 8385932 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1811 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646634081480 - UnpinTime: 1910 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2626188598 - DeserializeRowBatchTimer: 6396863390 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 627111956331 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99619 - SendersBlockedTimer: 334012433 - SendersBlockedTotalTimer(*): 2404909199 - TotalTime: 632663836027 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 15:1/242.88 MB 17:1/243.04 MB 10:1/242.94 MB 7:1/242.60 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:12 ExecOption: Codegen enabled: 0 out of 4 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 7.733952049497294E-4 - BytesRead: 55636709 - BytesReadDataNodeCache: 0 - BytesReadLocal: 55636709 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 55636709 - DecompressionTime: 227727567 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 4 - PeakMemoryUsage: 167567768 - PerReadThreadRawHdfsThroughput: 2011702226 - RemoteScanRanges: 0 - RowsRead: 7722028 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 2 - ScannerThreadsTotalWallClockTime: 967598054 - MaterializeTupleTime(*): 639025168 - ScannerThreadsSysTime: 4997000 - ScannerThreadsUserTime: 876864000 - ScannerThreadsVoluntaryContextSwitches: 40 - TotalRawHdfsReadTime(*): 27656533 - TotalReadThroughput: 85925 - TotalTime: 376571574 Instance 544466204e0abc59:f772f133ae0a68be (host=demeter-csmau08-20.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 8:1/242.79 MB 20:1/234.13 MB 12:1/242.94 MB 19:1/242.86 MB 10:1/242.75 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0015444015444017 - InactiveTotalTime: 0 - PeakMemoryUsage: 4636179224 - PerHostPeakMemUsage: 4703288088 - PrepareTime: 386080558 - RowsProduced: 1 - TotalCpuTime: 663476588271 - TotalNetworkReceiveTime: 631311323600 - TotalNetworkSendTime: 519623 - TotalStorageWaitTime: 131862179 - TotalTime: 646955808868 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 341317 - CompileTime: 90451594 - InactiveTotalTime: 0 - LoadTime: 183800 - ModuleFileSize: 2309684 - OptimizationTime: 502880743 - PrepareTime: 372387492 - TotalTime: 968540164 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 38146 - OverallThroughput: 302863 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9754 - ThriftTransmitTime(*): 419440 - TotalTime: 52829 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 175502 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646954918851 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646228341648 - BuildPartitionTime: 7732973259 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2880980581 - GetNewBlockTime: 7475201 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1724 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646580355596 - UnpinTime: 888 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2319275114 - DeserializeRowBatchTimer: 4602793685 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630396709256 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99438 - SendersBlockedTimer: 222838253 - SendersBlockedTotalTimer(*): 1524977848 - TotalTime: 633813791677 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 8:1/242.79 MB 20:1/234.13 MB 12:1/242.94 MB 19:1/242.86 MB 10:1/242.75 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0038669760247486465 - BytesRead: 68837355 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68837355 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68837355 - DecompressionTime: 253369798 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 214166296 - PerReadThreadRawHdfsThroughput: 1792127623 - RemoteScanRanges: 0 - RowsRead: 9568612 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 1141176427 - MaterializeTupleTime(*): 731486861 - ScannerThreadsSysTime: 10995000 - ScannerThreadsUserTime: 989847000 - ScannerThreadsVoluntaryContextSwitches: 61 - TotalRawHdfsReadTime(*): 38410967 - TotalReadThroughput: 106394 - TotalTime: 337309386 Instance 544466204e0abc59:f772f133ae0a68fa (host=demeter-csmaz10-3.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 22:1/242.78 MB 0:1/233.74 MB 19:1/242.84 MB 17:1/242.72 MB 18:1/242.82 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9984555984555985 - InactiveTotalTime: 0 - PeakMemoryUsage: 4633826776 - PerHostPeakMemUsage: 4700935640 - PrepareTime: 358916387 - RowsProduced: 1 - TotalCpuTime: 663465680680 - TotalNetworkReceiveTime: 631368350709 - TotalNetworkSendTime: 457662 - TotalStorageWaitTime: 104828949 - TotalTime: 646948688575 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 319217 - CompileTime: 116775548 - InactiveTotalTime: 0 - LoadTime: 169298 - ModuleFileSize: 2309684 - OptimizationTime: 496959686 - PrepareTime: 345407014 - TotalTime: 962052927 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 43244 - OverallThroughput: 281819 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12781 - ThriftTransmitTime(*): 369992 - TotalTime: 56774 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 207539 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646947868131 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646200331666 - BuildPartitionTime: 7752030593 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2817228001 - GetNewBlockTime: 4510349 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1584 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646600127163 - UnpinTime: 1913 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2266351198 - DeserializeRowBatchTimer: 4457082354 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630537906813 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99439 - SendersBlockedTimer: 228840326 - SendersBlockedTotalTimer(*): 1563927039 - TotalTime: 633805739056 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 22:1/242.78 MB 0:1/233.74 MB 19:1/242.84 MB 17:1/242.72 MB 18:1/242.82 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 7.733952049497294E-4 - BytesRead: 68965186 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68965186 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68965186 - DecompressionTime: 284051222 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 211813848 - PerReadThreadRawHdfsThroughput: 1887033062 - RemoteScanRanges: 0 - RowsRead: 9582388 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 1175044222 - MaterializeTupleTime(*): 763358755 - ScannerThreadsSysTime: 22995000 - ScannerThreadsUserTime: 1040839000 - ScannerThreadsVoluntaryContextSwitches: 57 - TotalRawHdfsReadTime(*): 36546888 - TotalReadThroughput: 0 - TotalTime: 385839657 Instance 544466204e0abc59:f772f133ae0a68f0 (host=demeter-csmaz10-15.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 16:1/242.38 MB 2:1/242.79 MB 3:1/234.32 MB 22:1/242.90 MB 19:1/242.73 MB 14:1/243.12 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0023166023166024 - InactiveTotalTime: 0 - PeakMemoryUsage: 4685867456 - PerHostPeakMemUsage: 4752976320 - PrepareTime: 366911535 - RowsProduced: 1 - TotalCpuTime: 665281960792 - TotalNetworkReceiveTime: 629683236097 - TotalNetworkSendTime: 515791 - TotalStorageWaitTime: 136522719 - TotalTime: 646933129264 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 330276 - CompileTime: 134085158 - InactiveTotalTime: 0 - LoadTime: 147536 - ModuleFileSize: 2309684 - OptimizationTime: 566923857 - PrepareTime: 353362308 - TotalTime: 1057102301 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 39127 - OverallThroughput: 261237 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12160 - ThriftTransmitTime(*): 408915 - TotalTime: 61247 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 213110 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646932273971 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646177114652 - BuildPartitionTime: 8844965709 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2900195209 - GetNewBlockTime: 8678979 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 2133 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646576749870 - UnpinTime: 1905 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2641391110 - DeserializeRowBatchTimer: 6494203057 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 627008994595 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99632 - SendersBlockedTimer: 321597886 - SendersBlockedTotalTimer(*): 1652871817 - TotalTime: 632582839789 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 16:1/242.38 MB 2:1/242.79 MB 3:1/234.32 MB 22:1/242.90 MB 19:1/242.73 MB 14:1/243.12 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.004640371229698376 - BytesRead: 82689848 - BytesReadDataNodeCache: 0 - BytesReadLocal: 82689848 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 82689848 - DecompressionTime: 300288406 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 263854528 - PerReadThreadRawHdfsThroughput: 1785280204 - RemoteScanRanges: 0 - RowsRead: 11485917 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 2 - ScannerThreadsTotalWallClockTime: 1289447122 - MaterializeTupleTime(*): 822123574 - ScannerThreadsSysTime: 14995000 - ScannerThreadsUserTime: 1128825000 - ScannerThreadsVoluntaryContextSwitches: 74 - TotalRawHdfsReadTime(*): 46317574 - TotalReadThroughput: 127805 - TotalTime: 384988801 Instance 544466204e0abc59:f772f133ae0a68b6 (host=demeter-csmaz10-4.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 9:1/242.97 MB 7:1/242.86 MB 14:1/242.89 MB 13:1/243.02 MB 4:1/243.01 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9984555984555985 - InactiveTotalTime: 0 - PeakMemoryUsage: 4617729792 - PerHostPeakMemUsage: 4684838656 - PrepareTime: 391700832 - RowsProduced: 1 - TotalCpuTime: 663040244385 - TotalNetworkReceiveTime: 631766827575 - TotalNetworkSendTime: 455864 - TotalStorageWaitTime: 116603833 - TotalTime: 646930202549 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 334159 - CompileTime: 136386823 - InactiveTotalTime: 0 - LoadTime: 250131 - ModuleFileSize: 2309684 - OptimizationTime: 497957386 - PrepareTime: 377804337 - TotalTime: 1015092663 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 45196 - OverallThroughput: 277628 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12284 - ThriftTransmitTime(*): 354007 - TotalTime: 57631 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 165873 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646929337345 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646148082891 - BuildPartitionTime: 7348834378 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2915908151 - GetNewBlockTime: 7958448 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1594 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646549125132 - UnpinTime: 1901 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2261769787 - DeserializeRowBatchTimer: 4721371337 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630852542886 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99377 - SendersBlockedTimer: 268204577 - SendersBlockedTotalTimer(*): 1595217872 - TotalTime: 634201614433 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 9:1/242.97 MB 7:1/242.86 MB 14:1/242.89 MB 13:1/243.02 MB 4:1/243.01 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 7.733952049497294E-4 - BytesRead: 69544730 - BytesReadDataNodeCache: 0 - BytesReadLocal: 69544730 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 69544730 - DecompressionTime: 273071706 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 195716864 - PerReadThreadRawHdfsThroughput: 1674270912 - RemoteScanRanges: 0 - RowsRead: 9636467 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 10 - ScannerThreadsTotalWallClockTime: 1210123306 - MaterializeTupleTime(*): 796410596 - ScannerThreadsSysTime: 16996000 - ScannerThreadsUserTime: 1070834000 - ScannerThreadsVoluntaryContextSwitches: 65 - TotalRawHdfsReadTime(*): 41537322 - TotalReadThroughput: 107404 - TotalTime: 387298981 Instance 544466204e0abc59:f772f133ae0a6904 (host=demeter-csmau10-20.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 23:1/243.03 MB 9:1/234.27 MB 2:1/242.87 MB 5:1/242.68 MB 10:1/234.48 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9984555984555985 - InactiveTotalTime: 0 - PeakMemoryUsage: 4617727640 - PerHostPeakMemUsage: 4684836504 - PrepareTime: 367798773 - RowsProduced: 1 - TotalCpuTime: 662846833088 - TotalNetworkReceiveTime: 631915263859 - TotalNetworkSendTime: 547495 - TotalStorageWaitTime: 114388597 - TotalTime: 646928125769 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 324752 - CompileTime: 81278531 - InactiveTotalTime: 0 - LoadTime: 166233 - ModuleFileSize: 2309684 - OptimizationTime: 504828894 - PrepareTime: 354292394 - TotalTime: 943101786 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 35749 - OverallThroughput: 276984 - PeakMemoryUsage: 16384 - SerializeBatchTime: 13007 - ThriftTransmitTime(*): 447564 - TotalTime: 57765 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 195716 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646927323921 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646175171056 - BuildPartitionTime: 7252574680 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2824146990 - GetNewBlockTime: 6361179 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1543 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646570908040 - UnpinTime: 885 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2259854008 - DeserializeRowBatchTimer: 4918940659 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630899277869 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99354 - SendersBlockedTimer: 209990593 - SendersBlockedTotalTimer(*): 1020162420 - TotalTime: 634348877416 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 23:1/243.03 MB 9:1/234.27 MB 2:1/242.87 MB 5:1/242.68 MB 10:1/234.48 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 7.733952049497294E-4 - BytesRead: 68467248 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68467248 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68467248 - DecompressionTime: 263281427 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 195714712 - PerReadThreadRawHdfsThroughput: 1658347862 - RemoteScanRanges: 0 - RowsRead: 9511953 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 13 - ScannerThreadsTotalWallClockTime: 1185948760 - MaterializeTupleTime(*): 781871597 - ScannerThreadsSysTime: 5996000 - ScannerThreadsUserTime: 1058837000 - ScannerThreadsVoluntaryContextSwitches: 60 - TotalRawHdfsReadTime(*): 41286421 - TotalReadThroughput: 105822 - TotalTime: 381227326 Instance 544466204e0abc59:f772f133ae0a68ef (host=demeter-csmaz08-8.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 8:1/234.44 MB 10:1/242.71 MB 6:1/242.68 MB 5:1/242.88 MB 0:1/243.02 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.998454404945904 - InactiveTotalTime: 0 - PeakMemoryUsage: 4635978000 - PerHostPeakMemUsage: 4703086864 - PrepareTime: 365283451 - RowsProduced: 1 - TotalCpuTime: 663011892363 - TotalNetworkReceiveTime: 631722619243 - TotalNetworkSendTime: 532635 - TotalStorageWaitTime: 105761613 - TotalTime: 646913556696 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 339379 - CompileTime: 138358068 - InactiveTotalTime: 0 - LoadTime: 162922 - ModuleFileSize: 2309684 - OptimizationTime: 510903989 - PrepareTime: 351785667 - TotalTime: 1003783583 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 37481 - OverallThroughput: 246100 - PeakMemoryUsage: 16384 - SerializeBatchTime: 15240 - ThriftTransmitTime(*): 426880 - TotalTime: 65014 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 227142 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646912717290 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646158755058 - BuildPartitionTime: 7301015849 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2871652300 - GetNewBlockTime: 13702325 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1711 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646558648977 - UnpinTime: 1998 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2293325251 - DeserializeRowBatchTimer: 4526060591 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630818774851 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99377 - SendersBlockedTimer: 269788101 - SendersBlockedTotalTimer(*): 2135989474 - TotalTime: 634205398631 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 8:1/234.44 MB 10:1/242.71 MB 6:1/242.68 MB 5:1/242.88 MB 0:1/243.02 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 69073881 - BytesReadDataNodeCache: 0 - BytesReadLocal: 69073881 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 69073881 - DecompressionTime: 254608312 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 213965072 - PerReadThreadRawHdfsThroughput: 1694991107 - RemoteScanRanges: 0 - RowsRead: 9557252 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 1117621722 - MaterializeTupleTime(*): 729505774 - ScannerThreadsSysTime: 8995000 - ScannerThreadsUserTime: 995847000 - ScannerThreadsVoluntaryContextSwitches: 68 - TotalRawHdfsReadTime(*): 40751766 - TotalReadThroughput: 0 - TotalTime: 385900517 Instance 544466204e0abc59:f772f133ae0a68c4 (host=demeter-csmaz08-5.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 15:1/242.77 MB 11:1/242.91 MB 8:1/242.76 MB 14:1/242.43 MB 12:1/242.84 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9984555984555985 - InactiveTotalTime: 0 - PeakMemoryUsage: 4646440840 - PerHostPeakMemUsage: 4713549704 - PrepareTime: 368918483 - RowsProduced: 1 - TotalCpuTime: 662366223327 - TotalNetworkReceiveTime: 632332837314 - TotalNetworkSendTime: 474656 - TotalStorageWaitTime: 99250700 - TotalTime: 646905431520 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 358963 - CompileTime: 126548117 - InactiveTotalTime: 0 - LoadTime: 149218 - ModuleFileSize: 2309684 - OptimizationTime: 502746612 - PrepareTime: 355127030 - TotalTime: 987265506 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 43051 - OverallThroughput: 293126 - PeakMemoryUsage: 16384 - SerializeBatchTime: 10517 - ThriftTransmitTime(*): 371648 - TotalTime: 54584 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 180666 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646904505705 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646237523069 - BuildPartitionTime: 6979604697 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2955905540 - GetNewBlockTime: 5825080 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1940 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646547185003 - UnpinTime: 2071 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2104499228 - DeserializeRowBatchTimer: 4178168718 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 631600502475 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99314 - SendersBlockedTimer: 243893096 - SendersBlockedTotalTimer(*): 1188680435 - TotalTime: 634604024359 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 15:1/242.77 MB 11:1/242.91 MB 8:1/242.76 MB 14:1/242.43 MB 12:1/242.84 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 68468349 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68468349 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68468349 - DecompressionTime: 231728635 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 224427912 - PerReadThreadRawHdfsThroughput: 1793567927 - RemoteScanRanges: 0 - RowsRead: 9591189 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 2 - ScannerThreadsTotalWallClockTime: 1025180538 - MaterializeTupleTime(*): 669309923 - ScannerThreadsSysTime: 14995000 - ScannerThreadsUserTime: 902860000 - ScannerThreadsVoluntaryContextSwitches: 67 - TotalRawHdfsReadTime(*): 38174383 - TotalReadThroughput: 0 - TotalTime: 294762088 Instance 544466204e0abc59:f772f133ae0a68d8 (host=demeter-csmau10-2.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 2:1/242.91 MB 0:1/242.68 MB 3:1/242.82 MB 1:1/243.09 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.998454404945904 - InactiveTotalTime: 0 - PeakMemoryUsage: 4613737152 - PerHostPeakMemUsage: 4680846016 - PrepareTime: 369131878 - RowsProduced: 1 - TotalCpuTime: 663091973719 - TotalNetworkReceiveTime: 631447212889 - TotalNetworkSendTime: 513119 - TotalStorageWaitTime: 93607773 - TotalTime: 646898779261 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 334216 - CompileTime: 135797654 - InactiveTotalTime: 0 - LoadTime: 127036 - ModuleFileSize: 2309684 - OptimizationTime: 493273153 - PrepareTime: 355397349 - TotalTime: 987195700 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 39289 - OverallThroughput: 284237 - PeakMemoryUsage: 16384 - SerializeBatchTime: 10400 - ThriftTransmitTime(*): 407237 - TotalTime: 56291 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 168423 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646897876649 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646320472968 - BuildPartitionTime: 7663630319 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2882906269 - GetNewBlockTime: 3321937 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 2440 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646540405052 - UnpinTime: 2182 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2271513962 - DeserializeRowBatchTimer: 4447012966 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630623005759 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99426 - SendersBlockedTimer: 247902241 - SendersBlockedTotalTimer(*): 1454287469 - TotalTime: 633890681667 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 2:1/242.91 MB 0:1/242.68 MB 3:1/242.82 MB 1:1/243.09 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:12 ExecOption: Codegen enabled: 0 out of 4 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 55760189 - BytesReadDataNodeCache: 0 - BytesReadLocal: 55760189 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 55760189 - DecompressionTime: 204495508 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 4 - PeakMemoryUsage: 191724224 - PerReadThreadRawHdfsThroughput: 1595523124 - RemoteScanRanges: 0 - RowsRead: 7724756 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 783100465 - MaterializeTupleTime(*): 469664998 - ScannerThreadsSysTime: 42991000 - ScannerThreadsUserTime: 637901000 - ScannerThreadsVoluntaryContextSwitches: 149 - TotalRawHdfsReadTime(*): 34947904 - TotalReadThroughput: 0 - TotalTime: 206174441 Instance 544466204e0abc59:f772f133ae0a68ba (host=demeter-csmaz11-1.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 20:1/242.95 MB 21:1/242.77 MB 3:1/234.70 MB 19:1/242.56 MB 18:1/242.85 MB 16:1/243.08 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0023166023166024 - InactiveTotalTime: 0 - PeakMemoryUsage: 4660787768 - PerHostPeakMemUsage: 4727896632 - PrepareTime: 371619924 - RowsProduced: 1 - TotalCpuTime: 663423539075 - TotalNetworkReceiveTime: 631544777120 - TotalNetworkSendTime: 441630 - TotalStorageWaitTime: 154759199 - TotalTime: 646885742669 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 337255 - CompileTime: 134582287 - InactiveTotalTime: 0 - LoadTime: 144446 - ModuleFileSize: 2309684 - OptimizationTime: 498174731 - PrepareTime: 357910340 - TotalTime: 993357484 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 46593 - OverallThroughput: 272814 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12910 - ThriftTransmitTime(*): 343398 - TotalTime: 58648 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 172363 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646884886033 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646265257321 - BuildPartitionTime: 7568435954 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2954500631 - GetNewBlockTime: 3767533 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1593 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646524863767 - UnpinTime: 839 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2262402253 - DeserializeRowBatchTimer: 4345698174 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630771926567 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99412 - SendersBlockedTimer: 246510374 - SendersBlockedTotalTimer(*): 1446633416 - TotalTime: 633980855667 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 20:1/242.95 MB 21:1/242.77 MB 3:1/234.70 MB 19:1/242.56 MB 18:1/242.85 MB 16:1/243.08 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.004640371229698376 - BytesRead: 82799109 - BytesReadDataNodeCache: 0 - BytesReadLocal: 82799109 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 82799109 - DecompressionTime: 306163780 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 238774840 - PerReadThreadRawHdfsThroughput: 1331162288 - RemoteScanRanges: 0 - RowsRead: 11487065 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 9 - ScannerThreadsTotalWallClockTime: 1338133114 - MaterializeTupleTime(*): 848902752 - ScannerThreadsSysTime: 48990000 - ScannerThreadsUserTime: 1123827000 - ScannerThreadsVoluntaryContextSwitches: 73 - TotalRawHdfsReadTime(*): 62200612 - TotalReadThroughput: 127973 - TotalTime: 245520689 Instance 544466204e0abc59:f772f133ae0a68d3 (host=demeter-csmaz11-17.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 16:1/64.37 MB 6:1/242.65 MB 7:1/242.97 MB 2:1/242.89 MB 17:1/233.89 MB 14:1/242.86 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0015444015444017 - InactiveTotalTime: 0 - PeakMemoryUsage: 4650320064 - PerHostPeakMemUsage: 4717428928 - PrepareTime: 371207671 - RowsProduced: 1 - TotalCpuTime: 663636672332 - TotalNetworkReceiveTime: 631108924443 - TotalNetworkSendTime: 415157 - TotalStorageWaitTime: 127261284 - TotalTime: 646875620085 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 356980 - CompileTime: 137055619 - InactiveTotalTime: 0 - LoadTime: 190320 - ModuleFileSize: 2309684 - OptimizationTime: 501606250 - PrepareTime: 357414323 - TotalTime: 998911192 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 43648 - OverallThroughput: 263543 - PeakMemoryUsage: 16384 - SerializeBatchTime: 11914 - ThriftTransmitTime(*): 366567 - TotalTime: 60711 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 210078 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646874706312 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646218190199 - BuildPartitionTime: 7871287964 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2916233504 - GetNewBlockTime: 8090212 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1659 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646514997534 - UnpinTime: 2284 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2357878857 - DeserializeRowBatchTimer: 4321955618 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630313306946 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99463 - SendersBlockedTimer: 259933400 - SendersBlockedTotalTimer(*): 2045730807 - TotalTime: 633654406033 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 16:1/64.37 MB 6:1/242.65 MB 7:1/242.97 MB 2:1/242.89 MB 17:1/233.89 MB 14:1/242.86 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0038669760247486465 - BytesRead: 72738435 - BytesReadDataNodeCache: 0 - BytesReadLocal: 72738435 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 72738435 - DecompressionTime: 260980010 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 228307136 - PerReadThreadRawHdfsThroughput: 2052223496 - RemoteScanRanges: 0 - RowsRead: 10085001 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 6 - ScannerThreadsTotalWallClockTime: 1139313075 - MaterializeTupleTime(*): 716964824 - ScannerThreadsSysTime: 17994000 - ScannerThreadsUserTime: 981848000 - ScannerThreadsVoluntaryContextSwitches: 79 - TotalRawHdfsReadTime(*): 35443720 - TotalReadThroughput: 112424 - TotalTime: 282361425 Instance 544466204e0abc59:f772f133ae0a68f3 (host=demeter-csmaz10-6.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 13:1/242.66 MB 10:1/242.47 MB 11:1/243.12 MB 19:1/242.86 MB 15:1/242.45 MB 14:1/242.82 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9984555984555985 - InactiveTotalTime: 0 - PeakMemoryUsage: 4688657632 - PerHostPeakMemUsage: 4755766496 - PrepareTime: 368971607 - RowsProduced: 1 - TotalCpuTime: 663286377894 - TotalNetworkReceiveTime: 631594156907 - TotalNetworkSendTime: 469880 - TotalStorageWaitTime: 126589919 - TotalTime: 646875354998 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 329504 - CompileTime: 139070391 - InactiveTotalTime: 0 - LoadTime: 141272 - ModuleFileSize: 2309684 - OptimizationTime: 507979664 - PrepareTime: 355487274 - TotalTime: 1005123273 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 43355 - OverallThroughput: 275510 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12476 - ThriftTransmitTime(*): 369045 - TotalTime: 58074 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 220477 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646874510077 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646197038928 - BuildPartitionTime: 7483496608 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2905417825 - GetNewBlockTime: 24797797 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1844 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646516974032 - UnpinTime: 1972 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2268686122 - DeserializeRowBatchTimer: 4384425144 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630789379891 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99401 - SendersBlockedTimer: 271620187 - SendersBlockedTotalTimer(*): 2156913609 - TotalTime: 634050729832 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 13:1/242.66 MB 10:1/242.47 MB 11:1/243.12 MB 19:1/242.86 MB 15:1/242.45 MB 14:1/242.82 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 7.733952049497294E-4 - BytesRead: 81459556 - BytesReadDataNodeCache: 0 - BytesReadLocal: 81459556 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 81459556 - DecompressionTime: 298417188 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 266644704 - PerReadThreadRawHdfsThroughput: 1707584479 - RemoteScanRanges: 0 - RowsRead: 11543766 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 1286648480 - MaterializeTupleTime(*): 830413911 - ScannerThreadsSysTime: 21993000 - ScannerThreadsUserTime: 1128825000 - ScannerThreadsVoluntaryContextSwitches: 75 - TotalRawHdfsReadTime(*): 47704554 - TotalReadThroughput: 125903 - TotalTime: 304477371 Instance 544466204e0abc59:f772f133ae0a690d (host=demeter-csmaz10-8.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 4:1/242.84 MB 20:1/242.83 MB 8:1/62.68 MB 3:1/242.67 MB 22:1/242.97 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9984555984555985 - InactiveTotalTime: 0 - PeakMemoryUsage: 4593946264 - PerHostPeakMemUsage: 4661055128 - PrepareTime: 364793543 - RowsProduced: 1 - TotalCpuTime: 663387498024 - TotalNetworkReceiveTime: 631065193842 - TotalNetworkSendTime: 465437 - TotalStorageWaitTime: 96245519 - TotalTime: 646814143197 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 344208 - CompileTime: 136227068 - InactiveTotalTime: 0 - LoadTime: 186266 - ModuleFileSize: 2309684 - OptimizationTime: 505005242 - PrepareTime: 351094232 - TotalTime: 995178677 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 43256 - OverallThroughput: 285581 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9730 - ThriftTransmitTime(*): 369890 - TotalTime: 56026 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 187449 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646813203326 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646043326903 - BuildPartitionTime: 7854411061 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2849734543 - GetNewBlockTime: 6296070 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 2123 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646459833234 - UnpinTime: 2033 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2401310555 - DeserializeRowBatchTimer: 4438913987 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630237033522 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99463 - SendersBlockedTimer: 264543849 - SendersBlockedTotalTimer(*): 1572366646 - TotalTime: 633653535938 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 4:1/242.84 MB 20:1/242.83 MB 8:1/62.68 MB 3:1/242.67 MB 22:1/242.97 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 7.733952049497294E-4 - BytesRead: 59277682 - BytesReadDataNodeCache: 0 - BytesReadLocal: 59277682 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 59277682 - DecompressionTime: 237252133 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 171933336 - PerReadThreadRawHdfsThroughput: 2156838568 - RemoteScanRanges: 0 - RowsRead: 8220500 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 1 - ScannerThreadsTotalWallClockTime: 1049254465 - MaterializeTupleTime(*): 688603532 - ScannerThreadsSysTime: 6997000 - ScannerThreadsUserTime: 937854000 - ScannerThreadsVoluntaryContextSwitches: 52 - TotalRawHdfsReadTime(*): 27483597 - TotalReadThroughput: 91619 - TotalTime: 402616786 Instance 544466204e0abc59:f772f133ae0a68c0 (host=demeter-csmaz08-15.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 7:1/234.56 MB 23:1/242.81 MB 1:1/242.74 MB 5:1/242.80 MB 16:1/242.89 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9992277992277991 - InactiveTotalTime: 0 - PeakMemoryUsage: 4634230680 - PerHostPeakMemUsage: 4701339544 - PrepareTime: 378456887 - RowsProduced: 1 - TotalCpuTime: 664463074023 - TotalNetworkReceiveTime: 630099146167 - TotalNetworkSendTime: 529770 - TotalStorageWaitTime: 116556511 - TotalTime: 646806474343 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 374310 - CompileTime: 115294727 - InactiveTotalTime: 0 - LoadTime: 173256 - ModuleFileSize: 2309684 - OptimizationTime: 571109290 - PrepareTime: 364255413 - TotalTime: 1053531954 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 38326 - OverallThroughput: 261177 - PeakMemoryUsage: 16384 - SerializeBatchTime: 10494 - ThriftTransmitTime(*): 417469 - TotalTime: 61261 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 251081 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646805386067 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646189832002 - BuildPartitionTime: 8471921152 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2898036415 - GetNewBlockTime: 8590714 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1764 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646438864783 - UnpinTime: 2335 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2609005510 - DeserializeRowBatchTimer: 6332588856 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 627422998845 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99574 - SendersBlockedTimer: 310667525 - SendersBlockedTotalTimer(*): 1890242105 - TotalTime: 632947863603 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 7:1/234.56 MB 23:1/242.81 MB 1:1/242.74 MB 5:1/242.80 MB 16:1/242.89 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0015467904098994587 - BytesRead: 68530904 - BytesReadDataNodeCache: 0 - BytesReadLocal: 68530904 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 68530904 - DecompressionTime: 215158118 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 212217752 - PerReadThreadRawHdfsThroughput: 1488791209 - RemoteScanRanges: 0 - RowsRead: 9583652 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 994878789 - MaterializeTupleTime(*): 641401434 - ScannerThreadsSysTime: 17994000 - ScannerThreadsUserTime: 853868000 - ScannerThreadsVoluntaryContextSwitches: 69 - TotalRawHdfsReadTime(*): 46031239 - TotalReadThroughput: 105921 - TotalTime: 234001519 Instance 544466204e0abc59:f772f133ae0a68c6 (host=demeter-csmau10-6.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 1:1/63.22 MB 0:1/242.42 MB 15:1/242.82 MB 2:1/242.44 MB 10:1/242.79 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0015444015444017 - InactiveTotalTime: 0 - PeakMemoryUsage: 4607135800 - PerHostPeakMemUsage: 4674244664 - PrepareTime: 360663958 - RowsProduced: 1 - TotalCpuTime: 662984248736 - TotalNetworkReceiveTime: 631352976851 - TotalNetworkSendTime: 479060 - TotalStorageWaitTime: 104983742 - TotalTime: 646803438670 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 321483 - CompileTime: 77752892 - InactiveTotalTime: 0 - LoadTime: 176304 - ModuleFileSize: 2309684 - OptimizationTime: 505749426 - PrepareTime: 347177066 - TotalTime: 933433742 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 41874 - OverallThroughput: 263669 - PeakMemoryUsage: 16384 - SerializeBatchTime: 13840 - ThriftTransmitTime(*): 382093 - TotalTime: 60682 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 198250 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646802520333 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646245057461 - BuildPartitionTime: 7772742327 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2786358418 - GetNewBlockTime: 3125290 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 2173 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646453065671 - UnpinTime: 2030 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2315185530 - DeserializeRowBatchTimer: 4858082842 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630367844813 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99433 - SendersBlockedTimer: 190206910 - SendersBlockedTotalTimer(*): 1109391994 - TotalTime: 633848502497 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 1:1/63.22 MB 0:1/242.42 MB 15:1/242.82 MB 2:1/242.44 MB 10:1/242.79 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0038669760247486465 - BytesRead: 57967668 - BytesReadDataNodeCache: 0 - BytesReadLocal: 57967668 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 57967668 - DecompressionTime: 194755598 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 185122872 - PerReadThreadRawHdfsThroughput: 1614473191 - RemoteScanRanges: 0 - RowsRead: 8145893 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 809301834 - MaterializeTupleTime(*): 491066208 - ScannerThreadsSysTime: 30992000 - ScannerThreadsUserTime: 664896000 - ScannerThreadsVoluntaryContextSwitches: 81 - TotalRawHdfsReadTime(*): 35905005 - TotalReadThroughput: 89594 - TotalTime: 194169688 Instance 544466204e0abc59:f772f133ae0a68c1 (host=demeter-csmaz11-16.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 15:1/242.44 MB 10:1/242.86 MB 11:1/242.89 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.998454404945904 - InactiveTotalTime: 0 - PeakMemoryUsage: 4565137728 - PerHostPeakMemUsage: 4632246592 - PrepareTime: 377226213 - RowsProduced: 1 - TotalCpuTime: 662534412409 - TotalNetworkReceiveTime: 631603948003 - TotalNetworkSendTime: 455188 - TotalStorageWaitTime: 51192727 - TotalTime: 646799017307 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 343312 - CompileTime: 69096931 - InactiveTotalTime: 0 - LoadTime: 155940 - ModuleFileSize: 2309684 - OptimizationTime: 498871190 - PrepareTime: 363171901 - TotalTime: 934038822 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 45068 - OverallThroughput: 270206 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9520 - ThriftTransmitTime(*): 355019 - TotalTime: 59214 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 201537 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646798040624 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646213896189 - BuildPartitionTime: 7548021042 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2838790449 - GetNewBlockTime: 29322930 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1819 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646432657257 - UnpinTime: 822 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2298803276 - DeserializeRowBatchTimer: 4426578226 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630790820315 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99397 - SendersBlockedTimer: 189985327 - SendersBlockedTotalTimer(*): 1292593212 - TotalTime: 634076653378 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 15:1/242.44 MB 10:1/242.86 MB 11:1/242.89 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:9 ExecOption: Codegen enabled: 0 out of 3 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 40715056 - BytesReadDataNodeCache: 0 - BytesReadLocal: 40715056 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 40715056 - DecompressionTime: 122915583 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 3 - NumScannerThreadsStarted: 3 - PeakMemoryUsage: 143124800 - PerReadThreadRawHdfsThroughput: 2157772916 - RemoteScanRanges: 0 - RowsRead: 5787795 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 3 - ScannerThreadsInvoluntaryContextSwitches: 2 - ScannerThreadsTotalWallClockTime: 607586407 - MaterializeTupleTime(*): 421878772 - ScannerThreadsSysTime: 7997000 - ScannerThreadsUserTime: 545916000 - ScannerThreadsVoluntaryContextSwitches: 35 - TotalRawHdfsReadTime(*): 18869018 - TotalReadThroughput: 0 - TotalTime: 205452848 Instance 544466204e0abc59:f772f133ae0a68db (host=demeter-csmau08-17.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 4:1/242.74 MB 3:1/242.42 MB 12:1/235.56 MB 21:1/242.89 MB 23:1/242.69 MB 11:1/234.71 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0023166023166024 - InactiveTotalTime: 0 - PeakMemoryUsage: 4664413968 - PerHostPeakMemUsage: 4731522832 - PrepareTime: 363344698 - RowsProduced: 1 - TotalCpuTime: 663166302068 - TotalNetworkReceiveTime: 631534123660 - TotalNetworkSendTime: 473941 - TotalStorageWaitTime: 134379799 - TotalTime: 646796116700 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 329213 - CompileTime: 125616375 - InactiveTotalTime: 0 - LoadTime: 147196 - ModuleFileSize: 2309684 - OptimizationTime: 497176050 - PrepareTime: 350081595 - TotalTime: 975459839 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 42855 - OverallThroughput: 274329 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12901 - ThriftTransmitTime(*): 373352 - TotalTime: 58324 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 196656 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646795301606 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646064129967 - BuildPartitionTime: 7584440127 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2792485200 - GetNewBlockTime: 5666494 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1970 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646443195748 - UnpinTime: 1811 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2259572392 - DeserializeRowBatchTimer: 4669399314 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630621134571 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99413 - SendersBlockedTimer: 236957149 - SendersBlockedTotalTimer(*): 1856014329 - TotalTime: 633971679111 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 4:1/242.74 MB 3:1/242.42 MB 12:1/235.56 MB 21:1/242.89 MB 23:1/242.69 MB 11:1/234.71 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.004640371229698376 - BytesRead: 80045871 - BytesReadDataNodeCache: 0 - BytesReadLocal: 80045871 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 80045871 - DecompressionTime: 283653770 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 242401040 - PerReadThreadRawHdfsThroughput: 1971738298 - RemoteScanRanges: 0 - RowsRead: 11384553 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 8 - ScannerThreadsTotalWallClockTime: 1350624748 - MaterializeTupleTime(*): 899918887 - ScannerThreadsSysTime: 6995000 - ScannerThreadsUserTime: 1199814000 - ScannerThreadsVoluntaryContextSwitches: 79 - TotalRawHdfsReadTime(*): 40596600 - TotalReadThroughput: 123718 - TotalTime: 364558500 Instance 544466204e0abc59:f772f133ae0a6906 (host=demeter-csmaz08-18.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 4:1/242.92 MB 19:1/242.74 MB 6:1/235.02 MB 1:1/242.45 MB 23:1/242.46 MB 2:1/242.86 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0023166023166024 - InactiveTotalTime: 0 - PeakMemoryUsage: 4659393240 - PerHostPeakMemUsage: 4726502104 - PrepareTime: 370425525 - RowsProduced: 1 - TotalCpuTime: 663121085188 - TotalNetworkReceiveTime: 631574514984 - TotalNetworkSendTime: 538664 - TotalStorageWaitTime: 123932773 - TotalTime: 646789750092 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 329578 - CompileTime: 65587890 - InactiveTotalTime: 0 - LoadTime: 149485 - ModuleFileSize: 2309684 - OptimizationTime: 510953893 - PrepareTime: 356776157 - TotalTime: 936014573 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 37872 - OverallThroughput: 270741 - PeakMemoryUsage: 16384 - SerializeBatchTime: 13527 - ThriftTransmitTime(*): 422465 - TotalTime: 59097 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 185515 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646788894424 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646175352378 - BuildPartitionTime: 7566372758 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2844258501 - GetNewBlockTime: 7074215 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 2028 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646430024143 - UnpinTime: 1104 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2259856884 - DeserializeRowBatchTimer: 4480363652 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630672634716 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99406 - SendersBlockedTimer: 203540176 - SendersBlockedTotalTimer(*): 1391149628 - TotalTime: 634017581319 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 4:1/242.92 MB 19:1/242.74 MB 6:1/235.02 MB 1:1/242.45 MB 23:1/242.46 MB 2:1/242.86 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.004640371229698376 - BytesRead: 80826256 - BytesReadDataNodeCache: 0 - BytesReadLocal: 80826256 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 80826256 - DecompressionTime: 260391622 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 6 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 237380312 - PerReadThreadRawHdfsThroughput: 1760785386 - RemoteScanRanges: 0 - RowsRead: 11505717 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 8 - ScannerThreadsTotalWallClockTime: 1276746998 - MaterializeTupleTime(*): 862571400 - ScannerThreadsSysTime: 13994000 - ScannerThreadsUserTime: 1128826000 - ScannerThreadsVoluntaryContextSwitches: 74 - TotalRawHdfsReadTime(*): 45903525 - TotalReadThroughput: 124924 - TotalTime: 240163108 Instance 544466204e0abc59:f772f133ae0a68f6 (host=demeter-csmaz11-18.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 12:1/242.90 MB 7:1/243.12 MB 9:1/242.81 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.998454404945904 - InactiveTotalTime: 0 - PeakMemoryUsage: 4545417720 - PerHostPeakMemUsage: 4612526584 - PrepareTime: 365955550 - RowsProduced: 1 - TotalCpuTime: 662685771233 - TotalNetworkReceiveTime: 631489109361 - TotalNetworkSendTime: 437161 - TotalStorageWaitTime: 57593817 - TotalTime: 646784146982 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 336415 - CompileTime: 114368032 - InactiveTotalTime: 0 - LoadTime: 136233 - ModuleFileSize: 2309684 - OptimizationTime: 505423438 - PrepareTime: 352417238 - TotalTime: 974851779 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 46826 - OverallThroughput: 287438 - PeakMemoryUsage: 16384 - SerializeBatchTime: 9883 - ThriftTransmitTime(*): 341687 - TotalTime: 55664 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 185813 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646783272463 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646166809938 - BuildPartitionTime: 7495478577 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2861260172 - GetNewBlockTime: 7556897 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1894 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646428905404 - UnpinTime: 2029 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2351974670 - DeserializeRowBatchTimer: 4391727159 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630680330264 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99405 - SendersBlockedTimer: 241146328 - SendersBlockedTotalTimer(*): 1654329892 - TotalTime: 634023822469 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 12:1/242.90 MB 7:1/243.12 MB 9:1/242.81 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:9 ExecOption: Codegen enabled: 0 out of 3 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 41767569 - BytesReadDataNodeCache: 0 - BytesReadLocal: 41767569 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 41767569 - DecompressionTime: 135004475 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 3 - NumScannerThreadsStarted: 3 - PeakMemoryUsage: 123404792 - PerReadThreadRawHdfsThroughput: 1724307194 - RemoteScanRanges: 0 - RowsRead: 5819481 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 3 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 660541332 - MaterializeTupleTime(*): 455256412 - ScannerThreadsSysTime: 4997000 - ScannerThreadsUserTime: 594908000 - ScannerThreadsVoluntaryContextSwitches: 35 - TotalRawHdfsReadTime(*): 24222812 - TotalReadThroughput: 0 - TotalTime: 248593954 Instance 544466204e0abc59:f772f133ae0a68f4 (host=demeter-csmau10-5.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 17:1/242.66 MB 2:1/242.26 MB 5:1/242.49 MB 19:1/242.98 MB 20:1/242.51 MB 4:1/234.85 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.998454404945904 - InactiveTotalTime: 0 - PeakMemoryUsage: 4653001360 - PerHostPeakMemUsage: 4720110224 - PrepareTime: 359946595 - RowsProduced: 1 - TotalCpuTime: 663410968915 - TotalNetworkReceiveTime: 631153116135 - TotalNetworkSendTime: 526854 - TotalStorageWaitTime: 184929996 - TotalTime: 646711256486 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 326643 - CompileTime: 136855129 - InactiveTotalTime: 0 - LoadTime: 159665 - ModuleFileSize: 2309684 - OptimizationTime: 510811056 - PrepareTime: 346401656 - TotalTime: 996798312 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 37599 - OverallThroughput: 268406 - PeakMemoryUsage: 16384 - SerializeBatchTime: 13376 - ThriftTransmitTime(*): 425536 - TotalTime: 59611 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 235193 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646710441364 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646102400429 - BuildPartitionTime: 7824248380 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2873671949 - GetNewBlockTime: 5271338 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 2146 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646361815821 - UnpinTime: 2051 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2351652510 - DeserializeRowBatchTimer: 4411093624 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630351363425 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99457 - SendersBlockedTimer: 261626115 - SendersBlockedTotalTimer(*): 1292820701 - TotalTime: 633691394125 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 17:1/242.66 MB 2:1/242.26 MB 5:1/242.49 MB 19:1/242.98 MB 20:1/242.51 MB 4:1/234.85 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 79582732 - BytesReadDataNodeCache: 0 - BytesReadLocal: 79582732 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 79582732 - DecompressionTime: 252024118 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 230988432 - PerReadThreadRawHdfsThroughput: 1653035346 - RemoteScanRanges: 0 - RowsRead: 11379455 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 9 - ScannerThreadsTotalWallClockTime: 1286743789 - MaterializeTupleTime(*): 823428921 - ScannerThreadsSysTime: 8996000 - ScannerThreadsUserTime: 1080832000 - ScannerThreadsVoluntaryContextSwitches: 68 - TotalRawHdfsReadTime(*): 48143394 - TotalReadThroughput: 0 - TotalTime: 244338949 Instance 544466204e0abc59:f772f133ae0a68bd (host=demeter-csmau10-7.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 11:1/242.52 MB 9:1/63.86 MB 12:1/233.79 MB 23:1/242.41 MB 13:1/163.52 MB 4:1/242.74 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.998454404945904 - InactiveTotalTime: 0 - PeakMemoryUsage: 4608459288 - PerHostPeakMemUsage: 4675568152 - PrepareTime: 360643465 - RowsProduced: 1 - TotalCpuTime: 663212414638 - TotalNetworkReceiveTime: 631053882872 - TotalNetworkSendTime: 519903 - TotalStorageWaitTime: 132109671 - TotalTime: 646697462796 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 331377 - CompileTime: 125890471 - InactiveTotalTime: 0 - LoadTime: 181331 - ModuleFileSize: 2309684 - OptimizationTime: 500820354 - PrepareTime: 347185579 - TotalTime: 976482797 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 38098 - OverallThroughput: 265260 - PeakMemoryUsage: 16384 - SerializeBatchTime: 13373 - ThriftTransmitTime(*): 419965 - TotalTime: 60318 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 215651 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646696541625 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646144495836 - BuildPartitionTime: 7974244202 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2793227541 - GetNewBlockTime: 3677834 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 2040 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646347384964 - UnpinTime: 2214 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2351001762 - DeserializeRowBatchTimer: 4692882651 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630160168594 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99473 - SendersBlockedTimer: 230523844 - SendersBlockedTotalTimer(*): 1432105899 - TotalTime: 633589332095 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 11:1/242.52 MB 9:1/63.86 MB 12:1/233.79 MB 23:1/242.41 MB 13:1/163.52 MB 4:1/242.74 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:18 ExecOption: Codegen enabled: 0 out of 6 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 66674466 - BytesReadDataNodeCache: 0 - BytesReadLocal: 66674466 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 66674466 - DecompressionTime: 215972500 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 6 - PeakMemoryUsage: 186446360 - PerReadThreadRawHdfsThroughput: 1866883830 - RemoteScanRanges: 0 - RowsRead: 9387575 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 6 - ScannerThreadsInvoluntaryContextSwitches: 2 - ScannerThreadsTotalWallClockTime: 928837184 - MaterializeTupleTime(*): 558870316 - ScannerThreadsSysTime: 31992000 - ScannerThreadsUserTime: 755883000 - ScannerThreadsVoluntaryContextSwitches: 77 - TotalRawHdfsReadTime(*): 35714309 - TotalReadThroughput: 0 - TotalTime: 188522640 Instance 544466204e0abc59:f772f133ae0a68f9 (host=demeter-csmau10-16.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 8:1/235.77 MB 0:1/242.81 MB 21:1/242.77 MB 19:1/242.80 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.998454404945904 - InactiveTotalTime: 0 - PeakMemoryUsage: 4607855864 - PerHostPeakMemUsage: 4674964728 - PrepareTime: 368780355 - RowsProduced: 1 - TotalCpuTime: 662575760467 - TotalNetworkReceiveTime: 631320931509 - TotalNetworkSendTime: 440780 - TotalStorageWaitTime: 101188301 - TotalTime: 646616205931 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 324365 - CompileTime: 117249485 - InactiveTotalTime: 0 - LoadTime: 168270 - ModuleFileSize: 2309684 - OptimizationTime: 498903418 - PrepareTime: 355248465 - TotalTime: 974142157 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 46760 - OverallThroughput: 268930 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12510 - ThriftTransmitTime(*): 342168 - TotalTime: 59495 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 248345 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646615346012 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646061475731 - BuildPartitionTime: 7706673754 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2786703473 - GetNewBlockTime: 6179045 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 1984 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646257786462 - UnpinTime: 1972 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2320525653 - DeserializeRowBatchTimer: 4516306287 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630472789981 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99436 - SendersBlockedTimer: 241373831 - SendersBlockedTotalTimer(*): 1889585842 - TotalTime: 633828512949 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 8:1/235.77 MB 0:1/242.81 MB 21:1/242.77 MB 19:1/242.80 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:12 ExecOption: Codegen enabled: 0 out of 4 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 55119014 - BytesReadDataNodeCache: 0 - BytesReadLocal: 55119014 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 55119014 - DecompressionTime: 160128960 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 4 - PeakMemoryUsage: 185842936 - PerReadThreadRawHdfsThroughput: 1855505011 - RemoteScanRanges: 0 - RowsRead: 7638627 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 703084272 - MaterializeTupleTime(*): 425795367 - ScannerThreadsSysTime: 8997000 - ScannerThreadsUserTime: 587909000 - ScannerThreadsVoluntaryContextSwitches: 49 - TotalRawHdfsReadTime(*): 29705667 - TotalReadThroughput: 0 - TotalTime: 182706026 Instance 544466204e0abc59:f772f133ae0a68e4 (host=demeter-csmaz10-2.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 6:1/242.99 MB 5:1/242.87 MB 7:1/243.04 MB 12:1/242.88 MB 14:1/242.89 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.998454404945904 - InactiveTotalTime: 0 - PeakMemoryUsage: 4661393320 - PerHostPeakMemUsage: 4728502184 - PrepareTime: 362571134 - RowsProduced: 1 - TotalCpuTime: 662654746732 - TotalNetworkReceiveTime: 631472268094 - TotalNetworkSendTime: 527713 - TotalStorageWaitTime: 107179190 - TotalTime: 646592975393 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 318277 - CompileTime: 136671093 - InactiveTotalTime: 0 - LoadTime: 129572 - ModuleFileSize: 2309684 - OptimizationTime: 502658889 - PrepareTime: 349126605 - TotalTime: 991140957 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 37038 - OverallThroughput: 293136 - PeakMemoryUsage: 16384 - SerializeBatchTime: 10457 - ThriftTransmitTime(*): 431978 - TotalTime: 54582 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 186147 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646592105386 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 646027093138 - BuildPartitionTime: 7551006351 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2789690244 - GetNewBlockTime: 28107405 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 2068 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646240920890 - UnpinTime: 2201 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2314128930 - DeserializeRowBatchTimer: 4505987320 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 630641157641 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99415 - SendersBlockedTimer: 254203801 - SendersBlockedTotalTimer(*): 1751272940 - TotalTime: 633959398287 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 6:1/242.99 MB 5:1/242.87 MB 7:1/243.04 MB 12:1/242.88 MB 14:1/242.89 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:15 ExecOption: Codegen enabled: 0 out of 5 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 69548251 - BytesReadDataNodeCache: 0 - BytesReadLocal: 69548251 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 69548251 - DecompressionTime: 255695997 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 5 - NumScannerThreadsStarted: 5 - PeakMemoryUsage: 239380392 - PerReadThreadRawHdfsThroughput: 1786096872 - RemoteScanRanges: 0 - RowsRead: 9704592 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 5 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 973565995 - MaterializeTupleTime(*): 590908629 - ScannerThreadsSysTime: 51990000 - ScannerThreadsUserTime: 803875000 - ScannerThreadsVoluntaryContextSwitches: 207 - TotalRawHdfsReadTime(*): 38938678 - TotalReadThroughput: 0 - TotalTime: 199679441 Instance 544466204e0abc59:f772f133ae0a68d0 (host=demeter-csmau08-9.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 3:1/242.55 MB 19:1/242.97 MB 20:1/242.55 MB 18:1/242.83 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.998454404945904 - InactiveTotalTime: 0 - PeakMemoryUsage: 4606284672 - PerHostPeakMemUsage: 4673393536 - PrepareTime: 377446724 - RowsProduced: 1 - TotalCpuTime: 661185573457 - TotalNetworkReceiveTime: 632716796494 - TotalNetworkSendTime: 478047 - TotalStorageWaitTime: 77074123 - TotalTime: 646587106982 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 417 - BlocksRecycled: 48 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 68719476736 - PeakMemoryUsage: 4487905280 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 331041 - CompileTime: 138964255 - InactiveTotalTime: 0 - LoadTime: 140010 - ModuleFileSize: 2309684 - OptimizationTime: 521308166 - PrepareTime: 363260696 - TotalTime: 1026205666 DataStreamSender (dst_id=5) - AsyncTotalTime: 0 - BytesSent: 16 - InactiveTotalTime: 0 - NetworkThroughput(*): 46164 - OverallThroughput: 278648 - PeakMemoryUsage: 16384 - SerializeBatchTime: 12114 - ThriftTransmitTime(*): 346583 - TotalTime: 57420 - UncompressedRowBatchSize: 16 AGGREGATION_NODE (id=3) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 224406 - GetResultsTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 0 - PeakMemoryUsage: 28672 - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646586214277 HASH_JOIN_NODE (id=2) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 645994015139 - BuildPartitionTime: 6567534783 - BuildRows: 63025520 - BuildRowsPartitioned: 63025520 - BuildTime: 2776251560 - GetNewBlockTime: 3789665 - HashBuckets: 134217728 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 4481179648 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 2046 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 646220851233 - UnpinTime: 2016 EXCHANGE_NODE (id=4) - AsyncTotalTime: 0 - BytesReceived: 990051001 - ConvertRowBatchTime: 2054055545 - DeserializeRowBatchTimer: 4520972787 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 631841134241 - PeakMemoryUsage: 0 - RowsReturned: 63025520 - RowsReturnedRate: 99262 - SendersBlockedTimer: 287336775 - SendersBlockedTotalTimer(*): 2013447934 - TotalTime: 634936296635 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 3:1/242.55 MB 19:1/242.97 MB 20:1/242.55 MB 18:1/242.83 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:12 ExecOption: Codegen enabled: 0 out of 4 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.0 - BytesRead: 55091184 - BytesReadDataNodeCache: 0 - BytesReadLocal: 55091184 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 55091184 - DecompressionTime: 180080926 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 4 - NumScannerThreadsStarted: 4 - PeakMemoryUsage: 184271744 - PerReadThreadRawHdfsThroughput: 2157162070 - RemoteScanRanges: 0 - RowsRead: 7674394 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 1 - ScannerThreadsTotalWallClockTime: 736743250 - MaterializeTupleTime(*): 464044422 - ScannerThreadsSysTime: 24995000 - ScannerThreadsUserTime: 629903000 - ScannerThreadsVoluntaryContextSwitches: 45 - TotalRawHdfsReadTime(*): 25538732 - TotalReadThroughput: 0 - TotalTime: 212207143 Fragment F01 Instance 544466204e0abc59:f772f133ae0a693a (host=demeter-csmau08-3.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 20:1/154.80 MB 18:1/125.47 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0023364485981308 - InactiveTotalTime: 0 - PeakMemoryUsage: 209914880 - PerHostPeakMemUsage: 3163915312 - PrepareTime: 148344 - RowsProduced: 6100806 - TotalCpuTime: 652655985156 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 633025705395 - TotalStorageWaitTime: 46494303 - TotalTime: 641908590622 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 8500901850 - InactiveTotalTime: 0 - NetworkThroughput(*): 4597529 - OverallThroughput: 13247361 - PeakMemoryUsage: 491040 - SerializeBatchTime: 1869789733 - ThriftTransmitTime(*): 1849015268635 - TotalTime: 641705282717 - UncompressedRowBatchSize: 18121538700 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 20:1/154.80 MB 18:1/125.47 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0023364485981308 - BytesRead: 272147701 - BytesReadDataNodeCache: 0 - BytesReadLocal: 272147701 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 272147701 - DecompressionTime: 176981549 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 209328608 - PerReadThreadRawHdfsThroughput: 1012927140 - RemoteScanRanges: 0 - RowsRead: 67894700 - RowsReturned: 6100806 - RowsReturnedRate: 38684464 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 665 - ScannerThreadsTotalWallClockTime: 643492547020 - MaterializeTupleTime(*): 643060713067 - ScannerThreadsSysTime: 232964000 - ScannerThreadsUserTime: 17520335000 - ScannerThreadsVoluntaryContextSwitches: 5795 - TotalRawHdfsReadTime(*): 268674508 - TotalReadThroughput: 423906 - TotalTime: 157706873 Instance 544466204e0abc59:f772f133ae0a6933 (host=demeter-csmau10-12.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 20:1/123.96 MB 21:1/129.46 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.272727272727273 - InactiveTotalTime: 0 - PeakMemoryUsage: 108921904 - PerHostPeakMemUsage: 1931468376 - PrepareTime: 162143 - RowsProduced: 6100806 - TotalCpuTime: 45685035200 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 16523549253 - TotalStorageWaitTime: 44490782 - TotalTime: 27051122920 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 8611403310 - InactiveTotalTime: 0 - NetworkThroughput(*): 13296252 - OverallThroughput: 320676656 - PeakMemoryUsage: 491040 - SerializeBatchTime: 1530624045 - ThriftTransmitTime(*): 647656409045 - TotalTime: 26853851527 - UncompressedRowBatchSize: 18121538700 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 20:1/123.96 MB 21:1/129.46 MB Hdfs Read Thread Concurrency Bucket: 0:98.15% 1:1.852% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.018518518518518517 - AverageScannerThreadConcurrency: 1.2962962962962963 - BytesRead: 248049599 - BytesReadDataNodeCache: 0 - BytesReadLocal: 248049599 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 248049599 - DecompressionTime: 147921238 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 108389904 - PerReadThreadRawHdfsThroughput: 1394715882 - RemoteScanRanges: 0 - RowsRead: 61874778 - RowsReturned: 6100806 - RowsReturnedRate: 39369469 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 1732 - ScannerThreadsTotalWallClockTime: 35101163138 - MaterializeTupleTime(*): 34849882444 - ScannerThreadsSysTime: 496923000 - ScannerThreadsUserTime: 12359120000 - ScannerThreadsVoluntaryContextSwitches: 5626 - TotalRawHdfsReadTime(*): 177849555 - TotalReadThroughput: 9019985 - TotalTime: 154962872 Instance 544466204e0abc59:f772f133ae0a693f (host=demeter-csmaz11-6.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 11:1/145.11 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9814814814814814 - InactiveTotalTime: 0 - PeakMemoryUsage: 50610872 - PerHostPeakMemUsage: 1930989112 - PrepareTime: 134882 - RowsProduced: 6277541 - TotalCpuTime: 37320460384 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 16312665525 - TotalStorageWaitTime: 21482292 - TotalTime: 26955823445 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 8721701190 - InactiveTotalTime: 0 - NetworkThroughput(*): 13835059 - OverallThroughput: 326117066 - PeakMemoryUsage: 491040 - SerializeBatchTime: 1536060356 - ThriftTransmitTime(*): 630405753954 - TotalTime: 26744080840 - UncompressedRowBatchSize: 18646503930 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 11:1/145.11 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.9814814814814815 - BytesRead: 140977105 - BytesReadDataNodeCache: 0 - BytesReadLocal: 140977105 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 140977105 - DecompressionTime: 67811639 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50078872 - PerReadThreadRawHdfsThroughput: 1258699382 - RemoteScanRanges: 0 - RowsRead: 35172064 - RowsReturned: 6277541 - RowsReturnedRate: 37752353 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 1272 - ScannerThreadsTotalWallClockTime: 26590897694 - MaterializeTupleTime(*): 26463239628 - ScannerThreadsSysTime: 263959000 - ScannerThreadsUserTime: 8452714000 - ScannerThreadsVoluntaryContextSwitches: 5851 - TotalRawHdfsReadTime(*): 112002204 - TotalReadThroughput: 5221374 - TotalTime: 166282110 Instance 544466204e0abc59:f772f133ae0a6957 (host=demeter-csmaz08-15.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 7:1/180.61 MB 5:1/120.04 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.3962264150943398 - InactiveTotalTime: 0 - PeakMemoryUsage: 108807192 - PerHostPeakMemUsage: 1921966544 - PrepareTime: 175309 - RowsProduced: 6100806 - TotalCpuTime: 50323152006 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 13476098462 - TotalStorageWaitTime: 54060955 - TotalTime: 26550257038 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 8410298850 - InactiveTotalTime: 0 - NetworkThroughput(*): 21741987 - OverallThroughput: 319321346 - PeakMemoryUsage: 491040 - SerializeBatchTime: 1594060644 - ThriftTransmitTime(*): 386822911552 - TotalTime: 26338041401 - UncompressedRowBatchSize: 18121538700 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 7:1/180.61 MB 5:1/120.04 MB Hdfs Read Thread Concurrency Bucket: 0:98.11% 1:1.887% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.018867924528301886 - AverageScannerThreadConcurrency: 1.3962264150943395 - BytesRead: 289032983 - BytesReadDataNodeCache: 0 - BytesReadLocal: 289032983 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 289032983 - DecompressionTime: 158002375 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 108209648 - PerReadThreadRawHdfsThroughput: 1509779317 - RemoteScanRanges: 0 - RowsRead: 72112173 - RowsReturned: 6100806 - RowsReturnedRate: 36039815 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 1435 - ScannerThreadsTotalWallClockTime: 37202751117 - MaterializeTupleTime(*): 36929015985 - ScannerThreadsSysTime: 514921000 - ScannerThreadsUserTime: 15291675000 - ScannerThreadsVoluntaryContextSwitches: 5651 - TotalRawHdfsReadTime(*): 191440550 - TotalReadThroughput: 10906905 - TotalTime: 169279614 Instance 544466204e0abc59:f772f133ae0a6919 (host=demeter-csmaz10-15.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 3:1/123.96 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9811320754716981 - InactiveTotalTime: 0 - PeakMemoryUsage: 49865312 - PerHostPeakMemUsage: 1922953840 - PrepareTime: 131822 - RowsProduced: 6100806 - TotalCpuTime: 38628671200 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 14178108885 - TotalStorageWaitTime: 22989387 - TotalTime: 26537308716 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 8251210980 - InactiveTotalTime: 0 - NetworkThroughput(*): 21767916 - OverallThroughput: 313568361 - PeakMemoryUsage: 491040 - SerializeBatchTime: 1572811961 - ThriftTransmitTime(*): 379053772495 - TotalTime: 26313914299 - UncompressedRowBatchSize: 18121538700 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 3:1/123.96 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.9811320754716981 - BytesRead: 122389737 - BytesReadDataNodeCache: 0 - BytesReadLocal: 122389737 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 122389737 - DecompressionTime: 66438060 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 49333312 - PerReadThreadRawHdfsThroughput: 1126178418 - RemoteScanRanges: 0 - RowsRead: 30529291 - RowsReturned: 6100806 - RowsReturnedRate: 34999414 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 1777 - ScannerThreadsTotalWallClockTime: 26186192413 - MaterializeTupleTime(*): 26058753711 - ScannerThreadsSysTime: 817875000 - ScannerThreadsUserTime: 7950791000 - ScannerThreadsVoluntaryContextSwitches: 5674 - TotalRawHdfsReadTime(*): 108677040 - TotalReadThroughput: 4618480 - TotalTime: 174311660 Instance 544466204e0abc59:f772f133ae0a694f (host=demeter-csmau08-10.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 16:1/156.56 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9807692307692308 - InactiveTotalTime: 0 - PeakMemoryUsage: 50462920 - PerHostPeakMemUsage: 1895746368 - PrepareTime: 356385 - RowsProduced: 6100806 - TotalCpuTime: 37095746894 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 13932567941 - TotalStorageWaitTime: 25945091 - TotalTime: 25709197306 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 8636584950 - InactiveTotalTime: 0 - NetworkThroughput(*): 21405554 - OverallThroughput: 338560933 - PeakMemoryUsage: 491040 - SerializeBatchTime: 1722011269 - ThriftTransmitTime(*): 403474013109 - TotalTime: 25509691479 - UncompressedRowBatchSize: 18121538700 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 16:1/156.56 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.9807692307692307 - BytesRead: 150539137 - BytesReadDataNodeCache: 0 - BytesReadLocal: 150539137 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 150539137 - DecompressionTime: 78948170 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 49910920 - PerReadThreadRawHdfsThroughput: 1057760103 - RemoteScanRanges: 0 - RowsRead: 37561286 - RowsReturned: 6100806 - RowsReturnedRate: 40695397 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 1283 - ScannerThreadsTotalWallClockTime: 25245161606 - MaterializeTupleTime(*): 25094985634 - ScannerThreadsSysTime: 342947000 - ScannerThreadsUserTime: 9789511000 - ScannerThreadsVoluntaryContextSwitches: 5343 - TotalRawHdfsReadTime(*): 142318789 - TotalReadThroughput: 5789966 - TotalTime: 149913906 Instance 544466204e0abc59:f772f133ae0a6934 (host=demeter-csmau08-5.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 11:1/153.21 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9803921568627452 - InactiveTotalTime: 0 - PeakMemoryUsage: 50414232 - PerHostPeakMemUsage: 1887046832 - PrepareTime: 131942 - RowsProduced: 6100806 - TotalCpuTime: 34996480174 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 15905692700 - TotalStorageWaitTime: 21086223 - TotalTime: 25637185971 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 8490049470 - InactiveTotalTime: 0 - NetworkThroughput(*): 13885239 - OverallThroughput: 333563144 - PeakMemoryUsage: 491040 - SerializeBatchTime: 1538521858 - ThriftTransmitTime(*): 611444207594 - TotalTime: 25452600561 - UncompressedRowBatchSize: 18121538700 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 11:1/153.21 MB Hdfs Read Thread Concurrency Bucket: 0:98.04% 1:1.961% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0196078431372549 - AverageScannerThreadConcurrency: 0.9803921568627451 - BytesRead: 149037492 - BytesReadDataNodeCache: 0 - BytesReadLocal: 149037492 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 149037492 - DecompressionTime: 74948316 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 49816688 - PerReadThreadRawHdfsThroughput: 1189144659 - RemoteScanRanges: 0 - RowsRead: 37185862 - RowsReturned: 6100806 - RowsReturnedRate: 42468772 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 1659 - ScannerThreadsTotalWallClockTime: 25194860208 - MaterializeTupleTime(*): 25058291561 - ScannerThreadsSysTime: 293955000 - ScannerThreadsUserTime: 9278589000 - ScannerThreadsVoluntaryContextSwitches: 5238 - TotalRawHdfsReadTime(*): 125331675 - TotalReadThroughput: 5844607 - TotalTime: 143653927 Instance 544466204e0abc59:f772f133ae0a6947 (host=demeter-csmaz10-12.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 21:1/126.20 MB 23:1/123.35 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.372549019607843 - InactiveTotalTime: 0 - PeakMemoryUsage: 100585784 - PerHostPeakMemUsage: 1873879264 - PrepareTime: 176543 - RowsProduced: 6100805 - TotalCpuTime: 46853827650 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 13292312570 - TotalStorageWaitTime: 51395579 - TotalTime: 25291402534 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 8757700020 - InactiveTotalTime: 0 - NetworkThroughput(*): 22163741 - OverallThroughput: 349214234 - PeakMemoryUsage: 491040 - SerializeBatchTime: 1727531306 - ThriftTransmitTime(*): 395136356737 - TotalTime: 25078301982 - UncompressedRowBatchSize: 18121535730 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 21:1/126.20 MB 23:1/123.35 MB Hdfs Read Thread Concurrency Bucket: 0:98.04% 1:1.961% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0196078431372549 - AverageScannerThreadConcurrency: 1.3725490196078431 - BytesRead: 243024462 - BytesReadDataNodeCache: 0 - BytesReadLocal: 243024462 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 243024462 - DecompressionTime: 174865007 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 99988240 - PerReadThreadRawHdfsThroughput: 1293600436 - RemoteScanRanges: 0 - RowsRead: 60619509 - RowsReturned: 6100805 - RowsReturnedRate: 35891568 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 1904 - ScannerThreadsTotalWallClockTime: 34805910533 - MaterializeTupleTime(*): 34521220234 - ScannerThreadsSysTime: 543916000 - ScannerThreadsUserTime: 15095704000 - ScannerThreadsVoluntaryContextSwitches: 5440 - TotalRawHdfsReadTime(*): 187866713 - TotalReadThroughput: 9530371 - TotalTime: 169978780 Instance 544466204e0abc59:f772f133ae0a6937 (host=demeter-csmaz11-20.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 18:1/156.17 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9803921568627452 - InactiveTotalTime: 0 - PeakMemoryUsage: 50414240 - PerHostPeakMemUsage: 1868179064 - PrepareTime: 168616 - RowsProduced: 6111824 - TotalCpuTime: 37615453572 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 12207866136 - TotalStorageWaitTime: 398062260 - TotalTime: 25259475566 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 9506263770 - InactiveTotalTime: 0 - NetworkThroughput(*): 25691238 - OverallThroughput: 388783550 - PeakMemoryUsage: 491040 - SerializeBatchTime: 1807579119 - ThriftTransmitTime(*): 370019679764 - TotalTime: 24451301405 - UncompressedRowBatchSize: 18154266480 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 18:1/156.17 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.9803921568627451 - BytesRead: 150030919 - BytesReadDataNodeCache: 0 - BytesReadLocal: 150030919 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 150030919 - DecompressionTime: 80012404 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 49816696 - PerReadThreadRawHdfsThroughput: 149874782 - RemoteScanRanges: 0 - RowsRead: 37432559 - RowsReturned: 6111824 - RowsReturnedRate: 7969841 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 2205 - ScannerThreadsTotalWallClockTime: 24870821893 - MaterializeTupleTime(*): 24350569599 - ScannerThreadsSysTime: 636903000 - ScannerThreadsUserTime: 9848502000 - ScannerThreadsVoluntaryContextSwitches: 5177 - TotalRawHdfsReadTime(*): 1001041782 - TotalReadThroughput: 5883565 - TotalTime: 766868912 Instance 544466204e0abc59:f772f133ae0a694c (host=demeter-csmau10-1.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 2:1/150.31 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.98 - InactiveTotalTime: 0 - PeakMemoryUsage: 50479784 - PerHostPeakMemUsage: 1859995784 - PrepareTime: 144332 - RowsProduced: 6100806 - TotalCpuTime: 35776939490 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 14033990920 - TotalStorageWaitTime: 22179443 - TotalTime: 25026655461 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 8603298540 - InactiveTotalTime: 0 - NetworkThroughput(*): 18624245 - OverallThroughput: 347985695 - PeakMemoryUsage: 491040 - SerializeBatchTime: 1690570631 - ThriftTransmitTime(*): 461940793273 - TotalTime: 24723138509 - UncompressedRowBatchSize: 18121538700 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 2:1/150.31 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.98 - BytesRead: 148276146 - BytesReadDataNodeCache: 0 - BytesReadLocal: 148276146 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 148276146 - DecompressionTime: 80359701 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 49882240 - PerReadThreadRawHdfsThroughput: 1192641628 - RemoteScanRanges: 0 - RowsRead: 36995671 - RowsReturned: 6100806 - RowsReturnedRate: 23724063 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3281 - ScannerThreadsTotalWallClockTime: 24700721087 - MaterializeTupleTime(*): 24548418391 - ScannerThreadsSysTime: 1362792000 - ScannerThreadsUserTime: 9494556000 - ScannerThreadsVoluntaryContextSwitches: 4337 - TotalRawHdfsReadTime(*): 124325818 - TotalReadThroughput: 5931045 - TotalTime: 257156869 Instance 544466204e0abc59:f772f133ae0a6954 (host=demeter-csmaz08-11.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 6:1/131.85 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 49803856 - PerHostPeakMemUsage: 1006095624 - PrepareTime: 148503 - RowsProduced: 1829708 - TotalCpuTime: 18481675648 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 6147502509 - TotalStorageWaitTime: 23455757 - TotalTime: 12292353798 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 2615177160 - InactiveTotalTime: 0 - NetworkThroughput(*): 20249825 - OverallThroughput: 253830938 - PeakMemoryUsage: 491040 - SerializeBatchTime: 536553460 - ThriftTransmitTime(*): 129145667076 - TotalTime: 10302830610 - UncompressedRowBatchSize: 5434876080 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 6:1/131.85 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 128144598 - BytesReadDataNodeCache: 0 - BytesReadLocal: 128144598 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 128144598 - DecompressionTime: 51191225 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 49271856 - PerReadThreadRawHdfsThroughput: 1535017225 - RemoteScanRanges: 0 - RowsRead: 31966916 - RowsReturned: 1829708 - RowsReturnedRate: 929960 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 271 - ScannerThreadsTotalWallClockTime: 12305975147 - MaterializeTupleTime(*): 12214443585 - ScannerThreadsSysTime: 51992000 - ScannerThreadsUserTime: 4651292000 - ScannerThreadsVoluntaryContextSwitches: 1654 - TotalRawHdfsReadTime(*): 83480886 - TotalReadThroughput: 10251567 - TotalTime: 1967511764 Instance 544466204e0abc59:f772f133ae0a6916 (host=demeter-csmaz08-5.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 5:1/153.21 MB 2:1/125.79 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.8666666666666667 - InactiveTotalTime: 0 - PeakMemoryUsage: 98572832 - PerHostPeakMemUsage: 599344392 - PrepareTime: 158116 - RowsProduced: 0 - TotalCpuTime: 20480646088 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 35662 - TotalStorageWaitTime: 46240745 - TotalTime: 7174858606 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 158996 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 5:1/153.21 MB 2:1/125.79 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.8666666666666667 - BytesRead: 272481714 - BytesReadDataNodeCache: 0 - BytesReadLocal: 272481714 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 272481714 - DecompressionTime: 80009443 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 98073600 - PerReadThreadRawHdfsThroughput: 1885939174 - RemoteScanRanges: 0 - RowsRead: 67978254 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 22 - ScannerThreadsTotalWallClockTime: 13319041968 - MaterializeTupleTime(*): 13169524578 - ScannerThreadsSysTime: 18997000 - ScannerThreadsUserTime: 13262983000 - ScannerThreadsVoluntaryContextSwitches: 13 - TotalRawHdfsReadTime(*): 144480648 - TotalReadThroughput: 36330895 - TotalTime: 7163673465 Instance 544466204e0abc59:f772f133ae0a693b (host=demeter-csmau08-9.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 7:1/153.48 MB 5:1/129.08 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.9285714285714284 - InactiveTotalTime: 0 - PeakMemoryUsage: 98916928 - PerHostPeakMemUsage: 599218448 - PrepareTime: 154963 - RowsProduced: 0 - TotalCpuTime: 20699741301 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 35432 - TotalStorageWaitTime: 46511983 - TotalTime: 7174639991 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 239992 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 7:1/153.48 MB 5:1/129.08 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.9285714285714286 - BytesRead: 273988058 - BytesReadDataNodeCache: 0 - BytesReadLocal: 273988058 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 273988058 - DecompressionTime: 79063679 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 98417696 - PerReadThreadRawHdfsThroughput: 2015468082 - RemoteScanRanges: 0 - RowsRead: 68355202 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 18 - ScannerThreadsTotalWallClockTime: 13525297727 - MaterializeTupleTime(*): 13374651233 - ScannerThreadsSysTime: 1998000 - ScannerThreadsUserTime: 13484949000 - ScannerThreadsVoluntaryContextSwitches: 15 - TotalRawHdfsReadTime(*): 135942643 - TotalReadThroughput: 39141151 - TotalTime: 7161923129 Instance 544466204e0abc59:f772f133ae0a6938 (host=demeter-csmau08-7.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 23:1/151.62 MB 0:1/152.61 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.875 - InactiveTotalTime: 0 - PeakMemoryUsage: 99592856 - PerHostPeakMemUsage: 379422192 - PrepareTime: 163503 - RowsProduced: 0 - TotalCpuTime: 11290870499 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 33749 - TotalStorageWaitTime: 309828906 - TotalTime: 4038704428 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 175956 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 23:1/151.62 MB 0:1/152.61 MB Hdfs Read Thread Concurrency Bucket: 0:62.5% 1:25% 2:12.5% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.5 - AverageScannerThreadConcurrency: 1.875 - BytesRead: 296400925 - BytesReadDataNodeCache: 0 - BytesReadLocal: 296400925 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 296400925 - DecompressionTime: 83317222 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 99093624 - PerReadThreadRawHdfsThroughput: 107958026 - RemoteScanRanges: 0 - RowsRead: 73952324 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 15 - ScannerThreadsTotalWallClockTime: 7529523173 - MaterializeTupleTime(*): 7109429779 - ScannerThreadsSysTime: 10998000 - ScannerThreadsUserTime: 7212902000 - ScannerThreadsVoluntaryContextSwitches: 13 - TotalRawHdfsReadTime(*): 2745520043 - TotalReadThroughput: 74100231 - TotalTime: 4026650165 Instance 544466204e0abc59:f772f133ae0a6942 (host=demeter-csmaz11-17.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 13:1/156.27 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 134489 - RowsProduced: 0 - TotalCpuTime: 7941070393 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 24200 - TotalStorageWaitTime: 20873436 - TotalTime: 3973402400 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 184783 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 13:1/156.27 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 150199348 - BytesReadDataNodeCache: 0 - BytesReadLocal: 150199348 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 150199348 - DecompressionTime: 61390916 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 1914069600 - RemoteScanRanges: 0 - RowsRead: 37476380 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 6 - ScannerThreadsTotalWallClockTime: 3961383462 - MaterializeTupleTime(*): 3867913666 - ScannerThreadsSysTime: 3999000 - ScannerThreadsUserTime: 3938401000 - ScannerThreadsVoluntaryContextSwitches: 6 - TotalRawHdfsReadTime(*): 78471205 - TotalReadThroughput: 37549837 - TotalTime: 3959984420 Instance 544466204e0abc59:f772f133ae0a6923 (host=demeter-csmau08-16.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 8:1/155.71 MB 7:1/149.69 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.857142857142857 - InactiveTotalTime: 0 - PeakMemoryUsage: 100780840 - PerHostPeakMemUsage: 380186456 - PrepareTime: 148231 - RowsProduced: 0 - TotalCpuTime: 11087044428 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 37524 - TotalStorageWaitTime: 45408930 - TotalTime: 3858137354 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 157789 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 8:1/155.71 MB 7:1/149.69 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.8571428571428572 - BytesRead: 294107467 - BytesReadDataNodeCache: 0 - BytesReadLocal: 294107467 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 294107467 - DecompressionTime: 88185200 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 100281608 - PerReadThreadRawHdfsThroughput: 1995790915 - RemoteScanRanges: 0 - RowsRead: 73380456 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 15 - ScannerThreadsTotalWallClockTime: 7222611193 - MaterializeTupleTime(*): 7059728181 - ScannerThreadsSysTime: 15997000 - ScannerThreadsUserTime: 7164910000 - ScannerThreadsVoluntaryContextSwitches: 16 - TotalRawHdfsReadTime(*): 147363867 - TotalReadThroughput: 83693582 - TotalTime: 3847315372 Instance 544466204e0abc59:f772f133ae0a6920 (host=demeter-csmau08-15.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 6:1/175.32 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 138055 - RowsProduced: 0 - TotalCpuTime: 7511341305 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 19580 - TotalStorageWaitTime: 19933965 - TotalTime: 3750841585 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 165058 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 6:1/175.32 MB Hdfs Read Thread Concurrency Bucket: 0:85.71% 1:14.29% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.14285714285714285 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 171266478 - BytesReadDataNodeCache: 0 - BytesReadLocal: 171266478 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 171266478 - DecompressionTime: 70128031 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 2079177236 - RemoteScanRanges: 0 - RowsRead: 42737697 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 6 - ScannerThreadsTotalWallClockTime: 3741220503 - MaterializeTupleTime(*): 3639964916 - ScannerThreadsSysTime: 9998000 - ScannerThreadsUserTime: 3713435000 - ScannerThreadsVoluntaryContextSwitches: 6 - TotalRawHdfsReadTime(*): 82372236 - TotalReadThroughput: 48933279 - TotalTime: 3739968656 Instance 544466204e0abc59:f772f133ae0a6911 (host=demeter-csmau08-20.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 6:1/154.55 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266824 - PerHostPeakMemUsage: 339192072 - PrepareTime: 140059 - RowsProduced: 0 - TotalCpuTime: 7499726150 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 21915 - TotalStorageWaitTime: 21471121 - TotalTime: 3747003896 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 180648 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 6:1/154.55 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 149523728 - BytesReadDataNodeCache: 0 - BytesReadLocal: 149523728 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 149523728 - DecompressionTime: 61390376 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767592 - PerReadThreadRawHdfsThroughput: 2114473394 - RemoteScanRanges: 0 - RowsRead: 37307607 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3736319371 - MaterializeTupleTime(*): 3643692260 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 3716435000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 70714405 - TotalReadThroughput: 37380932 - TotalTime: 3735663651 Instance 544466204e0abc59:f772f133ae0a6932 (host=demeter-csmaz08-14.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 12:1/127.19 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250448 - PerHostPeakMemUsage: 346625928 - PrepareTime: 153062 - RowsProduced: 0 - TotalCpuTime: 7449643117 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 19849 - TotalStorageWaitTime: 20728965 - TotalTime: 3727768081 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 163899 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 12:1/127.19 MB Hdfs Read Thread Concurrency Bucket: 0:87.5% 1:12.5% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.125 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 124616507 - BytesReadDataNodeCache: 0 - BytesReadLocal: 124616507 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 124616507 - DecompressionTime: 51008408 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751216 - PerReadThreadRawHdfsThroughput: 1852825604 - RemoteScanRanges: 0 - RowsRead: 31085058 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 6 - ScannerThreadsTotalWallClockTime: 3718016298 - MaterializeTupleTime(*): 3637713093 - ScannerThreadsSysTime: 15997000 - ScannerThreadsUserTime: 3683440000 - ScannerThreadsVoluntaryContextSwitches: 6 - TotalRawHdfsReadTime(*): 67257548 - TotalReadThroughput: 31154126 - TotalTime: 3717527278 Instance 544466204e0abc59:f772f133ae0a6939 (host=demeter-csmaz11-14.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 5:1/174.60 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.875 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 153406 - RowsProduced: 0 - TotalCpuTime: 7221043886 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 22418 - TotalStorageWaitTime: 25971076 - TotalTime: 3617097664 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 164879 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 5:1/174.60 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 171132489 - BytesReadDataNodeCache: 0 - BytesReadLocal: 171132489 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 171132489 - DecompressionTime: 73583653 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 2029902593 - RemoteScanRanges: 0 - RowsRead: 42705641 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 3606998726 - MaterializeTupleTime(*): 3496451921 - ScannerThreadsSysTime: 10998000 - ScannerThreadsUserTime: 3571457000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 84305764 - TotalReadThroughput: 47970026 - TotalTime: 3605973215 Instance 544466204e0abc59:f772f133ae0a6921 (host=demeter-csmau10-20.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 18:1/153.79 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266824 - PerHostPeakMemUsage: 339192072 - PrepareTime: 187999 - RowsProduced: 0 - TotalCpuTime: 7163860725 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 20718 - TotalStorageWaitTime: 22833910 - TotalTime: 3584599565 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 166049 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 18:1/153.79 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 150200731 - BytesReadDataNodeCache: 0 - BytesReadLocal: 150200731 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 150200731 - DecompressionTime: 66426983 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767592 - PerReadThreadRawHdfsThroughput: 1975193154 - RemoteScanRanges: 0 - RowsRead: 37476378 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3573931633 - MaterializeTupleTime(*): 3475106860 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 3553459000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 76043566 - TotalReadThroughput: 42914494 - TotalTime: 3572976832 Instance 544466204e0abc59:f772f133ae0a693e (host=demeter-csmaz10-17.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 12:1/156.67 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 144169 - RowsProduced: 0 - TotalCpuTime: 7153186576 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 22587 - TotalStorageWaitTime: 21073887 - TotalTime: 3580984808 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 169460 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 12:1/156.67 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 151690752 - BytesReadDataNodeCache: 0 - BytesReadLocal: 151690752 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 151690752 - DecompressionTime: 68558463 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 1945654343 - RemoteScanRanges: 0 - RowsRead: 37848855 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3569586195 - MaterializeTupleTime(*): 3469464121 - ScannerThreadsSysTime: 7998000 - ScannerThreadsUserTime: 3542461000 - ScannerThreadsVoluntaryContextSwitches: 6 - TotalRawHdfsReadTime(*): 77963875 - TotalReadThroughput: 43340214 - TotalTime: 3568686307 Instance 544466204e0abc59:f772f133ae0a6953 (host=demeter-csmaz10-16.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 5:1/153.12 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.875 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250448 - PerHostPeakMemUsage: 339175696 - PrepareTime: 191504 - RowsProduced: 0 - TotalCpuTime: 7127936634 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 24801 - TotalStorageWaitTime: 21229008 - TotalTime: 3566173563 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 161921 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 5:1/153.12 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 149050737 - BytesReadDataNodeCache: 0 - BytesReadLocal: 149050737 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 149050737 - DecompressionTime: 69971410 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751216 - PerReadThreadRawHdfsThroughput: 1933272091 - RemoteScanRanges: 0 - RowsRead: 37189814 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 3553708529 - MaterializeTupleTime(*): 3452476887 - ScannerThreadsSysTime: 9998000 - ScannerThreadsUserTime: 3524464000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 77097651 - TotalReadThroughput: 42585924 - TotalTime: 3552761163 Instance 544466204e0abc59:f772f133ae0a6951 (host=demeter-csmau10-8.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 8:1/153.94 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266824 - PerHostPeakMemUsage: 339192072 - PrepareTime: 190496 - RowsProduced: 0 - TotalCpuTime: 7059112727 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 37242 - TotalStorageWaitTime: 20963874 - TotalTime: 3526701234 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 141471 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 8:1/153.94 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 149815142 - BytesReadDataNodeCache: 0 - BytesReadLocal: 149815142 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 149815142 - DecompressionTime: 60935000 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767592 - PerReadThreadRawHdfsThroughput: 1545735161 - RemoteScanRanges: 0 - RowsRead: 37380425 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3515323012 - MaterializeTupleTime(*): 3424301242 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 3496468000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 96921611 - TotalReadThroughput: 42804326 - TotalTime: 3514908945 Instance 544466204e0abc59:f772f133ae0a6924 (host=demeter-csmau08-18.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 3:1/152.78 MB 1:1/120.17 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.857142857142857 - InactiveTotalTime: 0 - PeakMemoryUsage: 101555080 - PerHostPeakMemUsage: 127204232 - PrepareTime: 143462 - RowsProduced: 0 - TotalCpuTime: 9763357758 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 19207 - TotalStorageWaitTime: 82854906 - TotalTime: 3506531833 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 152386 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 3:1/152.78 MB 1:1/120.17 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.8571428571428572 - BytesRead: 263753383 - BytesReadDataNodeCache: 0 - BytesReadLocal: 263753383 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 263753383 - DecompressionTime: 87486810 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 101055848 - PerReadThreadRawHdfsThroughput: 1929854913 - RemoteScanRanges: 0 - RowsRead: 65798159 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 35 - ScannerThreadsTotalWallClockTime: 6307488737 - MaterializeTupleTime(*): 6119388611 - ScannerThreadsSysTime: 2998000 - ScannerThreadsUserTime: 6219054000 - ScannerThreadsVoluntaryContextSwitches: 21 - TotalRawHdfsReadTime(*): 136670058 - TotalReadThroughput: 75358109 - TotalTime: 3490547244 Instance 544466204e0abc59:f772f133ae0a692f (host=demeter-csmau10-3.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 20:1/156.44 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 201354 - RowsProduced: 0 - TotalCpuTime: 7046070499 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 64846 - TotalStorageWaitTime: 20170114 - TotalTime: 3505028767 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 156158 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 20:1/156.44 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 151494285 - BytesReadDataNodeCache: 0 - BytesReadLocal: 151494285 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 151494285 - DecompressionTime: 59092801 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 1518137953 - RemoteScanRanges: 0 - RowsRead: 37798849 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3493017359 - MaterializeTupleTime(*): 3404054993 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 3475471000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 99789538 - TotalReadThroughput: 43284081 - TotalTime: 3492548839 Instance 544466204e0abc59:f772f133ae0a6956 (host=demeter-csmaz11-11.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 11:1/126.66 MB 12:1/132.84 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.857142857142857 - InactiveTotalTime: 0 - PeakMemoryUsage: 101522312 - PerHostPeakMemUsage: 381321192 - PrepareTime: 154917 - RowsProduced: 0 - TotalCpuTime: 10007186717 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 35046 - TotalStorageWaitTime: 45022411 - TotalTime: 3469593175 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 182824 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 11:1/126.66 MB 12:1/132.84 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.8571428571428572 - BytesRead: 251211130 - BytesReadDataNodeCache: 0 - BytesReadLocal: 251211130 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 251211130 - DecompressionTime: 80641175 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 101023080 - PerReadThreadRawHdfsThroughput: 2073383822 - RemoteScanRanges: 0 - RowsRead: 62666049 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 16 - ScannerThreadsTotalWallClockTime: 6548358216 - MaterializeTupleTime(*): 6398789494 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 6505010000 - ScannerThreadsVoluntaryContextSwitches: 12 - TotalRawHdfsReadTime(*): 121159974 - TotalReadThroughput: 71774608 - TotalTime: 3458373580 Instance 544466204e0abc59:f772f133ae0a6929 (host=demeter-csmaz11-5.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 0:1/151.97 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 183175 - RowsProduced: 0 - TotalCpuTime: 6917773250 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 24472 - TotalStorageWaitTime: 21844994 - TotalTime: 3460149172 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 171076 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 0:1/151.97 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 149037709 - BytesReadDataNodeCache: 0 - BytesReadLocal: 149037709 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 149037709 - DecompressionTime: 69169000 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 1817057100 - RemoteScanRanges: 0 - RowsRead: 37185864 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 3449313194 - MaterializeTupleTime(*): 3347527010 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 3429478000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 82021478 - TotalReadThroughput: 42582202 - TotalTime: 3448500729 Instance 544466204e0abc59:f772f133ae0a6935 (host=demeter-csmaz11-18.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 19:1/148.87 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 163193 - RowsProduced: 0 - TotalCpuTime: 6910427712 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 21842 - TotalStorageWaitTime: 20999553 - TotalTime: 3456930694 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 157992 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 19:1/148.87 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 144053935 - BytesReadDataNodeCache: 0 - BytesReadLocal: 144053935 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 144053935 - DecompressionTime: 63316530 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 1941806818 - RemoteScanRanges: 0 - RowsRead: 35940884 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 3445942067 - MaterializeTupleTime(*): 3351899622 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 3427478000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 74185513 - TotalReadThroughput: 41158267 - TotalTime: 3445312783 Instance 544466204e0abc59:f772f133ae0a6958 (host=demeter-csmaz11-16.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 23:1/155.03 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.8571428571428572 - InactiveTotalTime: 0 - PeakMemoryUsage: 51234064 - PerHostPeakMemUsage: 339159312 - PrepareTime: 142169 - RowsProduced: 0 - TotalCpuTime: 6883725991 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 22221 - TotalStorageWaitTime: 21403777 - TotalTime: 3443904879 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 175183 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 23:1/155.03 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 151112343 - BytesReadDataNodeCache: 0 - BytesReadLocal: 151112343 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 151112343 - DecompressionTime: 68876557 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50734832 - PerReadThreadRawHdfsThroughput: 1894299845 - RemoteScanRanges: 0 - RowsRead: 37704818 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3432839321 - MaterializeTupleTime(*): 3332047922 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 3413481000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 79772135 - TotalReadThroughput: 47574985 - TotalTime: 3432194093 Instance 544466204e0abc59:f772f133ae0a691a (host=demeter-csmaz08-8.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 7:1/157.81 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 133346 - RowsProduced: 0 - TotalCpuTime: 6882060052 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 21974 - TotalStorageWaitTime: 21470096 - TotalTime: 3443539586 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 170451 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 7:1/157.81 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 151803230 - BytesReadDataNodeCache: 0 - BytesReadLocal: 151803230 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 151803230 - DecompressionTime: 60044226 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 2157269992 - RemoteScanRanges: 0 - RowsRead: 37876645 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 3431570432 - MaterializeTupleTime(*): 3340202881 - ScannerThreadsSysTime: 15997000 - ScannerThreadsUserTime: 3395483000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 70368211 - TotalReadThroughput: 43176198 - TotalTime: 3430927268 Instance 544466204e0abc59:f772f133ae0a6944 (host=demeter-csmaz10-1.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 6:1/154.91 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 136210 - RowsProduced: 0 - TotalCpuTime: 6877775199 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 21376 - TotalStorageWaitTime: 20594313 - TotalTime: 3441124269 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 164443 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 6:1/154.91 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 151419962 - BytesReadDataNodeCache: 0 - BytesReadLocal: 151419962 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 151419962 - DecompressionTime: 64881667 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 2054697577 - RemoteScanRanges: 0 - RowsRead: 37780629 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3431040346 - MaterializeTupleTime(*): 3334867023 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 3412481000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 73694525 - TotalReadThroughput: 43262846 - TotalTime: 3430443345 Instance 544466204e0abc59:f772f133ae0a6928 (host=demeter-csmaz08-6.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 19:1/152.94 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266824 - PerHostPeakMemUsage: 339192072 - PrepareTime: 172480 - RowsProduced: 0 - TotalCpuTime: 6849388610 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 24002 - TotalStorageWaitTime: 21178914 - TotalTime: 3425549528 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 184585 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 19:1/152.94 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 149037745 - BytesReadDataNodeCache: 0 - BytesReadLocal: 149037745 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 149037745 - DecompressionTime: 58258153 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767592 - PerReadThreadRawHdfsThroughput: 2044293938 - RemoteScanRanges: 0 - RowsRead: 37185863 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 3415025727 - MaterializeTupleTime(*): 3325745975 - ScannerThreadsSysTime: 3999000 - ScannerThreadsUserTime: 3392484000 - ScannerThreadsVoluntaryContextSwitches: 6 - TotalRawHdfsReadTime(*): 72904264 - TotalReadThroughput: 47575817 - TotalTime: 3413177024 Instance 544466204e0abc59:f772f133ae0a694b (host=demeter-csmau10-4.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 2:1/155.81 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266824 - PerHostPeakMemUsage: 339192072 - PrepareTime: 139410 - RowsProduced: 0 - TotalCpuTime: 6885126465 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 21728 - TotalStorageWaitTime: 21122408 - TotalTime: 3423912443 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 180919 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 2:1/155.81 MB Hdfs Read Thread Concurrency Bucket: 0:85.71% 1:14.29% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.14285714285714285 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 151795266 - BytesReadDataNodeCache: 0 - BytesReadLocal: 151795266 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 151795266 - DecompressionTime: 67830963 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767592 - PerReadThreadRawHdfsThroughput: 1765503203 - RemoteScanRanges: 0 - RowsRead: 37874733 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3412983500 - MaterializeTupleTime(*): 3314195994 - ScannerThreadsSysTime: 6998000 - ScannerThreadsUserTime: 3386485000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 85978471 - TotalReadThroughput: 43176106 - TotalTime: 3411943957 Instance 544466204e0abc59:f772f133ae0a6936 (host=demeter-csmau08-11.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 11:1/152.10 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266824 - PerHostPeakMemUsage: 339192072 - PrepareTime: 136690 - RowsProduced: 0 - TotalCpuTime: 6824281161 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 32489 - TotalStorageWaitTime: 20916888 - TotalTime: 3411517777 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 158896 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 11:1/152.10 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 147976005 - BytesReadDataNodeCache: 0 - BytesReadLocal: 147976005 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 147976005 - DecompressionTime: 42799818 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767592 - PerReadThreadRawHdfsThroughput: 2127943703 - RemoteScanRanges: 0 - RowsRead: 36920613 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 3400261910 - MaterializeTupleTime(*): 3329185532 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 3381485000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 69539436 - TotalReadThroughput: 42278858 - TotalTime: 3399602867 Instance 544466204e0abc59:f772f133ae0a6952 (host=demeter-csmaz10-14.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 0:1/151.68 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 137048 - RowsProduced: 0 - TotalCpuTime: 6813196530 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 20207 - TotalStorageWaitTime: 21723043 - TotalTime: 3409355952 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 166866 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 0:1/151.68 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 147900661 - BytesReadDataNodeCache: 0 - BytesReadLocal: 147900661 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 147900661 - DecompressionTime: 65172730 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 2116231726 - RemoteScanRanges: 0 - RowsRead: 36901348 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 3398894042 - MaterializeTupleTime(*): 3301734595 - ScannerThreadsSysTime: 18997000 - ScannerThreadsUserTime: 3360489000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 69888689 - TotalReadThroughput: 42257331 - TotalTime: 3398269080 Instance 544466204e0abc59:f772f133ae0a691d (host=demeter-csmau10-11.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 15:1/149.54 MB 14:1/122.30 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.857142857142857 - InactiveTotalTime: 0 - PeakMemoryUsage: 100616984 - PerHostPeakMemUsage: 379887416 - PrepareTime: 156316 - RowsProduced: 0 - TotalCpuTime: 9838510535 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 18828 - TotalStorageWaitTime: 46066295 - TotalTime: 3407097045 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 157969 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 15:1/149.54 MB 14:1/122.30 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.8571428571428572 - BytesRead: 264397665 - BytesReadDataNodeCache: 0 - BytesReadLocal: 264397665 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 264397665 - DecompressionTime: 100805630 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 100117752 - PerReadThreadRawHdfsThroughput: 2058432589 - RemoteScanRanges: 0 - RowsRead: 65959126 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 13 - ScannerThreadsTotalWallClockTime: 6448943064 - MaterializeTupleTime(*): 6280461238 - ScannerThreadsSysTime: 2999000 - ScannerThreadsUserTime: 6394027000 - ScannerThreadsVoluntaryContextSwitches: 14 - TotalRawHdfsReadTime(*): 128446113 - TotalReadThroughput: 75542190 - TotalTime: 3396435613 Instance 544466204e0abc59:f772f133ae0a6931 (host=demeter-csmau10-19.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 11:1/153.72 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 155719 - RowsProduced: 0 - TotalCpuTime: 6783929148 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 22670 - TotalStorageWaitTime: 20356454 - TotalTime: 3391951530 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 159687 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 11:1/153.72 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 149233134 - BytesReadDataNodeCache: 0 - BytesReadLocal: 149233134 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 149233134 - DecompressionTime: 61818756 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 2087594188 - RemoteScanRanges: 0 - RowsRead: 37233809 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3380889844 - MaterializeTupleTime(*): 3289731891 - ScannerThreadsSysTime: 2999000 - ScannerThreadsUserTime: 3359489000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 71485701 - TotalReadThroughput: 42638038 - TotalTime: 3380026682 Instance 544466204e0abc59:f772f133ae0a692c (host=demeter-csmau10-5.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 4:1/152.56 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 139893 - RowsProduced: 0 - TotalCpuTime: 6759844153 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 20476 - TotalStorageWaitTime: 20756050 - TotalTime: 3382307464 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 153875 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 4:1/152.56 MB Hdfs Read Thread Concurrency Bucket: 0:85.71% 1:14.29% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.14285714285714285 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 149039093 - BytesReadDataNodeCache: 0 - BytesReadLocal: 149039093 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 149039093 - DecompressionTime: 61768115 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 1990844610 - RemoteScanRanges: 0 - RowsRead: 37185864 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 3371372541 - MaterializeTupleTime(*): 3279977110 - ScannerThreadsSysTime: 2999000 - ScannerThreadsUserTime: 3349490000 - ScannerThreadsVoluntaryContextSwitches: 6 - TotalRawHdfsReadTime(*): 74862243 - TotalReadThroughput: 42582598 - TotalTime: 3370877152 Instance 544466204e0abc59:f772f133ae0a691e (host=demeter-csmaz08-12.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 2:1/153.89 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 150938 - RowsProduced: 0 - TotalCpuTime: 6752924784 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 25541 - TotalStorageWaitTime: 20458509 - TotalTime: 3377690662 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 185399 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 2:1/153.89 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 149788210 - BytesReadDataNodeCache: 0 - BytesReadLocal: 149788210 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 149788210 - DecompressionTime: 69312417 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 1960791785 - RemoteScanRanges: 0 - RowsRead: 37373827 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 3364735132 - MaterializeTupleTime(*): 3264442253 - ScannerThreadsSysTime: 10998000 - ScannerThreadsUserTime: 3335492000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 76391696 - TotalReadThroughput: 42796631 - TotalTime: 3363938607 Instance 544466204e0abc59:f772f133ae0a6925 (host=demeter-csmaz10-13.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 0:1/144.17 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 186490 - RowsProduced: 0 - TotalCpuTime: 6780454630 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 19935 - TotalStorageWaitTime: 21190341 - TotalTime: 3372440228 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 192795 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 0:1/144.17 MB Hdfs Read Thread Concurrency Bucket: 0:85.71% 1:14.29% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.14285714285714285 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 140969463 - BytesReadDataNodeCache: 0 - BytesReadLocal: 140969463 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 140969463 - DecompressionTime: 62962223 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 1977709375 - RemoteScanRanges: 0 - RowsRead: 35170166 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3360313649 - MaterializeTupleTime(*): 3266101923 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 3341492000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 71279160 - TotalReadThroughput: 40276989 - TotalTime: 3359621935 Instance 544466204e0abc59:f772f133ae0a6959 (host=demeter-csmaz10-8.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 5:1/151.56 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 188516 - RowsProduced: 0 - TotalCpuTime: 6735257648 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 21588 - TotalStorageWaitTime: 21624268 - TotalTime: 3370567714 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 163175 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 5:1/151.56 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 147657926 - BytesReadDataNodeCache: 0 - BytesReadLocal: 147657926 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 147657926 - DecompressionTime: 60586279 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 2110809415 - RemoteScanRanges: 0 - RowsRead: 36841009 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3360628114 - MaterializeTupleTime(*): 3268828738 - ScannerThreadsSysTime: 5999000 - ScannerThreadsUserTime: 3335492000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 69953225 - TotalReadThroughput: 42187978 - TotalTime: 3360139865 Instance 544466204e0abc59:f772f133ae0a694d (host=demeter-csmaz10-10.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 2:1/154.52 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.8571428571428572 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 148258 - RowsProduced: 0 - TotalCpuTime: 6735490039 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 20701 - TotalStorageWaitTime: 20310754 - TotalTime: 3369605046 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 163556 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 2:1/154.52 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 149945479 - BytesReadDataNodeCache: 0 - BytesReadLocal: 149945479 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 149945479 - DecompressionTime: 65649031 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 2230295341 - RemoteScanRanges: 0 - RowsRead: 37412386 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3359734216 - MaterializeTupleTime(*): 3262842936 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 3341492000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 67231221 - TotalReadThroughput: 47576173 - TotalTime: 3359020921 Instance 544466204e0abc59:f772f133ae0a6922 (host=demeter-csmau08-12.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 17:1/154.83 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266824 - PerHostPeakMemUsage: 339192072 - PrepareTime: 140825 - RowsProduced: 0 - TotalCpuTime: 6727658171 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 19290 - TotalStorageWaitTime: 21024529 - TotalTime: 3366151556 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 166565 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 17:1/154.83 MB Hdfs Read Thread Concurrency Bucket: 0:85.71% 1:14.29% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.14285714285714285 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 152526490 - BytesReadDataNodeCache: 0 - BytesReadLocal: 152526490 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 152526490 - DecompressionTime: 61284590 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767592 - PerReadThreadRawHdfsThroughput: 2165110624 - RemoteScanRanges: 0 - RowsRead: 38057408 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 3355929677 - MaterializeTupleTime(*): 3264421671 - ScannerThreadsSysTime: 10998000 - ScannerThreadsUserTime: 3325494000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 70447435 - TotalReadThroughput: 43578997 - TotalTime: 3355194919 Instance 544466204e0abc59:f772f133ae0a6948 (host=demeter-csmau08-8.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 17:1/153.73 MB 15:1/99.09 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.7142857142857144 - InactiveTotalTime: 0 - PeakMemoryUsage: 101669784 - PerHostPeakMemUsage: 380878760 - PrepareTime: 149202 - RowsProduced: 0 - TotalCpuTime: 9137087876 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 20532 - TotalStorageWaitTime: 43495121 - TotalTime: 3351898389 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 160616 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 17:1/153.73 MB 15:1/99.09 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.7142857142857142 - BytesRead: 246904741 - BytesReadDataNodeCache: 0 - BytesReadLocal: 246904741 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 246904741 - DecompressionTime: 97807309 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 101170552 - PerReadThreadRawHdfsThroughput: 1990759128 - RemoteScanRanges: 0 - RowsRead: 61589089 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 8 - ScannerThreadsTotalWallClockTime: 5799884152 - MaterializeTupleTime(*): 5642033340 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 5749125000 - ScannerThreadsVoluntaryContextSwitches: 18 - TotalRawHdfsReadTime(*): 124025422 - TotalReadThroughput: 70544211 - TotalTime: 3340858158 Instance 544466204e0abc59:f772f133ae0a692b (host=demeter-csmau08-14.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 2:1/154.72 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266840 - PerHostPeakMemUsage: 339192088 - PrepareTime: 135788 - RowsProduced: 0 - TotalCpuTime: 6671639252 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 17935 - TotalStorageWaitTime: 20731222 - TotalTime: 3336250929 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 172895 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 2:1/154.72 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 149131244 - BytesReadDataNodeCache: 0 - BytesReadLocal: 149131244 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 149131244 - DecompressionTime: 56699379 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767608 - PerReadThreadRawHdfsThroughput: 2028367998 - RemoteScanRanges: 0 - RowsRead: 37209337 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3326116661 - MaterializeTupleTime(*): 3239309912 - ScannerThreadsSysTime: 18997000 - ScannerThreadsUserTime: 3288500000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 73522775 - TotalReadThroughput: 42608926 - TotalTime: 3324475005 Instance 544466204e0abc59:f772f133ae0a6943 (host=demeter-csmaz11-1.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 21:1/152.07 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266824 - PerHostPeakMemUsage: 339192072 - PrepareTime: 189093 - RowsProduced: 0 - TotalCpuTime: 6583768844 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 23399 - TotalStorageWaitTime: 21891886 - TotalTime: 3293133965 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 182211 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 21:1/152.07 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 147918564 - BytesReadDataNodeCache: 0 - BytesReadLocal: 147918564 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 147918564 - DecompressionTime: 66435290 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767592 - PerReadThreadRawHdfsThroughput: 1943230451 - RemoteScanRanges: 0 - RowsRead: 36906411 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 3283774506 - MaterializeTupleTime(*): 3185761622 - ScannerThreadsSysTime: 2999000 - ScannerThreadsUserTime: 3260504000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 76119929 - TotalReadThroughput: 42262446 - TotalTime: 3282772179 Instance 544466204e0abc59:f772f133ae0a6918 (host=demeter-csmau08-13.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 12:1/152.40 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.8571428571428572 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 141566 - RowsProduced: 0 - TotalCpuTime: 6570974422 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 17934 - TotalStorageWaitTime: 21312168 - TotalTime: 3288083575 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 165429 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 12:1/152.40 MB Hdfs Read Thread Concurrency Bucket: 0:83.33% 1:16.67% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.16666666666666666 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 148318000 - BytesReadDataNodeCache: 0 - BytesReadLocal: 148318000 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 148318000 - DecompressionTime: 63047546 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 2165575361 - RemoteScanRanges: 0 - RowsRead: 37006140 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3277893807 - MaterializeTupleTime(*): 3184382150 - ScannerThreadsSysTime: 4999000 - ScannerThreadsUserTime: 3253505000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 68488958 - TotalReadThroughput: 42376571 - TotalTime: 3277325748 Instance 544466204e0abc59:f772f133ae0a694e (host=demeter-csmau10-15.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 2:1/153.50 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 139013 - RowsProduced: 0 - TotalCpuTime: 6539025785 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 20436 - TotalStorageWaitTime: 21417406 - TotalTime: 3270701444 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 168703 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 2:1/153.50 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 147876128 - BytesReadDataNodeCache: 0 - BytesReadLocal: 147876128 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 147876128 - DecompressionTime: 62406723 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 1996110398 - RemoteScanRanges: 0 - RowsRead: 36895350 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 3261474063 - MaterializeTupleTime(*): 3167901570 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 3239507000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 74082139 - TotalReadThroughput: 47576207 - TotalTime: 3260348283 Instance 544466204e0abc59:f772f133ae0a6941 (host=demeter-csmaz08-18.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 0:1/127.73 MB 1:1/125.74 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.857142857142857 - InactiveTotalTime: 0 - PeakMemoryUsage: 101538688 - PerHostPeakMemUsage: 381337568 - PrepareTime: 202183 - RowsProduced: 0 - TotalCpuTime: 9525766970 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 22625 - TotalStorageWaitTime: 42651507 - TotalTime: 3257039707 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 155703 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 0:1/127.73 MB 1:1/125.74 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.8571428571428572 - BytesRead: 248039019 - BytesReadDataNodeCache: 0 - BytesReadLocal: 248039019 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 248039019 - DecompressionTime: 85729601 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 101039456 - PerReadThreadRawHdfsThroughput: 1707126147 - RemoteScanRanges: 0 - RowsRead: 61872736 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 13 - ScannerThreadsTotalWallClockTime: 6285058308 - MaterializeTupleTime(*): 6137607278 - ScannerThreadsSysTime: 14997000 - ScannerThreadsUserTime: 6230052000 - ScannerThreadsVoluntaryContextSwitches: 14 - TotalRawHdfsReadTime(*): 145296245 - TotalReadThroughput: 70868291 - TotalTime: 3246550784 Instance 544466204e0abc59:f772f133ae0a694a (host=demeter-csmaz08-9.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 17:1/150.97 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266824 - PerHostPeakMemUsage: 339192072 - PrepareTime: 141593 - RowsProduced: 0 - TotalCpuTime: 6547790601 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 20289 - TotalStorageWaitTime: 19971736 - TotalTime: 3255673635 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 167996 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 17:1/150.97 MB Hdfs Read Thread Concurrency Bucket: 0:85.71% 1:14.29% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.14285714285714285 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 149036929 - BytesReadDataNodeCache: 0 - BytesReadLocal: 149036929 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 149036929 - DecompressionTime: 67393873 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767592 - PerReadThreadRawHdfsThroughput: 2133870258 - RemoteScanRanges: 0 - RowsRead: 37185865 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 3245670782 - MaterializeTupleTime(*): 3149287429 - ScannerThreadsSysTime: 17997000 - ScannerThreadsUserTime: 3209512000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 69843482 - TotalReadThroughput: 42581979 - TotalTime: 3245215789 Instance 544466204e0abc59:f772f133ae0a6940 (host=demeter-csmau08-1.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 15:1/153.96 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 140253 - RowsProduced: 0 - TotalCpuTime: 6496324288 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 40423 - TotalStorageWaitTime: 21060163 - TotalTime: 3246466075 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 194734 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 15:1/153.96 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 147843883 - BytesReadDataNodeCache: 0 - BytesReadLocal: 147843883 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 147843883 - DecompressionTime: 53141412 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 1521745876 - RemoteScanRanges: 0 - RowsRead: 36887351 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 13 - ScannerThreadsTotalWallClockTime: 3235929871 - MaterializeTupleTime(*): 3152772419 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 3215511000 - ScannerThreadsVoluntaryContextSwitches: 6 - TotalRawHdfsReadTime(*): 97154121 - TotalReadThroughput: 49281294 - TotalTime: 3234230283 Instance 544466204e0abc59:f772f133ae0a692e (host=demeter-csmau08-2.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 1:1/128.83 MB 2:1/126.28 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.857142857142857 - InactiveTotalTime: 0 - PeakMemoryUsage: 101325672 - PerHostPeakMemUsage: 380862376 - PrepareTime: 149580 - RowsProduced: 0 - TotalCpuTime: 9317020000 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 40352 - TotalStorageWaitTime: 44646439 - TotalTime: 3246259060 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 167994 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 1:1/128.83 MB 2:1/126.28 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.8571428571428572 - BytesRead: 247000548 - BytesReadDataNodeCache: 0 - BytesReadLocal: 247000548 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 247000548 - DecompressionTime: 82307958 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 100826440 - PerReadThreadRawHdfsThroughput: 1927815775 - RemoteScanRanges: 0 - RowsRead: 61614348 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 10 - ScannerThreadsTotalWallClockTime: 6081822299 - MaterializeTupleTime(*): 5934387810 - ScannerThreadsSysTime: 12997000 - ScannerThreadsUserTime: 6027083000 - ScannerThreadsVoluntaryContextSwitches: 16 - TotalRawHdfsReadTime(*): 128124560 - TotalReadThroughput: 70571585 - TotalTime: 3235539595 Instance 544466204e0abc59:f772f133ae0a6927 (host=demeter-csmau08-19.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 22:1/153.39 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266824 - PerHostPeakMemUsage: 339192072 - PrepareTime: 187326 - RowsProduced: 0 - TotalCpuTime: 6481226104 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 18973 - TotalStorageWaitTime: 20495334 - TotalTime: 3241952019 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 172152 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 22:1/153.39 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 147607470 - BytesReadDataNodeCache: 0 - BytesReadLocal: 147607470 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 147607470 - DecompressionTime: 63430031 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767592 - PerReadThreadRawHdfsThroughput: 2062349840 - RemoteScanRanges: 0 - RowsRead: 36828037 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 3229247660 - MaterializeTupleTime(*): 3135834165 - ScannerThreadsSysTime: 7998000 - ScannerThreadsUserTime: 3202513000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 71572469 - TotalReadThroughput: 42173562 - TotalTime: 3228808314 Instance 544466204e0abc59:f772f133ae0a695b (host=demeter-csmaz08-20.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 13:1/153.08 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 173883 - RowsProduced: 0 - TotalCpuTime: 6457755331 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 22399 - TotalStorageWaitTime: 22003622 - TotalTime: 3231443089 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 170237 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 13:1/153.08 MB Hdfs Read Thread Concurrency Bucket: 0:85.71% 1:14.29% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.14285714285714285 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 149038549 - BytesReadDataNodeCache: 0 - BytesReadLocal: 149038549 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 149038549 - DecompressionTime: 66870193 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 1789577423 - RemoteScanRanges: 0 - RowsRead: 37185865 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 3220974141 - MaterializeTupleTime(*): 3122532305 - ScannerThreadsSysTime: 9998000 - ScannerThreadsUserTime: 3190514000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 83281420 - TotalReadThroughput: 42582442 - TotalTime: 3220238951 Instance 544466204e0abc59:f772f133ae0a6912 (host=demeter-csmaz11-7.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 11:1/151.12 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51266832 - PerHostPeakMemUsage: 339192080 - PrepareTime: 142038 - RowsProduced: 0 - TotalCpuTime: 6436728095 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 21178 - TotalStorageWaitTime: 20360675 - TotalTime: 3218601555 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 167270 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 11:1/151.12 MB Hdfs Read Thread Concurrency Bucket: 0:83.33% 1:16.67% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.16666666666666666 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 147976424 - BytesReadDataNodeCache: 0 - BytesReadLocal: 147976424 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 147976424 - DecompressionTime: 65180122 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50767600 - PerReadThreadRawHdfsThroughput: 2015471645 - RemoteScanRanges: 0 - RowsRead: 36920612 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 6 - ScannerThreadsTotalWallClockTime: 3207353991 - MaterializeTupleTime(*): 3112228522 - ScannerThreadsSysTime: 8998000 - ScannerThreadsUserTime: 3179516000 - ScannerThreadsVoluntaryContextSwitches: 6 - TotalRawHdfsReadTime(*): 73420246 - TotalReadThroughput: 49325474 - TotalTime: 3206698998 Instance 544466204e0abc59:f772f133ae0a6917 (host=demeter-csmaz08-16.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 8:1/129.00 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250448 - PerHostPeakMemUsage: 339175696 - PrepareTime: 143169 - RowsProduced: 0 - TotalCpuTime: 6428053005 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 22437 - TotalStorageWaitTime: 19606809 - TotalTime: 3215179760 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 173228 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 8:1/129.00 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 124614788 - BytesReadDataNodeCache: 0 - BytesReadLocal: 124614788 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 124614788 - DecompressionTime: 54407194 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751216 - PerReadThreadRawHdfsThroughput: 2075576842 - RemoteScanRanges: 0 - RowsRead: 31085059 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 3204522238 - MaterializeTupleTime(*): 3121935351 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 3186515000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 60038629 - TotalReadThroughput: 41538262 - TotalTime: 3203841579 Instance 544466204e0abc59:f772f133ae0a6946 (host=demeter-csmau10-17.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 9:1/121.78 MB 11:1/126.54 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.7142857142857144 - InactiveTotalTime: 0 - PeakMemoryUsage: 101669776 - PerHostPeakMemUsage: 381571056 - PrepareTime: 159062 - RowsProduced: 0 - TotalCpuTime: 9179994463 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 43641 - TotalStorageWaitTime: 48664577 - TotalTime: 3200398240 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 162072 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 9:1/121.78 MB 11:1/126.54 MB Hdfs Read Thread Concurrency Bucket: 0:83.33% 1:16.67% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:4 ExecOption: Codegen enabled: 0 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.16666666666666666 - AverageScannerThreadConcurrency: 2.0 - BytesRead: 240993047 - BytesReadDataNodeCache: 0 - BytesReadLocal: 240993047 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 240993047 - DecompressionTime: 80809806 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 2 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 101170544 - PerReadThreadRawHdfsThroughput: 2047180000 - RemoteScanRanges: 0 - RowsRead: 60112627 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 11 - ScannerThreadsTotalWallClockTime: 5990411499 - MaterializeTupleTime(*): 5839331051 - ScannerThreadsSysTime: 5999000 - ScannerThreadsUserTime: 5937097000 - ScannerThreadsVoluntaryContextSwitches: 15 - TotalRawHdfsReadTime(*): 117719520 - TotalReadThroughput: 80331015 - TotalTime: 3188091498 Instance 544466204e0abc59:f772f133ae0a691c (host=demeter-csmaz11-3.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 6:1/154.21 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51279112 - PerHostPeakMemUsage: 339204360 - PrepareTime: 136785 - RowsProduced: 0 - TotalCpuTime: 6381828869 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 19843 - TotalStorageWaitTime: 20037533 - TotalTime: 3192767124 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 160935 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 6:1/154.21 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 148044812 - BytesReadDataNodeCache: 0 - BytesReadLocal: 148044812 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 148044812 - DecompressionTime: 61428041 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50779880 - PerReadThreadRawHdfsThroughput: 2120479755 - RemoteScanRanges: 0 - RowsRead: 36935315 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 9 - ScannerThreadsTotalWallClockTime: 3182694236 - MaterializeTupleTime(*): 3091652806 - ScannerThreadsSysTime: 17997000 - ScannerThreadsUserTime: 3145521000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 69816659 - TotalReadThroughput: 49348270 - TotalTime: 3182076695 Instance 544466204e0abc59:f772f133ae0a6914 (host=demeter-csmaz08-7.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 2:1/129.62 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250448 - PerHostPeakMemUsage: 339175696 - PrepareTime: 130055 - RowsProduced: 0 - TotalCpuTime: 6341765359 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 19201 - TotalStorageWaitTime: 19491581 - TotalTime: 3172344577 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 164346 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 2:1/129.62 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 124931626 - BytesReadDataNodeCache: 0 - BytesReadLocal: 124931626 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 124931626 - DecompressionTime: 53362175 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751216 - PerReadThreadRawHdfsThroughput: 1987972517 - RemoteScanRanges: 0 - RowsRead: 31164046 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 3162004484 - MaterializeTupleTime(*): 3080755314 - ScannerThreadsSysTime: 6998000 - ScannerThreadsUserTime: 3137523000 - ScannerThreadsVoluntaryContextSwitches: 6 - TotalRawHdfsReadTime(*): 62843739 - TotalReadThroughput: 41643875 - TotalTime: 3161551232 Instance 544466204e0abc59:f772f133ae0a6910 (host=demeter-csmaz08-4.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 7:1/126.52 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250448 - PerHostPeakMemUsage: 339175696 - PrepareTime: 135115 - RowsProduced: 0 - TotalCpuTime: 6313047531 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 20161 - TotalStorageWaitTime: 19865417 - TotalTime: 3158344244 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 187139 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 7:1/126.52 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 122389958 - BytesReadDataNodeCache: 0 - BytesReadLocal: 122389958 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 122389958 - DecompressionTime: 51746713 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751216 - PerReadThreadRawHdfsThroughput: 2082997762 - RemoteScanRanges: 0 - RowsRead: 30529291 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 3148195470 - MaterializeTupleTime(*): 3067752725 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 3129524000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 58756644 - TotalReadThroughput: 34968559 - TotalTime: 3147706133 Instance 544466204e0abc59:f772f133ae0a693d (host=demeter-csmaz10-20.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 22:1/125.73 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250448 - PerHostPeakMemUsage: 339175696 - PrepareTime: 133368 - RowsProduced: 0 - TotalCpuTime: 6304287992 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 22904 - TotalStorageWaitTime: 20235595 - TotalTime: 3153633983 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 161654 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 22:1/125.73 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 123580200 - BytesReadDataNodeCache: 0 - BytesReadLocal: 123580200 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 123580200 - DecompressionTime: 52759537 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751216 - PerReadThreadRawHdfsThroughput: 1971210861 - RemoteScanRanges: 0 - RowsRead: 30826183 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 3142877938 - MaterializeTupleTime(*): 3062271457 - ScannerThreadsSysTime: 15997000 - ScannerThreadsUserTime: 3108527000 - ScannerThreadsVoluntaryContextSwitches: 8 - TotalRawHdfsReadTime(*): 62692532 - TotalReadThroughput: 35308628 - TotalTime: 3142518539 Instance 544466204e0abc59:f772f133ae0a692a (host=demeter-csmaz11-15.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 15:1/126.92 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250440 - PerHostPeakMemUsage: 347078600 - PrepareTime: 136910 - RowsProduced: 0 - TotalCpuTime: 6107981907 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 20614 - TotalStorageWaitTime: 191460439 - TotalTime: 3124177730 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 184609 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 15:1/126.92 MB Hdfs Read Thread Concurrency Bucket: 0:33.33% 1:66.67% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.6666666666666666 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 124614477 - BytesReadDataNodeCache: 0 - BytesReadLocal: 124614477 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 124614477 - DecompressionTime: 51402737 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751208 - PerReadThreadRawHdfsThroughput: 94410159 - RemoteScanRanges: 0 - RowsRead: 31085059 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 3114233542 - MaterializeTupleTime(*): 2863690768 - ScannerThreadsSysTime: 1999000 - ScannerThreadsUserTime: 2922555000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 1319926563 - TotalReadThroughput: 41538159 - TotalTime: 3113598336 Instance 544466204e0abc59:f772f133ae0a6950 (host=demeter-csmau10-13.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 20:1/130.80 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250448 - PerHostPeakMemUsage: 339175696 - PrepareTime: 203056 - RowsProduced: 0 - TotalCpuTime: 6207369507 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 19181 - TotalStorageWaitTime: 22321664 - TotalTime: 3107135191 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 211774 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 20:1/130.80 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 126911995 - BytesReadDataNodeCache: 0 - BytesReadLocal: 126911995 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 126911995 - DecompressionTime: 40225409 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751216 - PerReadThreadRawHdfsThroughput: 1757514540 - RemoteScanRanges: 0 - RowsRead: 31658935 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 7 - ScannerThreadsTotalWallClockTime: 3096394723 - MaterializeTupleTime(*): 3025441640 - ScannerThreadsSysTime: 7998000 - ScannerThreadsUserTime: 3067533000 - ScannerThreadsVoluntaryContextSwitches: 8 - TotalRawHdfsReadTime(*): 72211064 - TotalReadThroughput: 42303998 - TotalTime: 3095668244 Instance 544466204e0abc59:f772f133ae0a6955 (host=demeter-csmaz08-10.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 20:1/130.39 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51234064 - PerHostPeakMemUsage: 339159312 - PrepareTime: 128921 - RowsProduced: 0 - TotalCpuTime: 6189765697 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 18530 - TotalStorageWaitTime: 21487142 - TotalTime: 3097864471 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 153857 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 20:1/130.39 MB Hdfs Read Thread Concurrency Bucket: 0:83.33% 1:16.67% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.16666666666666666 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 125789453 - BytesReadDataNodeCache: 0 - BytesReadLocal: 125789453 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 125789453 - DecompressionTime: 52126084 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50734832 - PerReadThreadRawHdfsThroughput: 1833368735 - RemoteScanRanges: 0 - RowsRead: 31379665 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3087796311 - MaterializeTupleTime(*): 3005749570 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 3067533000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 68611104 - TotalReadThroughput: 41929817 - TotalTime: 3087318705 Instance 544466204e0abc59:f772f133ae0a6930 (host=demeter-csmaz10-19.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 2:1/128.13 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250448 - PerHostPeakMemUsage: 339175696 - PrepareTime: 133312 - RowsProduced: 0 - TotalCpuTime: 6115183995 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 21173 - TotalStorageWaitTime: 20743124 - TotalTime: 3058613662 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 159889 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 2:1/128.13 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 125893586 - BytesReadDataNodeCache: 0 - BytesReadLocal: 125893586 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 125893586 - DecompressionTime: 46247545 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751216 - PerReadThreadRawHdfsThroughput: 1887946567 - RemoteScanRanges: 0 - RowsRead: 31404625 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 3048804858 - MaterializeTupleTime(*): 2972666544 - ScannerThreadsSysTime: 5999000 - ScannerThreadsUserTime: 3023540000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 66682812 - TotalReadThroughput: 41964528 - TotalTime: 3047398391 Instance 544466204e0abc59:f772f133ae0a695a (host=demeter-csmaz10-5.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 6:1/125.22 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250448 - PerHostPeakMemUsage: 339175696 - PrepareTime: 140263 - RowsProduced: 0 - TotalCpuTime: 6095961501 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 20056 - TotalStorageWaitTime: 21004483 - TotalTime: 3051087499 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 172580 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 6:1/125.22 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 124210542 - BytesReadDataNodeCache: 0 - BytesReadLocal: 124210542 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 124210542 - DecompressionTime: 53837299 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751216 - PerReadThreadRawHdfsThroughput: 1935121445 - RemoteScanRanges: 0 - RowsRead: 30984152 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 3040730216 - MaterializeTupleTime(*): 2957714048 - ScannerThreadsSysTime: 12998000 - ScannerThreadsUserTime: 3008542000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 64187466 - TotalReadThroughput: 41403514 - TotalTime: 3040110724 Instance 544466204e0abc59:f772f133ae0a691b (host=demeter-csmau10-16.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 12:1/130.10 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250448 - PerHostPeakMemUsage: 339175696 - PrepareTime: 138376 - RowsProduced: 0 - TotalCpuTime: 6055088328 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 21242 - TotalStorageWaitTime: 21641005 - TotalTime: 3030970497 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 176073 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 12:1/130.10 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 126932036 - BytesReadDataNodeCache: 0 - BytesReadLocal: 126932036 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 126932036 - DecompressionTime: 55574862 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751216 - PerReadThreadRawHdfsThroughput: 1951301773 - RemoteScanRanges: 0 - RowsRead: 31663777 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 3019545355 - MaterializeTupleTime(*): 2933932749 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 2999544000 - ScannerThreadsVoluntaryContextSwitches: 6 - TotalRawHdfsReadTime(*): 65049926 - TotalReadThroughput: 42310678 - TotalTime: 3018870336 Instance 544466204e0abc59:f772f133ae0a6915 (host=demeter-csmau10-10.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 8:1/129.06 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250448 - PerHostPeakMemUsage: 339175696 - PrepareTime: 112472 - RowsProduced: 0 - TotalCpuTime: 6018881296 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 16495 - TotalStorageWaitTime: 13903319 - TotalTime: 3007411777 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 93457 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 8:1/129.06 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 124102255 - BytesReadDataNodeCache: 0 - BytesReadLocal: 124102255 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 124102255 - DecompressionTime: 34703346 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751216 - PerReadThreadRawHdfsThroughput: 2513778893 - RemoteScanRanges: 0 - RowsRead: 30957045 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 2999652605 - MaterializeTupleTime(*): 2944185076 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 2987545000 - ScannerThreadsVoluntaryContextSwitches: 5 - TotalRawHdfsReadTime(*): 49368803 - TotalReadThroughput: 41367418 - TotalTime: 2999563809 Instance 544466204e0abc59:f772f133ae0a6949 (host=demeter-csmaz11-12.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 11:1/129.77 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250448 - PerHostPeakMemUsage: 339175696 - PrepareTime: 143255 - RowsProduced: 0 - TotalCpuTime: 6009383812 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 23450 - TotalStorageWaitTime: 21252946 - TotalTime: 3004679793 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 169535 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 11:1/129.77 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 124555095 - BytesReadDataNodeCache: 0 - BytesReadLocal: 124555095 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 124555095 - DecompressionTime: 58376245 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751216 - PerReadThreadRawHdfsThroughput: 1875102718 - RemoteScanRanges: 0 - RowsRead: 31070269 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 5 - ScannerThreadsTotalWallClockTime: 2994689468 - MaterializeTupleTime(*): 2905991264 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 2975547000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 66425745 - TotalReadThroughput: 41518365 - TotalTime: 2994200919 Instance 544466204e0abc59:f772f133ae0a6926 (host=demeter-csmaz08-13.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 18:1/129.85 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250440 - PerHostPeakMemUsage: 339175688 - PrepareTime: 138940 - RowsProduced: 0 - TotalCpuTime: 5978803669 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 19154 - TotalStorageWaitTime: 21868021 - TotalTime: 2993013637 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 204690 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 18:1/129.85 MB Hdfs Read Thread Concurrency Bucket: 0:83.33% 1:16.67% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.16666666666666666 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 124730257 - BytesReadDataNodeCache: 0 - BytesReadLocal: 124730257 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 124730257 - DecompressionTime: 56860249 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751208 - PerReadThreadRawHdfsThroughput: 1889990548 - RemoteScanRanges: 0 - RowsRead: 31113913 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 2982263441 - MaterializeTupleTime(*): 2894707245 - ScannerThreadsSysTime: 4999000 - ScannerThreadsUserTime: 2957550000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 65995175 - TotalReadThroughput: 41576752 - TotalTime: 2981767020 Instance 544466204e0abc59:f772f133ae0a692d (host=demeter-csmaz10-6.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 9:1/126.00 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250440 - PerHostPeakMemUsage: 339175688 - PrepareTime: 140259 - RowsProduced: 0 - TotalCpuTime: 5877392553 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 21455 - TotalStorageWaitTime: 21018571 - TotalTime: 2942160825 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 162533 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 9:1/126.00 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 123149963 - BytesReadDataNodeCache: 0 - BytesReadLocal: 123149963 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 123149963 - DecompressionTime: 53557298 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751208 - PerReadThreadRawHdfsThroughput: 1917068465 - RemoteScanRanges: 0 - RowsRead: 30719056 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 2931582916 - MaterializeTupleTime(*): 2848918130 - ScannerThreadsSysTime: 11998000 - ScannerThreadsUserTime: 2900559000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 64238688 - TotalReadThroughput: 41049987 - TotalTime: 2931178713 Instance 544466204e0abc59:f772f133ae0a6945 (host=demeter-csmau08-17.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 5:1/131.04 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51234064 - PerHostPeakMemUsage: 339159312 - PrepareTime: 145022 - RowsProduced: 0 - TotalCpuTime: 5868000594 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 21113 - TotalStorageWaitTime: 19446472 - TotalTime: 2936251167 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 154042 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 5:1/131.04 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 125720720 - BytesReadDataNodeCache: 0 - BytesReadLocal: 125720720 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 125720720 - DecompressionTime: 51965388 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50734832 - PerReadThreadRawHdfsThroughput: 1863998430 - RemoteScanRanges: 0 - RowsRead: 31362264 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 2925845895 - MaterializeTupleTime(*): 2846331563 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 2908557000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 67446795 - TotalReadThroughput: 41906906 - TotalTime: 2925402956 Instance 544466204e0abc59:f772f133ae0a691f (host=demeter-csmaz11-8.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 12:1/126.90 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250440 - PerHostPeakMemUsage: 339175688 - PrepareTime: 162207 - RowsProduced: 0 - TotalCpuTime: 5832827544 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 21542 - TotalStorageWaitTime: 21132583 - TotalTime: 2917563220 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 158863 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 12:1/126.90 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 125287880 - BytesReadDataNodeCache: 0 - BytesReadLocal: 125287880 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 125287880 - DecompressionTime: 47677715 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751208 - PerReadThreadRawHdfsThroughput: 1917790114 - RemoteScanRanges: 0 - RowsRead: 31253237 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 6 - ScannerThreadsTotalWallClockTime: 2906816939 - MaterializeTupleTime(*): 2829709138 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 2886561000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 65329297 - TotalReadThroughput: 41762626 - TotalTime: 2905539292 Instance 544466204e0abc59:f772f133ae0a693c (host=demeter-csmaz11-10.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 13:1/128.40 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250448 - PerHostPeakMemUsage: 339175696 - PrepareTime: 131532 - RowsProduced: 0 - TotalCpuTime: 5748285627 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 21827 - TotalStorageWaitTime: 20538557 - TotalTime: 2876885288 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 153523 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 13:1/128.40 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 125778781 - BytesReadDataNodeCache: 0 - BytesReadLocal: 125778781 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 125778781 - DecompressionTime: 52590525 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751216 - PerReadThreadRawHdfsThroughput: 1778084166 - RemoteScanRanges: 0 - RowsRead: 31375573 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 4 - ScannerThreadsTotalWallClockTime: 2866353193 - MaterializeTupleTime(*): 2784012772 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 2846567000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 70738373 - TotalReadThroughput: 41926260 - TotalTime: 2866021454 Instance 544466204e0abc59:f772f133ae0a6913 (host=demeter-csmaz11-4.demeter.hpc.mssm.edu:22000) Hdfs split stats (:<# splits>/): 2:1/129.26 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 51250448 - PerHostPeakMemUsage: 339175696 - PrepareTime: 131072 - RowsProduced: 0 - TotalCpuTime: 5659810734 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 26489 - TotalStorageWaitTime: 19737330 - TotalTime: 2831046345 DataStreamSender (dst_id=4) - AsyncTotalTime: 0 - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 491040 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 158138 - UncompressedRowBatchSize: 0 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 2:1/129.26 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% File Formats: PARQUET/SNAPPY:2 ExecOption: Codegen enabled: 0 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 126114193 - BytesReadDataNodeCache: 0 - BytesReadLocal: 126114193 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 126114193 - DecompressionTime: 48862604 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50751216 - PerReadThreadRawHdfsThroughput: 2144165760 - RemoteScanRanges: 0 - RowsRead: 31459650 - RowsReturned: 0 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 3 - ScannerThreadsTotalWallClockTime: 2821010623 - MaterializeTupleTime(*): 2744361956 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 2802573000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 58817371 - TotalReadThroughput: 42038064 - TotalTime: 2820428287