Query (id=7347b5216e8840b9:a19c3b0800000000): DEBUG MODE WARNING: Query profile created while running a DEBUG build of Impala. Use RELEASE builds to measure query performance. - InactiveTotalTime: 0.000ns - TotalTime: 0.000ns Summary: Session ID: e74e1f9a7e7db6a6:42870e99b380a6a4 Session Type: HIVESERVER2 HiveServer2 Protocol Version: V6 Start Time: 2023-03-23 10:23:00.459491000 End Time: 2023-03-23 10:23:12.904154000 Query Type: DDL Query State: FINISHED Impala Query State: FINISHED Query Status: OK Impala Version: impalad version 4.3.0-SNAPSHOT DEBUG (build 1cfd41e8b10f6e91fc79d50ab58e671d63b65eec) User: joemcdonnell Connected User: joemcdonnell Delegated User: Network Address: 127.0.0.1:55866 Default Db: default Sql Statement: create table whatever5 as select * from functional.alltypes Coordinator: joemcdonnell-22743:27000 Query Options (set by configuration): TIMEZONE=America/Los_Angeles,CLIENT_IDENTIFIER=Impala Shell v4.3.0-SNAPSHOT (1cfd41e) built on Thu 23 Mar 2023 10:11:03 AM PDT Query Options (set by configuration and planner): MT_DOP=0,TIMEZONE=America/Los_Angeles,CLIENT_IDENTIFIER=Impala Shell v4.3.0-SNAPSHOT (1cfd41e) built on Thu 23 Mar 2023 10:11:03 AM PDT,MINMAX_FILTER_THRESHOLD=0.5,MINMAX_FILTERING_LEVEL=PAGE DDL Type: CREATE_TABLE_AS_SELECT DDL execution mode: asynchronous Plan: ---------------- Max Per-Host Resource Reservation: Memory=32.00KB Threads=2 Per-Host Resource Estimates: Memory=160MB Codegen disabled by planner Analyzed query: SELECT * FROM functional.alltypes F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3 | Per-Host Resources: mem-estimate=160.10MB mem-reservation=32.00KB thread-reservation=2 WRITE TO HDFS [default.whatever5, OVERWRITE=false] | partitions=1 | output exprs: functional.alltypes.id, functional.alltypes.bool_col, functional.alltypes.tinyint_col, functional.alltypes.smallint_col, functional.alltypes.int_col, functional.alltypes.bigint_col, functional.alltypes.float_col, functional.alltypes.double_col, functional.alltypes.date_string_col, functional.alltypes.string_col, functional.alltypes.timestamp_col, functional.alltypes.year, functional.alltypes.month | mem-estimate=100.00KB mem-reservation=0B thread-reservation=0 | 00:SCAN HDFS [functional.alltypes, RANDOM] HDFS partitions=24/24 files=24 size=478.45KB stored statistics: table: rows=7.30K size=478.45KB partitions: 24/24 rows=7.30K columns: all extrapolated-rows=disabled max-scan-range-rows=310 mem-estimate=160.00MB mem-reservation=32.00KB thread-reservation=1 tuple-ids=0 row-size=89B cardinality=7.30K in pipelines: 00(GETNEXT) ---------------- Estimated Per-Host Mem: 167874560 Request Pool: default-pool Per Host Min Memory Reservation: joemcdonnell-22743:27000(32.00 KB) joemcdonnell-22743:27001(32.00 KB) joemcdonnell-22743:27002(32.00 KB) Per Host Number of Fragment Instances: joemcdonnell-22743:27000(1) joemcdonnell-22743:27001(1) joemcdonnell-22743:27002(1) Admission result: Admitted immediately Cluster Memory Admitted: 480.29 MB Executor Group: default ExecSummary: Operator #Hosts #Inst Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail --------------------------------------------------------------------------------------------------------------------- F00:HDFS WRITER 3 3 60.919ms 69.031ms 50.00 KB 100.00 KB 00:SCAN HDFS 3 3 3.224ms 3.656ms 7.30K 7.30K 1.93 MB 160.00 MB functional.alltypes Errors: Query Compilation: 112.910ms - Metadata of all 1 tables cached: 14.492ms (14.492ms) - Analysis finished: 83.470ms (68.978ms) - Authorization finished (noop): 83.564ms (93.901us) - Value transfer graph computed: 101.401ms (17.836ms) - Single node plan created: 105.293ms (3.892ms) - Runtime filters computed: 105.328ms (34.774us) - Distributed plan created: 106.331ms (1.003ms) - Planning finished: 112.910ms (6.578ms) Query Timeline: 12s696ms - Query submitted: 64.303us (64.303us) - Planning finished: 117.816ms (117.752ms) - Submit for admission: 12s360ms (12s243ms) - Completed admission: 12s361ms (135.763us) - Ready to start on 3 backends: 12s361ms (66.217us) - All 3 execution backends (3 fragment instances) started: 12s362ms (1.061ms) - Last row fetched: 12s444ms (82.377ms) - Released admission control resources: 12s444ms (26.829us) - DML data written: 12s444ms (105.084us) - DML Metastore update finished: 12s636ms (192.154ms) - Rows available: 12s637ms (105.540us) - Unregister query: 12s696ms (59.646ms) - AdmissionControlTimeSinceLastUpdate: 5.000ms - ComputeScanRangeAssignmentTimer: 52.712us - InactiveTotalTime: 0.000ns - TotalTime: 0.000ns Frontend: Referenced Tables: functional.alltypes - ExecutorGroupsConsidered: 1 (1) ImpalaServer: - CatalogOpExecTimer: 12s235ms - ClientFetchWaitTimer: 59.585ms - InactiveTotalTime: 0.000ns - MetastoreUpdateTimer: 192.219ms - NumRowsFetched: 0 (0) - NumRowsFetchedFromCache: 0 (0) - RowMaterializationRate: 0 - RowMaterializationTimer: 0.000ns - TotalTime: 0.000ns Execution Profile 7347b5216e8840b9:a19c3b0800000000: Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled Bloom Size Est fpp Min value Max value In-list size ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Backend startup latencies: Count: 3, min / max: 1ms / 1ms, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 1ms Slowest backend to start up: joemcdonnell-22743:27002 Per Node Peak Memory Usage: joemcdonnell-22743:27002(2.16 MB) joemcdonnell-22743:27001(2.19 MB) joemcdonnell-22743:27000(2.23 MB) Per Node Bytes Read: joemcdonnell-22743:27002(161.09 KB) joemcdonnell-22743:27001(158.70 KB) joemcdonnell-22743:27000(158.66 KB) Per Node User Time: joemcdonnell-22743:27002(17.274ms) joemcdonnell-22743:27001(24.717ms) joemcdonnell-22743:27000(20.337ms) Per Node System Time: joemcdonnell-22743:27002(3.861ms) joemcdonnell-22743:27001(7.066ms) joemcdonnell-22743:27000(3.410ms) DML Stats: Partition: Default NumModifiedRows: 7300 BytesWritten: 638.54 KB - ExchangeScanRatio: 0.00 - FiltersReceived: 0 (0) - FinalizationTimer: 9.010ms - FileDeletionTimer: 3.915ms - FileMoveTimer: 3.324ms - Overwrite/PartitionCreationTimer: 645.589us - InactiveTotalTime: 0.000ns - InnerNodeSelectivityRatio: 0.00 - NumBackends: 3 (3) - NumCompletedBackends: 3 (3) - NumFragmentInstances: 3 (3) - NumFragments: 1 (1) - TotalBytesRead: 478.45 KB (489934) - TotalBytesSent: 0 - TotalCpuTime: 76.665ms - TotalInnerBytesSent: 0 - TotalScanBytesSent: 0 - TotalTime: 83.556ms Per Node Profiles: - InactiveTotalTime: 0.000ns - TotalTime: 0.000ns joemcdonnell-22743:27002: - AdmissionSlots: 1 (1) - BloomFilterBytes: 0 - InListFilterItems: 0 (0) - InactiveTotalTime: 0.000ns - ScratchBytesRead: 0 - ScratchBytesReadUseLocalDisk: 0 - ScratchBytesReadUseMem: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchReadsUseLocalDisk: 0 (0) - ScratchReadsUseMem: 0 (0) - ScratchWrites: 0 (0) - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns - TotalTime: 0.000ns - UncompressedScratchBytesWritten: 0 Buffer pool: - AllocTime: 0.000ns - CompressionTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - EncryptionTime: 0.000ns - InactiveTotalTime: 0.000ns - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - SystemAllocTime: 0.000ns - TotalTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns JVM: - GcCount: 0 (0) - GcNumInfoThresholdExceeded: 0 (0) - GcNumWarnThresholdExceeded: 0 (0) - GcTimeMillis: 0 - GcTotalExtraSleepTimeMillis: 0 - InactiveTotalTime: 0.000ns - TotalTime: 0.000ns Fragment F00: - InactiveTotalTime: 0.000ns - TotalTime: 0.000ns joemcdonnell-22743:27001: - AdmissionSlots: 1 (1) - BloomFilterBytes: 0 - InListFilterItems: 0 (0) - InactiveTotalTime: 0.000ns - ScratchBytesRead: 0 - ScratchBytesReadUseLocalDisk: 0 - ScratchBytesReadUseMem: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchReadsUseLocalDisk: 0 (0) - ScratchReadsUseMem: 0 (0) - ScratchWrites: 0 (0) - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns - TotalTime: 0.000ns - UncompressedScratchBytesWritten: 0 Buffer pool: - AllocTime: 0.000ns - CompressionTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - EncryptionTime: 0.000ns - InactiveTotalTime: 0.000ns - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - SystemAllocTime: 0.000ns - TotalTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns JVM: - GcCount: 0 (0) - GcNumInfoThresholdExceeded: 0 (0) - GcNumWarnThresholdExceeded: 0 (0) - GcTimeMillis: 0 - GcTotalExtraSleepTimeMillis: 0 - InactiveTotalTime: 0.000ns - TotalTime: 0.000ns Fragment F00: - InactiveTotalTime: 0.000ns - TotalTime: 0.000ns joemcdonnell-22743:27000: - AdmissionSlots: 1 (1) - BloomFilterBytes: 0 - InListFilterItems: 0 (0) - InactiveTotalTime: 0.000ns - ScratchBytesRead: 0 - ScratchBytesReadUseLocalDisk: 0 - ScratchBytesReadUseMem: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchReadsUseLocalDisk: 0 (0) - ScratchReadsUseMem: 0 (0) - ScratchWrites: 0 (0) - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns - TotalTime: 0.000ns - UncompressedScratchBytesWritten: 0 Buffer pool: - AllocTime: 0.000ns - CompressionTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - EncryptionTime: 0.000ns - InactiveTotalTime: 0.000ns - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - SystemAllocTime: 0.000ns - TotalTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns JVM: - GcCount: 0 (0) - GcNumInfoThresholdExceeded: 0 (0) - GcNumWarnThresholdExceeded: 0 (0) - GcTimeMillis: 0 - GcTotalExtraSleepTimeMillis: 0 - InactiveTotalTime: 0.000ns - TotalTime: 0.000ns Fragment F00: - InactiveTotalTime: 0.000ns - TotalTime: 0.000ns Averaged Fragment F00 [3 instances]:(Total: 64.286ms, non-child: 142.632us, % non-child: 0.22%) split sizes: min: 158.66 KB, max: 161.09 KB, avg: 159.48 KB, stddev: 1.14 KB completion times: min:56.499ms max:72.974ms mean: 65.112ms stddev:6.746ms execution rates: min:2.12 MB/sec max:2.78 MB/sec mean:2.42 MB/sec stddev:280.47 KB/sec num instances: 3 - AverageThreadTokens: 0.00 - BytesAssigned: 159.48 KB (163311) - CompletionTime: 65.112ms - ExchangeScanRatio: 0.00 - ExecutionRate: 2.42 MB/sec - InactiveTotalTime: 0.000ns - PeakMemoryUsage: 2.19 MB (2298538) - PeakReservation: 256.00 KB (262144) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 2.20 MB (2302634) - RowsProduced: 2.43K (2433) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 23.637ms - TotalThreadsInvoluntaryContextSwitches: 1 (1) - TotalThreadsTotalWallClockTime: 99.912ms - TotalThreadsSysTime: 4.779ms - TotalThreadsUserTime: 20.776ms - TotalThreadsVoluntaryContextSwitches: 110 (110) - TotalTime: 64.286ms Fragment Instance Lifecycle Timings [3 instances]: - ExecTime: 64.195ms - ExecTreeExecTime: 3.157ms - InactiveTotalTime: 0.000ns - OpenTime: 30.729us - ExecTreeOpenTime: 28.415us - PrepareTime: 45.809us - ExecTreePrepareTime: 23.465us - TotalTime: 0.000ns HdfsTableSink [3 instances]:(Total: 60.919ms, non-child: 60.919ms, % non-child: 100.00%) - BytesWritten: 212.84 KB (217953) - CompressTimer: 0.000ns - EncodeTimer: 2.756ms - FilesCreated: 1 (1) - FinalizePartitionFileTimer: 37.230ms - HdfsWriteTimer: 1.269ms - InactiveTotalTime: 0.000ns - PartitionsCreated: 1 (1) - PeakMemoryUsage: 50.00 KB (51200) - RowsInserted: 2.43K (2433) - TmpFileCreateTimer: 19.563ms - TotalTime: 60.919ms HDFS_SCAN_NODE (id=0) [3 instances]:(Total: 3.224ms, non-child: 3.224ms, % non-child: 100.00%) - InitialRangeActualReservation: (Avg: 32.00 KB (32768) ; Min: 32.00 KB (32768) ; Max: 32.00 KB (32768) ; Number of samples: 24) - InitialRangeIdealReservation: (Avg: 32.00 KB (32768) ; Min: 32.00 KB (32768) ; Max: 32.00 KB (32768) ; Number of samples: 24) - ParquetCompressedBytesReadPerColumn: 0 (Number of samples: 0) - ParquetUncompressedBytesReadPerColumn: 0 (Number of samples: 0) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 159.48 KB (163311) - BytesReadDataNodeCache: 0 - BytesReadEncrypted: 0 - BytesReadErasureCoded: 0 - BytesReadLocal: 159.48 KB (163311) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 159.48 KB (163311) - CachedFileHandlesHitCount: 8 (8) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0.000ns - DelimiterParseTime: 810.065us - InactiveTotalTime: 0.000ns - MaterializeTupleTime: 2.388ms - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 8 (8) - PeakMemoryUsage: 1.90 MB (1987242) - PeakScannerThreadConcurrency: 8 (8) - PerReadThreadRawHdfsThroughput: 6.57 MB/sec - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 1.95 MB (2048000) - RowBatchQueueGetWaitTime: 2.379ms - RowBatchQueuePeakMemoryUsage: 1.83 MB (1918976) - RowBatchQueuePutWaitTime: 0.000ns - RowBatchesEnqueued: 16 (16) - RowsRead: 2.43K (2433) - RowsReturned: 2.43K (2433) - RowsReturnedRate: 765.79 K/sec - ScanRangesComplete: 8 (8) - ScannerIoWaitTime: 23.637ms - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 35.703ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 2.877ms - ScannerThreadsVoluntaryContextSwitches: 69 (69) - TotalRawHdfsOpenFileTime: 16.544us - TotalRawHdfsReadTime: 24.468ms - TotalReadThroughput: 0.00 /sec - TotalTime: 3.224ms Buffer pool [3 instances]: - AllocTime: 64.260us - CompressionTime: 0.000ns - CumulativeAllocationBytes: 256.00 KB (262144) - CumulativeAllocations: 8 (8) - EncryptionTime: 0.000ns - InactiveTotalTime: 0.000ns - PeakReservation: 256.00 KB (262144) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 256.00 KB (262144) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 54.773us - TotalTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment F00: - InactiveTotalTime: 0.000ns - TotalTime: 0.000ns Instance 7347b5216e8840b9:a19c3b0800000001 (host=joemcdonnell-22743:27001): Last report received time: 2023-03-23 10:23:12.894 Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:8/158.70 KB Fragment Instance Lifecycle Event Timeline: 72.094ms - Prepare Finished: 230.946us (230.946us) - Open Finished: 263.473us (32.527us) - First Batch Produced: 2.811ms (2.548ms) - First Batch Sent: 24.437ms (21.626ms) - ExecInternal Finished: 72.094ms (47.656ms) - AverageThreadTokens: 0.00 - BytesAssigned: 158.70 KB (162508) - CompletionTime: 72.974ms - ExchangeScanRatio: 0.00 - ExecutionRate: 2.12 MB/sec - InactiveTotalTime: 0.000ns - PeakMemoryUsage: 2.19 MB (2295808) - PeakReservation: 256.00 KB (262144) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 2.19 MB (2299904) - RowsProduced: 2.42K (2420) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 21.323ms - TotalThreadsInvoluntaryContextSwitches: 1 (1) - TotalThreadsTotalWallClockTime: 104.640ms - TotalThreadsSysTime: 7.066ms - TotalThreadsUserTime: 24.717ms - TotalThreadsVoluntaryContextSwitches: 115 (115) - TotalTime: 71.861ms Fragment Instance Lifecycle Timings: - ExecTime: 71.769ms - ExecTreeExecTime: 2.651ms - InactiveTotalTime: 0.000ns - OpenTime: 31.400us - ExecTreeOpenTime: 29.195us - PrepareTime: 40.025us - ExecTreePrepareTime: 20.028us - TotalTime: 0.000ns HdfsTableSink: - BytesWritten: 211.77 KB (216854) - CompressTimer: 0.000ns - EncodeTimer: 3.222ms - FilesCreated: 1 (1) - FinalizePartitionFileTimer: 43.113ms - HdfsWriteTimer: 1.308ms - InactiveTotalTime: 0.000ns - PartitionsCreated: 1 (1) - PeakMemoryUsage: 50.00 KB (51200) - RowsInserted: 2.42K (2420) - TmpFileCreateTimer: 21.280ms - TotalTime: 69.031ms HDFS_SCAN_NODE (id=0): Table Name: functional.alltypes Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:8/158.70 KB Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0% 29:0% File Formats: TEXT/NONE:8 ExecOption: Codegen enabled: 0 out of 8, Codegen Disabled: disabled due to optimization hints Node Lifecycle Event Timeline: 28.907ms - Open Started: 231.753us (231.753us) - Open Finished: 260.692us (28.939us) - First Batch Requested: 264.767us (4.075us) - First Batch Returned: 2.811ms (2.546ms) - Last Batch Returned: 28.846ms (26.035ms) - Closed: 28.907ms (61.299us) - InitialRangeActualReservation: (Avg: 32.00 KB (32768) ; Min: 32.00 KB (32768) ; Max: 32.00 KB (32768) ; Number of samples: 8) - InitialRangeIdealReservation: (Avg: 32.00 KB (32768) ; Min: 32.00 KB (32768) ; Max: 32.00 KB (32768) ; Number of samples: 8) - ParquetCompressedBytesReadPerColumn: 0 (Number of samples: 0) - ParquetUncompressedBytesReadPerColumn: 0 (Number of samples: 0) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 158.70 KB (162508) - BytesReadDataNodeCache: 0 - BytesReadEncrypted: 0 - BytesReadErasureCoded: 0 - BytesReadLocal: 158.70 KB (162508) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 158.70 KB (162508) - CachedFileHandlesHitCount: 8 (8) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0.000ns - DelimiterParseTime: 1.127ms - InactiveTotalTime: 0.000ns - MaterializeTupleTime: 2.794ms - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 8 (8) - PeakMemoryUsage: 1.89 MB (1984512) - PeakScannerThreadConcurrency: 8 (8) - PerReadThreadRawHdfsThroughput: 7.03 MB/sec - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 1.95 MB (2048000) - RowBatchQueueGetWaitTime: 2.102ms - RowBatchQueuePeakMemoryUsage: 1.83 MB (1918976) - RowBatchQueuePutWaitTime: 0.000ns - RowBatchesEnqueued: 16 (16) - RowsRead: 2.42K (2420) - RowsReturned: 2.42K (2420) - RowsReturnedRate: 891.46 K/sec - ScanRangesComplete: 8 (8) - ScannerIoWaitTime: 21.323ms - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 32.857ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 3.518ms - ScannerThreadsVoluntaryContextSwitches: 72 (72) - TotalRawHdfsOpenFileTime: 14.709us - TotalRawHdfsReadTime: 22.048ms - TotalReadThroughput: 0.00 /sec - TotalTime: 2.714ms Buffer pool: - AllocTime: 67.275us - CompressionTime: 0.000ns - CumulativeAllocationBytes: 256.00 KB (262144) - CumulativeAllocations: 8 (8) - EncryptionTime: 0.000ns - InactiveTotalTime: 0.000ns - PeakReservation: 256.00 KB (262144) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 256.00 KB (262144) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 57.286us - TotalTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Instance 7347b5216e8840b9:a19c3b0800000002 (host=joemcdonnell-22743:27000): Last report received time: 2023-03-23 10:23:12.887 Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:8/158.66 KB Fragment Instance Lifecycle Event Timeline: 65.209ms - Prepare Finished: 196.565us (196.565us) - Open Finished: 226.672us (30.107us) - First Batch Produced: 3.357ms (3.131ms) - First Batch Sent: 22.388ms (19.030ms) - ExecInternal Finished: 65.209ms (42.820ms) - AverageThreadTokens: 0.00 - BytesAssigned: 158.66 KB (162470) - CompletionTime: 65.863ms - ExchangeScanRatio: 0.00 - ExecutionRate: 2.35 MB/sec - InactiveTotalTime: 0.000ns - PeakMemoryUsage: 2.23 MB (2336768) - PeakReservation: 256.00 KB (262144) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 2.23 MB (2340864) - RowsProduced: 2.42K (2420) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 18.957ms - TotalThreadsInvoluntaryContextSwitches: 1 (1) - TotalThreadsTotalWallClockTime: 97.820ms - TotalThreadsSysTime: 3.410ms - TotalThreadsUserTime: 20.337ms - TotalThreadsVoluntaryContextSwitches: 126 (126) - TotalTime: 65.056ms Fragment Instance Lifecycle Timings: - ExecTime: 64.968ms - ExecTreeExecTime: 3.232ms - InactiveTotalTime: 0.000ns - OpenTime: 28.806us - ExecTreeOpenTime: 26.563us - PrepareTime: 48.545us - ExecTreePrepareTime: 26.978us - TotalTime: 0.000ns HdfsTableSink: - BytesWritten: 211.72 KB (216804) - CompressTimer: 0.000ns - EncodeTimer: 2.420ms - FilesCreated: 1 (1) - FinalizePartitionFileTimer: 39.087ms - HdfsWriteTimer: 1.314ms - InactiveTotalTime: 0.000ns - PartitionsCreated: 1 (1) - PeakMemoryUsage: 50.00 KB (51200) - RowsInserted: 2.42K (2420) - TmpFileCreateTimer: 18.674ms - TotalTime: 61.621ms HDFS_SCAN_NODE (id=0): Table Name: functional.alltypes Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:8/158.66 KB Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0% 29:0% File Formats: TEXT/NONE:8 ExecOption: Codegen enabled: 0 out of 8, Codegen Disabled: disabled due to optimization hints Node Lifecycle Event Timeline: 26.097ms - Open Started: 197.377us (197.377us) - Open Finished: 223.670us (26.293us) - First Batch Requested: 227.227us (3.557us) - First Batch Returned: 3.356ms (3.128ms) - Last Batch Returned: 26.029ms (22.673ms) - Closed: 26.097ms (67.837us) - InitialRangeActualReservation: (Avg: 32.00 KB (32768) ; Min: 32.00 KB (32768) ; Max: 32.00 KB (32768) ; Number of samples: 8) - InitialRangeIdealReservation: (Avg: 32.00 KB (32768) ; Min: 32.00 KB (32768) ; Max: 32.00 KB (32768) ; Number of samples: 8) - ParquetCompressedBytesReadPerColumn: 0 (Number of samples: 0) - ParquetUncompressedBytesReadPerColumn: 0 (Number of samples: 0) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 158.66 KB (162470) - BytesReadDataNodeCache: 0 - BytesReadEncrypted: 0 - BytesReadErasureCoded: 0 - BytesReadLocal: 158.66 KB (162470) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 158.66 KB (162470) - CachedFileHandlesHitCount: 8 (8) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0.000ns - DelimiterParseTime: 498.655us - InactiveTotalTime: 0.000ns - MaterializeTupleTime: 1.946ms - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 8 (8) - PeakMemoryUsage: 1.93 MB (2025472) - PeakScannerThreadConcurrency: 8 (8) - PerReadThreadRawHdfsThroughput: 7.62 MB/sec - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 1.95 MB (2048000) - RowBatchQueueGetWaitTime: 1.907ms - RowBatchQueuePeakMemoryUsage: 1.83 MB (1918976) - RowBatchQueuePutWaitTime: 0.000ns - RowBatchesEnqueued: 16 (16) - RowsRead: 2.42K (2420) - RowsReturned: 2.42K (2420) - RowsReturnedRate: 733.17 K/sec - ScanRangesComplete: 8 (8) - ScannerIoWaitTime: 18.957ms - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 32.838ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 3.286ms - ScannerThreadsVoluntaryContextSwitches: 81 (81) - TotalRawHdfsOpenFileTime: 17.899us - TotalRawHdfsReadTime: 20.333ms - TotalReadThroughput: 0.00 /sec - TotalTime: 3.300ms Buffer pool: - AllocTime: 71.777us - CompressionTime: 0.000ns - CumulativeAllocationBytes: 256.00 KB (262144) - CumulativeAllocations: 8 (8) - EncryptionTime: 0.000ns - InactiveTotalTime: 0.000ns - PeakReservation: 256.00 KB (262144) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 256.00 KB (262144) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 62.523us - TotalTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Instance 7347b5216e8840b9:a19c3b0800000000 (host=joemcdonnell-22743:27002): Last report received time: 2023-03-23 10:23:12.878 Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:8/161.09 KB Fragment Instance Lifecycle Event Timeline: 56.055ms - Prepare Finished: 155.414us (155.414us) - Open Finished: 188.404us (32.990us) - First Batch Produced: 3.687ms (3.498ms) - First Batch Sent: 22.765ms (19.078ms) - ExecInternal Finished: 56.055ms (33.289ms) - AverageThreadTokens: 0.00 - BytesAssigned: 161.09 KB (164956) - CompletionTime: 56.499ms - ExchangeScanRatio: 0.00 - ExecutionRate: 2.78 MB/sec - InactiveTotalTime: 0.000ns - PeakMemoryUsage: 2.16 MB (2263040) - PeakReservation: 256.00 KB (262144) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 2.16 MB (2267136) - RowsProduced: 2.46K (2460) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 30.633ms - TotalThreadsInvoluntaryContextSwitches: 3 (3) - TotalThreadsTotalWallClockTime: 97.276ms - TotalThreadsSysTime: 3.861ms - TotalThreadsUserTime: 17.274ms - TotalThreadsVoluntaryContextSwitches: 89 (89) - TotalTime: 55.940ms Fragment Instance Lifecycle Timings: - ExecTime: 55.848ms - ExecTreeExecTime: 3.588ms - InactiveTotalTime: 0.000ns - OpenTime: 31.981us - ExecTreeOpenTime: 29.487us - PrepareTime: 48.858us - ExecTreePrepareTime: 23.389us - TotalTime: 0.000ns HdfsTableSink: - BytesWritten: 215.04 KB (220202) - CompressTimer: 0.000ns - EncodeTimer: 2.628ms - FilesCreated: 1 (1) - FinalizePartitionFileTimer: 29.489ms - HdfsWriteTimer: 1.186ms - InactiveTotalTime: 0.000ns - PartitionsCreated: 1 (1) - PeakMemoryUsage: 50.00 KB (51200) - RowsInserted: 2.46K (2460) - TmpFileCreateTimer: 18.734ms - TotalTime: 52.106ms HDFS_SCAN_NODE (id=0): Table Name: functional.alltypes Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:8/161.09 KB Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0% 29:0% File Formats: TEXT/NONE:8 ExecOption: Codegen enabled: 0 out of 8, Codegen Disabled: disabled due to optimization hints Node Lifecycle Event Timeline: 26.538ms - Open Started: 156.306us (156.306us) - Open Finished: 185.455us (29.149us) - First Batch Requested: 190.781us (5.326us) - First Batch Returned: 3.686ms (3.495ms) - Last Batch Returned: 26.481ms (22.794ms) - Closed: 26.538ms (57.182us) - InitialRangeActualReservation: (Avg: 32.00 KB (32768) ; Min: 32.00 KB (32768) ; Max: 32.00 KB (32768) ; Number of samples: 8) - InitialRangeIdealReservation: (Avg: 32.00 KB (32768) ; Min: 32.00 KB (32768) ; Max: 32.00 KB (32768) ; Number of samples: 8) - ParquetCompressedBytesReadPerColumn: 0 (Number of samples: 0) - ParquetUncompressedBytesReadPerColumn: 0 (Number of samples: 0) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 161.09 KB (164956) - BytesReadDataNodeCache: 0 - BytesReadEncrypted: 0 - BytesReadErasureCoded: 0 - BytesReadLocal: 161.09 KB (164956) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 161.09 KB (164956) - CachedFileHandlesHitCount: 8 (8) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0.000ns - DelimiterParseTime: 803.807us - InactiveTotalTime: 0.000ns - MaterializeTupleTime: 2.425ms - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 8 (8) - PeakMemoryUsage: 1.86 MB (1951744) - PeakScannerThreadConcurrency: 8 (8) - PerReadThreadRawHdfsThroughput: 5.07 MB/sec - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 1.95 MB (2048000) - RowBatchQueueGetWaitTime: 3.128ms - RowBatchQueuePeakMemoryUsage: 1.83 MB (1918976) - RowBatchQueuePutWaitTime: 0.000ns - RowBatchesEnqueued: 16 (16) - RowsRead: 2.46K (2460) - RowsReturned: 2.46K (2460) - RowsReturnedRate: 672.75 K/sec - ScanRangesComplete: 8 (8) - ScannerIoWaitTime: 30.633ms - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 41.413ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 1.829ms - ScannerThreadsVoluntaryContextSwitches: 55 (55) - TotalRawHdfsOpenFileTime: 17.024us - TotalRawHdfsReadTime: 31.023ms - TotalReadThroughput: 0.00 /sec - TotalTime: 3.656ms Buffer pool: - AllocTime: 53.731us - CompressionTime: 0.000ns - CumulativeAllocationBytes: 256.00 KB (262144) - CumulativeAllocations: 8 (8) - EncryptionTime: 0.000ns - InactiveTotalTime: 0.000ns - PeakReservation: 256.00 KB (262144) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 256.00 KB (262144) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 44.512us - TotalTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns