+--------------+----+-----+----------------+-----+-----+ | table_source | c1 | c2 | table_source_a | c1a | c2a | +--------------+----+-----+----------------+-----+-----+ | ONE | 1 | one | ONE | 1 | one | +--------------+----+-----+----------------+-----+-----+ Query Runtime Profile: Query (id=fe43fc7b5ea71656:4a5d2eea00000000): DEBUG MODE WARNING: Query profile created while running a DEBUG build of Impala. Use RELEASE builds to measure query performance. Summary: Session ID: 494013ae21652e8f:8a5efefc3301df80 Session Type: BEESWAX Start Time: 2020-01-15 15:06:33.271537000 End Time: 2020-01-15 15:06:33.703091000 Query Type: QUERY Query State: FINISHED Impala Query State: FINISHED Query Status: OK Impala Version: impalad version 3.4.0-SNAPSHOT DEBUG (build 6e3d1216f1680e7fca4b5979f182a19673018a2f) User: fangyurao Connected User: fangyurao Delegated User: Network Address: ::ffff:127.0.0.1:51420 Default Db: default Sql Statement: select * from default.myview_1_on_2_parquet_tables a, myview_2_on_2_parquet_tables b where a.table_source = 'ONE' and a.table_source = b.table_source_a Coordinator: fangyu-upstream-fe-dev.gce.cloudera.com:22000 Query Options (set by configuration): TIMEZONE=America/Los_Angeles,CLIENT_IDENTIFIER=Impala Shell v3.4.0-SNAPSHOT (6e3d121) built on Thu Jan 9 08:30:23 PST 2020 Query Options (set by configuration and planner): MT_DOP=0,TIMEZONE=America/Los_Angeles,CLIENT_IDENTIFIER=Impala Shell v3.4.0-SNAPSHOT (6e3d121) built on Thu Jan 9 08:30:23 PST 2020 Plan: ---------------- Max Per-Host Resource Reservation: Memory=1.97MB Threads=5 Per-Host Resource Estimates: Memory=66MB WARNING: The following tables are missing relevant table and/or column statistics. default.pt1, default.pta1, default.pta2 Analyzed query: SELECT * FROM `default`.myview_1_on_2_parquet_tables a, `default`.myview_2_on_2_parquet_tables b WHERE a.table_source = 'ONE' AND a.table_source = b.table_source_a F05:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=42.66KB mem-reservation=0B thread-reservation=1 PLAN-ROOT SINK | output exprs: table_source, c1, c2, table_source_a, c1a, c2a | mem-estimate=0B mem-reservation=0B thread-reservation=0 | 07:EXCHANGE [UNPARTITIONED] | mem-estimate=42.66KB mem-reservation=0B thread-reservation=0 | tuple-ids=6,2 row-size=56B cardinality=364 | in pipelines: 03(GETNEXT), 04(GETNEXT) | F02:PLAN FRAGMENT [RANDOM] hosts=1 instances=1 Per-Host Resources: mem-estimate=33.95MB mem-reservation=1.95MB thread-reservation=2 05:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: table_source_a = table_source | fk/pk conjuncts: assumed fk/pk | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 | tuple-ids=6,2 row-size=56B cardinality=364 | in pipelines: 03(GETNEXT), 04(GETNEXT), 01(OPEN) | |--06:EXCHANGE [BROADCAST] | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0 | | tuple-ids=2 row-size=28B cardinality=180 | | in pipelines: 01(GETNEXT) | | | F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1 | Per-Host Resources: mem-estimate=32.00MB mem-reservation=16.00KB thread-reservation=2 | 00:UNION | | mem-estimate=0B mem-reservation=0B thread-reservation=0 | | tuple-ids=2 row-size=28B cardinality=180 | | in pipelines: 01(GETNEXT) | | | 01:SCAN HDFS [default.pt1, RANDOM] | HDFS partitions=1/1 files=1 size=579B | stored statistics: | table: rows=unavailable size=unavailable | columns: unavailable | extrapolated-rows=disabled max-scan-range-rows=unavailable | mem-estimate=32.00MB mem-reservation=16.00KB thread-reservation=1 | tuple-ids=0 row-size=16B cardinality=180 | in pipelines: 01(GETNEXT) | 02:UNION | mem-estimate=0B mem-reservation=0B thread-reservation=0 | tuple-ids=6 row-size=28B cardinality=364 | in pipelines: 03(GETNEXT), 04(GETNEXT) | |--04:SCAN HDFS [default.pta2, RANDOM] | HDFS partitions=1/1 files=1 size=585B | stored statistics: | table: rows=unavailable size=unavailable | columns: unavailable | extrapolated-rows=disabled max-scan-range-rows=unavailable | mem-estimate=32.00MB mem-reservation=16.00KB thread-reservation=1 | tuple-ids=5 row-size=16B cardinality=182 | in pipelines: 04(GETNEXT) | 03:SCAN HDFS [default.pta1, RANDOM] HDFS partitions=1/1 files=1 size=585B stored statistics: table: rows=unavailable size=unavailable columns: unavailable extrapolated-rows=disabled max-scan-range-rows=unavailable mem-estimate=32.00MB mem-reservation=16.00KB thread-reservation=1 tuple-ids=4 row-size=16B cardinality=182 in pipelines: 03(GETNEXT) ---------------- Estimated Per-Host Mem: 69200544 Tables Missing Stats: default.pt1,default.pta1,default.pta2 Request Pool: default-pool Per Host Min Memory Reservation: fangyu-upstream-fe-dev.gce.cloudera.com:22001(1.97 MB) fangyu-upstream-fe-dev.gce.cloudera.com:22000(0) Per Host Number of Fragment Instances: fangyu-upstream-fe-dev.gce.cloudera.com:22001(2) fangyu-upstream-fe-dev.gce.cloudera.com:22000(1) Admission result: Admitted immediately Cluster Memory Admitted: 131.99 MB Executor Group: default ExecSummary: Operator #Hosts #Inst Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ------------------------------------------------------------------------------------------------------------------------------ F05:ROOT 1 1 0.000ns 0.000ns 0 0 07:EXCHANGE 1 1 0.000ns 0.000ns 1 364 16.00 KB 42.66 KB UNPARTITIONED F02:EXCHANGE SENDER 1 1 4.000ms 4.000ms 4.41 KB 0 05:HASH JOIN 1 1 4.000ms 4.000ms 1 364 2.06 MB 1.94 MB INNER JOIN, BROADCAST |--06:EXCHANGE 1 1 0.000ns 0.000ns 1 180 16.00 KB 16.00 KB BROADCAST | F04:EXCHANGE SENDER 1 1 0.000ns 0.000ns 4.41 KB 0 | 00:UNION 1 1 0.000ns 0.000ns 1 180 29.00 KB 0 | 01:SCAN HDFS 1 1 4.000ms 4.000ms 1 180 61.02 KB 32.00 MB default.pt1 02:UNION 1 1 0.000ns 0.000ns 2 364 29.00 KB 0 |--04:SCAN HDFS 1 1 4.000ms 4.000ms 1 182 69.00 KB 32.00 MB default.pta2 03:SCAN HDFS 1 1 8.000ms 8.000ms 1 182 69.00 KB 32.00 MB default.pta1 Errors: Query Compilation: 12.526ms - Metadata of all 6 tables cached: 667.226us (667.226us) - Analysis finished: 3.136ms (2.469ms) - Authorization finished (noop): 3.357ms (221.531us) - Value transfer graph computed: 3.791ms (433.546us) - Single node plan created: 9.156ms (5.364ms) - Runtime filters computed: 9.425ms (269.505us) - Distributed plan created: 9.476ms (50.659us) - Planning finished: 12.526ms (3.049ms) Query Timeline: 440.038ms - Query submitted: 0.000ns (0.000ns) - Planning finished: 28.002ms (28.002ms) - Submit for admission: 28.002ms (0.000ns) - Completed admission: 28.002ms (0.000ns) - Ready to start on 2 backends: 28.002ms (0.000ns) - All 2 execution backends (3 fragment instances) started: 28.002ms (0.000ns) - Rows available: 400.035ms (372.032ms) - First row fetched: 428.037ms (28.002ms) - Last row fetched: 428.037ms (0.000ns) - Released admission control resources: 432.037ms (4.000ms) - Unregister query: 436.038ms (4.000ms) - AdmissionControlTimeSinceLastUpdate: 44.000ms - ComputeScanRangeAssignmentTimer: 0.000ns Frontend: ImpalaServer: - ClientFetchWaitTimer: 32.002ms - NumRowsFetched: 1 (1) - NumRowsFetchedFromCache: 0 (0) - RowMaterializationRate: 249.00 /sec - RowMaterializationTimer: 4.000ms Execution Profile fe43fc7b5ea71656:4a5d2eea00000000:(Total: 376.032ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- Backend startup latencies: Count: 2, min / max: 1ms / 1ms, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 1ms Slowest backend to start up: fangyu-upstream-fe-dev.gce.cloudera.com:22001 Per Node Peak Memory Usage: fangyu-upstream-fe-dev.gce.cloudera.com:22001(3.94 MB) fangyu-upstream-fe-dev.gce.cloudera.com:22000(98.00 KB) Per Node Bytes Read: fangyu-upstream-fe-dev.gce.cloudera.com:22001(1.99 KB) fangyu-upstream-fe-dev.gce.cloudera.com:22000(0) Per Node User Time: fangyu-upstream-fe-dev.gce.cloudera.com:22001(408.000ms) fangyu-upstream-fe-dev.gce.cloudera.com:22000(28.000ms) Per Node System Time: fangyu-upstream-fe-dev.gce.cloudera.com:22001(8.000ms) fangyu-upstream-fe-dev.gce.cloudera.com:22000(4.000ms) - ExchangeScanRatio: 0.05 - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns - InnerNodeSelectivityRatio: 0.00 - NumBackends: 2 (2) - NumCompletedBackends: 2 (2) - NumFragmentInstances: 3 (3) - NumFragments: 3 (3) - TotalBytesRead: 1.99 KB (2040) - TotalBytesSent: 92.00 B (92) - TotalCpuTime: 448.000ms - TotalInnerBytesSent: 0 - TotalScanBytesSent: 92.00 B (92) Per Node Profiles: fangyu-upstream-fe-dev.gce.cloudera.com:22001: - AdmissionSlots: 1 (1) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns fangyu-upstream-fe-dev.gce.cloudera.com:22000: - AdmissionSlots: 1 (1) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns Averaged Fragment F05:(Total: 400.035ms, non-child: 0.000ns, % non-child: 0.00%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:404.035ms max:404.035ms mean: 404.035ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - ExchangeScanRatio: 0.00 - PeakMemoryUsage: 98.00 KB (100352) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 98.00 KB (100352) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 340.029ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 400.035ms - TotalThreadsSysTime: 4.000ms - TotalThreadsUserTime: 28.000ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 28.002ms - ExecTreeExecTime: 0.000ns - OpenTime: 372.032ms - ExecTreeOpenTime: 340.029ms - PrepareTime: 0.000ns - ExecTreePrepareTime: 0.000ns PLAN_ROOT_SINK:(Total: 28.002ms, non-child: 28.002ms, % non-child: 100.00%) - PeakMemoryUsage: 0 - RowsSent: 1 (1) - RowsSentRate: 35.00 /sec EXCHANGE_NODE (id=7):(Total: 340.029ms, non-child: 340.029ms, % non-child: 100.00%) - ConvertRowBatchTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - RowsReturnedRate: 2.00 /sec Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 16.00 KB (16384) - CumulativeAllocations: 2 (2) - PeakReservation: 16.00 KB (16384) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 16.00 KB (16384) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Dequeue: - FirstBatchWaitTime: 340.029ms - TotalBytesDequeued: 86.00 B (86) - TotalGetBatchTime: 340.029ms - DataWaitTime: 340.029ms Enqueue: - DeserializeRowBatchTime: 0.000ns - TotalBatchesEnqueued: 1 (1) - TotalBatchesReceived: 1 (1) - TotalBytesReceived: 60.00 B (60) - TotalEarlySenders: 0 (0) - TotalEosReceived: 1 (1) - TotalHasDeferredRPCsTime: 0.000ns - TotalRPCsDeferred: 0 (0) CodeGen:(Total: 32.002ms, non-child: 32.002ms, % non-child: 100.00%) - CodegenInvoluntaryContextSwitches: 0 (0) - CodegenTotalWallClockTime: 32.002ms - CodegenSysTime: 0.000ns - CodegenUserTime: 28.000ms - CodegenVoluntaryContextSwitches: 0 (0) - CompileTime: 0.000ns - IrGenerationTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 2.48 MB (2604708) - NumFunctions: 6 (6) - NumInstructions: 164 (164) - OptimizationTime: 12.001ms - PeakMemoryUsage: 82.00 KB (83968) - PrepareTime: 20.001ms Coordinator Fragment F05: Instance fe43fc7b5ea71656:4a5d2eea00000000 (host=fangyu-upstream-fe-dev.gce.cloudera.com:22000):(Total: 400.035ms, non-child: 0.000ns, % non-child: 0.00%) Last report received time: 2020-01-15 15:06:33.702 Fragment Instance Lifecycle Event Timeline: 400.035ms - Prepare Finished: 0.000ns (0.000ns) - Open Finished: 372.032ms (372.032ms) - First Batch Produced: 372.032ms (0.000ns) - First Batch Sent: 400.035ms (28.002ms) - ExecInternal Finished: 400.035ms (0.000ns) - MemoryUsage (500.000ms): 32.67 KB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - ExchangeScanRatio: 0.00 - PeakMemoryUsage: 98.00 KB (100352) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 98.00 KB (100352) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 340.029ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 400.035ms - TotalThreadsSysTime: 4.000ms - TotalThreadsUserTime: 28.000ms - TotalThreadsVoluntaryContextSwitches: 3 (3) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 28.002ms - ExecTreeExecTime: 0.000ns - OpenTime: 372.032ms - ExecTreeOpenTime: 340.029ms - PrepareTime: 0.000ns - ExecTreePrepareTime: 0.000ns PLAN_ROOT_SINK:(Total: 28.002ms, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 0 - RowsSent: 1 (1) - RowsSentRate: 35.00 /sec EXCHANGE_NODE (id=7):(Total: 340.029ms, non-child: 0.000ns, % non-child: 0.00%) Node Lifecycle Event Timeline: 400.035ms - Open Started: 32.002ms (32.002ms) - Open Finished: 372.032ms (340.029ms) - First Batch Requested: 372.032ms (0.000ns) - First Batch Returned: 372.032ms (0.000ns) - Last Batch Returned: 372.032ms (0.000ns) - Closed: 400.035ms (28.002ms) - ConvertRowBatchTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - RowsReturnedRate: 2.00 /sec Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 16.00 KB (16384) - CumulativeAllocations: 2 (2) - PeakReservation: 16.00 KB (16384) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 16.00 KB (16384) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Dequeue: - BytesDequeued (500.000ms): 86.00 B - FirstBatchWaitTime: 340.029ms - TotalBytesDequeued: 86.00 B (86) - TotalGetBatchTime: 340.029ms - DataWaitTime: 340.029ms Enqueue: - BytesReceived (500.000ms): 60.00 B - DeferredQueueSize (500.000ms): 0 - DispatchTime: (Avg: 109.960us ; Min: 109.960us ; Max: 109.960us ; Number of samples: 1) - DeserializeRowBatchTime: 0.000ns - TotalBatchesEnqueued: 1 (1) - TotalBatchesReceived: 1 (1) - TotalBytesReceived: 60.00 B (60) - TotalEarlySenders: 0 (0) - TotalEosReceived: 1 (1) - TotalHasDeferredRPCsTime: 0.000ns - TotalRPCsDeferred: 0 (0) CodeGen:(Total: 32.002ms, non-child: 32.002ms, % non-child: 100.00%) - CodegenInvoluntaryContextSwitches: 0 (0) - CodegenTotalWallClockTime: 32.002ms - CodegenSysTime: 0.000ns - CodegenUserTime: 28.000ms - CodegenVoluntaryContextSwitches: 0 (0) - CompileTime: 0.000ns - IrGenerationTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 2.48 MB (2604708) - NumFunctions: 6 (6) - NumInstructions: 164 (164) - OptimizationTime: 12.001ms - PeakMemoryUsage: 82.00 KB (83968) - PrepareTime: 20.001ms Averaged Fragment F02:(Total: 372.032ms, non-child: 0.000ns, % non-child: 0.00%) split sizes: min: 1.14 KB, max: 1.14 KB, avg: 1.14 KB, stddev: 0 completion times: min:376.032ms max:376.032ms mean: 376.032ms stddev:0.000ns execution rates: min:3.04 KB/sec max:3.04 KB/sec mean:3.04 KB/sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - ExchangeScanRatio: 0.04 - PeakMemoryUsage: 2.19 MB (2296365) - PeakReservation: 1.95 MB (2048000) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 3.94 MB (4136000) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 8.000ms - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 380.033ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 356.000ms - TotalThreadsVoluntaryContextSwitches: 20 (20) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 4.000ms - ExecTreeExecTime: 0.000ns - OpenTime: 364.031ms - ExecTreeOpenTime: 12.001ms - PrepareTime: 4.000ms - ExecTreePrepareTime: 4.000ms KrpcDataStreamSender (dst_id=7):(Total: 4.000ms, non-child: 4.000ms, % non-child: 100.00%) - EosSent: 1 (1) - PeakMemoryUsage: 4.41 KB (4512) - RowsSent: 1 (1) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0.000ns - TotalBytesSent: 60.00 B (60) - UncompressedRowBatchSize: 86.00 B (86) HASH_JOIN_NODE (id=5):(Total: 16.001ms, non-child: 4.000ms, % non-child: 25.00%) - BuildRows: 1 (1) - BuildTime: 4.000ms - PeakMemoryUsage: 2.06 MB (2163968) - ProbeRows: 2 (2) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 1 (1) - RowsReturnedRate: 62.00 /sec Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 128.00 KB (131072) - CumulativeAllocations: 2 (2) - PeakReservation: 1.94 MB (2031616) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 128.00 KB (131072) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Hash Join Builder (join_node_id=5): - BuildRowsPartitionTime: 0.000ns - BuildRowsPartitioned: 1 (1) - HashTablesBuildTime: 0.000ns - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumHashTableBuildsSkipped: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 25.12 KB (25728) - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) Hash Table: - HashBuckets: 1 (1) - HashCollisions: 0 (0) - Probes: 3 (3) - Resizes: 0 (0) - Travel: 1 (1) EXCHANGE_NODE (id=6): - ConvertRowBatchTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - RowsReturnedRate: 0 Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 16.00 KB (16384) - CumulativeAllocations: 2 (2) - PeakReservation: 16.00 KB (16384) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 16.00 KB (16384) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Dequeue: - FirstBatchWaitTime: 0.000ns - TotalBytesDequeued: 43.00 B (43) - TotalGetBatchTime: 0.000ns - DataWaitTime: 0.000ns Enqueue: - DeserializeRowBatchTime: 0.000ns - TotalBatchesEnqueued: 1 (1) - TotalBatchesReceived: 1 (1) - TotalBytesReceived: 32.00 B (32) - TotalEarlySenders: 0 (0) - TotalEosReceived: 1 (1) - TotalHasDeferredRPCsTime: 0.000ns - TotalRPCsDeferred: 0 (0) UNION_NODE (id=2):(Total: 12.001ms, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 29.00 KB (29696) - RowsReturned: 2 (2) - RowsReturnedRate: 166.00 /sec HDFS_SCAN_NODE (id=4):(Total: 4.000ms, non-child: 4.000ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 682.00 B (682) - BytesReadDataNodeCache: 0 - BytesReadLocal: 682.00 B (682) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 682.00 B (682) - CachedFileHandlesHitCount: 3 (3) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0.000ns - MaterializeTupleTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 69.00 KB (70656) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 166.49 KB/sec - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 25.00 KB (25600) - RowBatchQueueGetWaitTime: 4.000ms - RowBatchQueuePeakMemoryUsage: 41.00 KB (41984) - RowBatchQueuePutWaitTime: 0.000ns - RowBatchesEnqueued: 3 (3) - RowsRead: 1 (1) - RowsReturned: 1 (1) - RowsReturnedRate: 249.00 /sec - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 4.000ms - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 4.000ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 5 (5) - TotalRawHdfsOpenFileTime: 0.000ns - TotalRawHdfsReadTime: 4.000ms - TotalReadThroughput: 0.00 /sec Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 24.00 KB (24576) - CumulativeAllocations: 3 (3) - PeakReservation: 16.00 KB (16384) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 16.00 KB (16384) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns HDFS_SCAN_NODE (id=3):(Total: 8.000ms, non-child: 8.000ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 682.00 B (682) - BytesReadDataNodeCache: 0 - BytesReadLocal: 682.00 B (682) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 682.00 B (682) - CachedFileHandlesHitCount: 3 (3) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0.000ns - MaterializeTupleTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 69.00 KB (70656) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 166.49 KB/sec - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 25.00 KB (25600) - RowBatchQueueGetWaitTime: 4.000ms - RowBatchQueuePeakMemoryUsage: 49.00 KB (50176) - RowBatchQueuePutWaitTime: 0.000ns - RowBatchesEnqueued: 3 (3) - RowsRead: 1 (1) - RowsReturned: 1 (1) - RowsReturnedRate: 124.00 /sec - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 4.000ms - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 4.000ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 4 (4) - TotalRawHdfsOpenFileTime: 0.000ns - TotalRawHdfsReadTime: 4.000ms - TotalReadThroughput: 0.00 /sec Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 24.00 KB (24576) - CumulativeAllocations: 3 (3) - PeakReservation: 16.00 KB (16384) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 16.00 KB (16384) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns CodeGen:(Total: 352.030ms, non-child: 352.030ms, % non-child: 100.00%) - CodegenInvoluntaryContextSwitches: 0 (0) - CodegenTotalWallClockTime: 352.030ms - CodegenSysTime: 0.000ns - CodegenUserTime: 352.000ms - CodegenVoluntaryContextSwitches: 0 (0) - CompileTime: 28.002ms - IrGenerationTime: 16.001ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.48 MB (2604708) - NumFunctions: 77 (77) - NumInstructions: 3.46K (3460) - OptimizationTime: 288.025ms - PeakMemoryUsage: 1.69 MB (1771520) - PrepareTime: 20.001ms Fragment F02: Instance fe43fc7b5ea71656:4a5d2eea00000002 (host=fangyu-upstream-fe-dev.gce.cloudera.com:22001):(Total: 372.032ms, non-child: 0.000ns, % non-child: 0.00%) Last report received time: 2020-01-15 15:06:33.674 Hdfs split stats (:<# splits>/): 1:1/585.00 B 0:1/585.00 B Fragment Instance Lifecycle Event Timeline: 372.032ms - Prepare Finished: 4.000ms (4.000ms) - Open Finished: 368.032ms (364.031ms) - First Batch Produced: 368.032ms (0.000ns) - First Batch Sent: 372.032ms (4.000ms) - ExecInternal Finished: 372.032ms (0.000ns) - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - ExchangeScanRatio: 0.04 - PeakMemoryUsage: 2.19 MB (2296365) - PeakReservation: 1.95 MB (2048000) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 3.94 MB (4136000) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 8.000ms - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 380.033ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 356.000ms - TotalThreadsVoluntaryContextSwitches: 20 (20) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 4.000ms - ExecTreeExecTime: 0.000ns - OpenTime: 364.031ms - ExecTreeOpenTime: 12.001ms - PrepareTime: 4.000ms - ExecTreePrepareTime: 4.000ms KrpcDataStreamSender (dst_id=7):(Total: 4.000ms, non-child: 4.000ms, % non-child: 100.00%) ExecOption: Unpartitioned Sender Codegen Disabled: not needed - NetworkThroughput: (Avg: 176.76 KB/sec ; Min: 176.76 KB/sec ; Max: 176.76 KB/sec ; Number of samples: 1) - RpcNetworkTime: (Avg: 218.971us ; Min: 106.463us ; Max: 331.479us ; Number of samples: 2) - RpcRecvrTime: (Avg: 107.057us ; Min: 60.878us ; Max: 153.236us ; Number of samples: 2) - EosSent: 1 (1) - PeakMemoryUsage: 4.41 KB (4512) - RowsSent: 1 (1) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0.000ns - TotalBytesSent: 60.00 B (60) - UncompressedRowBatchSize: 86.00 B (86) HASH_JOIN_NODE (id=5):(Total: 16.001ms, non-child: 4.000ms, % non-child: 25.00%) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously Node Lifecycle Event Timeline: 372.032ms - Open Started: 356.031ms (356.031ms) - Open Finished: 368.032ms (12.001ms) - First Batch Requested: 368.032ms (0.000ns) - First Batch Returned: 368.032ms (0.000ns) - Last Batch Returned: 372.032ms (4.000ms) - Closed: 372.032ms (0.000ns) - BuildRows: 1 (1) - BuildTime: 4.000ms - PeakMemoryUsage: 2.06 MB (2163968) - ProbeRows: 2 (2) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0.000ns - RowsReturned: 1 (1) - RowsReturnedRate: 62.00 /sec Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 128.00 KB (131072) - CumulativeAllocations: 2 (2) - PeakReservation: 1.94 MB (2031616) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 128.00 KB (131072) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Hash Join Builder (join_node_id=5): ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 0.000ns - BuildRowsPartitioned: 1 (1) - HashTablesBuildTime: 0.000ns - LargestPartitionPercent: 100 (100) - MaxPartitionLevel: 0 (0) - NumHashTableBuildsSkipped: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 25.12 KB (25728) - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) Hash Table: - HashBuckets: 1 (1) - HashCollisions: 0 (0) - Probes: 3 (3) - Resizes: 0 (0) - Travel: 1 (1) EXCHANGE_NODE (id=6): Node Lifecycle Event Timeline: 360.031ms - Open Started: 356.031ms (356.031ms) - Open Finished: 356.031ms (0.000ns) - First Batch Requested: 360.031ms (4.000ms) - First Batch Returned: 360.031ms (0.000ns) - Last Batch Returned: 360.031ms (0.000ns) - Closed: 360.031ms (0.000ns) - ConvertRowBatchTime: 0.000ns - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - RowsReturnedRate: 0 Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 16.00 KB (16384) - CumulativeAllocations: 2 (2) - PeakReservation: 16.00 KB (16384) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 16.00 KB (16384) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Dequeue: - FirstBatchWaitTime: 0.000ns - TotalBytesDequeued: 43.00 B (43) - TotalGetBatchTime: 0.000ns - DataWaitTime: 0.000ns Enqueue: - DispatchTime: (Avg: 96.003us ; Min: 96.003us ; Max: 96.003us ; Number of samples: 1) - DeserializeRowBatchTime: 0.000ns - TotalBatchesEnqueued: 1 (1) - TotalBatchesReceived: 1 (1) - TotalBytesReceived: 32.00 B (32) - TotalEarlySenders: 0 (0) - TotalEosReceived: 1 (1) - TotalHasDeferredRPCsTime: 0.000ns - TotalRPCsDeferred: 0 (0) UNION_NODE (id=2):(Total: 12.001ms, non-child: 0.000ns, % non-child: 0.00%) ExecOption: Codegen Enabled Node Lifecycle Event Timeline: 372.032ms - Open Started: 356.031ms (356.031ms) - Open Finished: 360.031ms (4.000ms) - First Batch Requested: 360.031ms (0.000ns) - First Batch Returned: 368.032ms (8.000ms) - Last Batch Returned: 368.032ms (0.000ns) - Closed: 372.032ms (4.000ms) - PeakMemoryUsage: 29.00 KB (29696) - RowsReturned: 2 (2) - RowsReturnedRate: 166.00 /sec HDFS_SCAN_NODE (id=4):(Total: 4.000ms, non-child: 4.000ms, % non-child: 100.00%) Table Name: default.pta2 Hdfs split stats (:<# splits>/): 1:1/585.00 B ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:1 Node Lifecycle Event Timeline: 368.032ms - Open Started: 364.031ms (364.031ms) - Open Finished: 364.031ms (0.000ns) - First Batch Requested: 364.031ms (0.000ns) - First Batch Returned: 368.032ms (4.000ms) - Last Batch Returned: 368.032ms (0.000ns) - Closed: 368.032ms (0.000ns) - FooterProcessingTime: (Avg: 4.000ms ; Min: 4.000ms ; Max: 4.000ms ; Number of samples: 1) - InitialRangeActualReservation: (Avg: 16.00 KB (16384) ; Min: 16.00 KB (16384) ; Max: 16.00 KB (16384) ; Number of samples: 1) - InitialRangeIdealReservation: (Avg: 8.00 KB (8192) ; Min: 8.00 KB (8192) ; Max: 8.00 KB (8192) ; Number of samples: 1) - PageIndexProcessingTime: 0.000ns (Number of samples: 0) - ParquetCompressedBytesReadPerColumn: (Avg: 11.00 B (11) ; Min: 11.00 B (11) ; Max: 11.00 B (11) ; Number of samples: 2) - ParquetCompressedPageSize: (Avg: 11.00 B (11) ; Min: 11.00 B (11) ; Max: 11.00 B (11) ; Number of samples: 2) - ParquetRowGroupActualReservation: (Avg: 16.00 KB (16384) ; Min: 16.00 KB (16384) ; Max: 16.00 KB (16384) ; Number of samples: 1) - ParquetRowGroupIdealReservation: (Avg: 16.00 KB (16384) ; Min: 16.00 KB (16384) ; Max: 16.00 KB (16384) ; Number of samples: 1) - ParquetUncompressedBytesReadPerColumn: (Avg: 9.00 B (9) ; Min: 9.00 B (9) ; Max: 9.00 B (9) ; Number of samples: 2) - ParquetUncompressedPageSize: (Avg: 9.00 B (9) ; Min: 9.00 B (9) ; Max: 9.00 B (9) ; Number of samples: 2) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 682.00 B (682) - BytesReadDataNodeCache: 0 - BytesReadLocal: 682.00 B (682) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 682.00 B (682) - CachedFileHandlesHitCount: 3 (3) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0.000ns - MaterializeTupleTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 69.00 KB (70656) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 166.49 KB/sec - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 25.00 KB (25600) - RowBatchQueueGetWaitTime: 4.000ms - RowBatchQueuePeakMemoryUsage: 41.00 KB (41984) - RowBatchQueuePutWaitTime: 0.000ns - RowBatchesEnqueued: 3 (3) - RowsRead: 1 (1) - RowsReturned: 1 (1) - RowsReturnedRate: 249.00 /sec - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 4.000ms - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 4.000ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 5 (5) - TotalRawHdfsOpenFileTime: 0.000ns - TotalRawHdfsReadTime: 4.000ms - TotalReadThroughput: 0.00 /sec Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 24.00 KB (24576) - CumulativeAllocations: 3 (3) - PeakReservation: 16.00 KB (16384) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 16.00 KB (16384) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns HDFS_SCAN_NODE (id=3):(Total: 8.000ms, non-child: 8.000ms, % non-child: 100.00%) Table Name: default.pta1 Hdfs split stats (:<# splits>/): 0:1/585.00 B ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:1 Node Lifecycle Event Timeline: 364.031ms - Open Started: 356.031ms (356.031ms) - Open Finished: 360.031ms (4.000ms) - First Batch Requested: 360.031ms (0.000ns) - First Batch Returned: 364.031ms (4.000ms) - Last Batch Returned: 364.031ms (0.000ns) - Closed: 364.031ms (0.000ns) - FooterProcessingTime: (Avg: 4.000ms ; Min: 4.000ms ; Max: 4.000ms ; Number of samples: 1) - InitialRangeActualReservation: (Avg: 16.00 KB (16384) ; Min: 16.00 KB (16384) ; Max: 16.00 KB (16384) ; Number of samples: 1) - InitialRangeIdealReservation: (Avg: 8.00 KB (8192) ; Min: 8.00 KB (8192) ; Max: 8.00 KB (8192) ; Number of samples: 1) - PageIndexProcessingTime: 0.000ns (Number of samples: 0) - ParquetCompressedBytesReadPerColumn: (Avg: 11.00 B (11) ; Min: 11.00 B (11) ; Max: 11.00 B (11) ; Number of samples: 2) - ParquetCompressedPageSize: (Avg: 11.00 B (11) ; Min: 11.00 B (11) ; Max: 11.00 B (11) ; Number of samples: 2) - ParquetRowGroupActualReservation: (Avg: 16.00 KB (16384) ; Min: 16.00 KB (16384) ; Max: 16.00 KB (16384) ; Number of samples: 1) - ParquetRowGroupIdealReservation: (Avg: 16.00 KB (16384) ; Min: 16.00 KB (16384) ; Max: 16.00 KB (16384) ; Number of samples: 1) - ParquetUncompressedBytesReadPerColumn: (Avg: 9.00 B (9) ; Min: 9.00 B (9) ; Max: 9.00 B (9) ; Number of samples: 2) - ParquetUncompressedPageSize: (Avg: 9.00 B (9) ; Min: 9.00 B (9) ; Max: 9.00 B (9) ; Number of samples: 2) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 682.00 B (682) - BytesReadDataNodeCache: 0 - BytesReadLocal: 682.00 B (682) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 682.00 B (682) - CachedFileHandlesHitCount: 3 (3) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0.000ns - MaterializeTupleTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 69.00 KB (70656) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 166.49 KB/sec - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 25.00 KB (25600) - RowBatchQueueGetWaitTime: 4.000ms - RowBatchQueuePeakMemoryUsage: 49.00 KB (50176) - RowBatchQueuePutWaitTime: 0.000ns - RowBatchesEnqueued: 3 (3) - RowsRead: 1 (1) - RowsReturned: 1 (1) - RowsReturnedRate: 124.00 /sec - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 4.000ms - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 4.000ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 4 (4) - TotalRawHdfsOpenFileTime: 0.000ns - TotalRawHdfsReadTime: 4.000ms - TotalReadThroughput: 0.00 /sec Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 24.00 KB (24576) - CumulativeAllocations: 3 (3) - PeakReservation: 16.00 KB (16384) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 16.00 KB (16384) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns CodeGen:(Total: 352.030ms, non-child: 352.030ms, % non-child: 100.00%) - CodegenInvoluntaryContextSwitches: 0 (0) - CodegenTotalWallClockTime: 352.030ms - CodegenSysTime: 0.000ns - CodegenUserTime: 352.000ms - CodegenVoluntaryContextSwitches: 0 (0) - CompileTime: 28.002ms - IrGenerationTime: 16.001ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.48 MB (2604708) - NumFunctions: 77 (77) - NumInstructions: 3.46K (3460) - OptimizationTime: 288.025ms - PeakMemoryUsage: 1.69 MB (1771520) - PrepareTime: 20.001ms Averaged Fragment F04:(Total: 64.005ms, non-child: 0.000ns, % non-child: 0.00%) split sizes: min: 579.00 B, max: 579.00 B, avg: 579.00 B, stddev: 0 completion times: min:372.032ms max:372.032ms mean: 372.032ms stddev:0.000ns execution rates: min:1.52 KB/sec max:1.52 KB/sec mean:1.52 KB/sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - ExchangeScanRatio: 0.05 - PeakMemoryUsage: 222.41 KB (227744) - PeakReservation: 16.00 KB (16384) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 3.94 MB (4136000) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 68.005ms - TotalThreadsSysTime: 8.000ms - TotalThreadsUserTime: 52.000ms - TotalThreadsVoluntaryContextSwitches: 9 (9) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 4.000ms - ExecTreeExecTime: 4.000ms - OpenTime: 60.005ms - ExecTreeOpenTime: 0.000ns - PrepareTime: 0.000ns - ExecTreePrepareTime: 0.000ns KrpcDataStreamSender (dst_id=6): - EosSent: 1 (1) - PeakMemoryUsage: 4.41 KB (4512) - RowsSent: 1 (1) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0.000ns - TotalBytesSent: 32.00 B (32) - UncompressedRowBatchSize: 43.00 B (43) UNION_NODE (id=0):(Total: 4.000ms, non-child: 0.000ns, % non-child: 0.00%) - PeakMemoryUsage: 29.00 KB (29696) - RowsReturned: 1 (1) - RowsReturnedRate: 249.00 /sec HDFS_SCAN_NODE (id=1):(Total: 4.000ms, non-child: 4.000ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 676.00 B (676) - BytesReadDataNodeCache: 0 - BytesReadLocal: 676.00 B (676) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 676.00 B (676) - CachedFileHandlesHitCount: 3 (3) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0.000ns - MaterializeTupleTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 61.02 KB (62480) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 0.00 /sec - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 25.00 KB (25600) - RowBatchQueueGetWaitTime: 4.000ms - RowBatchQueuePeakMemoryUsage: 41.00 KB (41984) - RowBatchQueuePutWaitTime: 0.000ns - RowBatchesEnqueued: 3 (3) - RowsRead: 1 (1) - RowsReturned: 1 (1) - RowsReturnedRate: 249.00 /sec - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 0.000ns - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 4.000ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 4 (4) - TotalRawHdfsOpenFileTime: 0.000ns - TotalRawHdfsReadTime: 0.000ns - TotalReadThroughput: 0.00 /sec Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 24.00 KB (24576) - CumulativeAllocations: 3 (3) - PeakReservation: 16.00 KB (16384) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 16.00 KB (16384) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns CodeGen:(Total: 60.005ms, non-child: 60.005ms, % non-child: 100.00%) - CodegenInvoluntaryContextSwitches: 0 (0) - CodegenTotalWallClockTime: 60.005ms - CodegenSysTime: 4.000ms - CodegenUserTime: 52.000ms - CodegenVoluntaryContextSwitches: 0 (0) - CompileTime: 4.000ms - IrGenerationTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 2.48 MB (2604708) - NumFunctions: 18 (18) - NumInstructions: 420 (420) - OptimizationTime: 32.002ms - PeakMemoryUsage: 210.00 KB (215040) - PrepareTime: 20.001ms Fragment F04: Instance fe43fc7b5ea71656:4a5d2eea00000001 (host=fangyu-upstream-fe-dev.gce.cloudera.com:22001):(Total: 64.005ms, non-child: 0.000ns, % non-child: 0.00%) Last report received time: 2020-01-15 15:06:33.673 Hdfs split stats (:<# splits>/): 2:1/579.00 B Fragment Instance Lifecycle Event Timeline: 68.005ms - Prepare Finished: 4.000ms (4.000ms) - Open Finished: 64.005ms (60.005ms) - First Batch Produced: 68.005ms (4.000ms) - First Batch Sent: 68.005ms (0.000ns) - ExecInternal Finished: 68.005ms (0.000ns) - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - ExchangeScanRatio: 0.05 - PeakMemoryUsage: 222.41 KB (227744) - PeakReservation: 16.00 KB (16384) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 3.94 MB (4136000) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 68.005ms - TotalThreadsSysTime: 8.000ms - TotalThreadsUserTime: 52.000ms - TotalThreadsVoluntaryContextSwitches: 9 (9) Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 0 - CumulativeAllocations: 0 (0) - PeakReservation: 0 - PeakUnpinnedBytes: 0 - PeakUsedReservation: 0 - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - ReservationLimit: 0 - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Fragment Instance Lifecycle Timings: - ExecTime: 4.000ms - ExecTreeExecTime: 4.000ms - OpenTime: 60.005ms - ExecTreeOpenTime: 0.000ns - PrepareTime: 0.000ns - ExecTreePrepareTime: 0.000ns KrpcDataStreamSender (dst_id=6): ExecOption: Unpartitioned Sender Codegen Disabled: not needed - NetworkThroughput: (Avg: 103.91 KB/sec ; Min: 103.91 KB/sec ; Max: 103.91 KB/sec ; Number of samples: 1) - RpcNetworkTime: (Avg: 197.351us ; Min: 93.975us ; Max: 300.728us ; Number of samples: 2) - RpcRecvrTime: (Avg: 92.236us ; Min: 55.295us ; Max: 129.176us ; Number of samples: 2) - EosSent: 1 (1) - PeakMemoryUsage: 4.41 KB (4512) - RowsSent: 1 (1) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0.000ns - TotalBytesSent: 32.00 B (32) - UncompressedRowBatchSize: 43.00 B (43) UNION_NODE (id=0):(Total: 4.000ms, non-child: 0.000ns, % non-child: 0.00%) ExecOption: Codegen Enabled Node Lifecycle Event Timeline: 68.005ms - Open Started: 64.005ms (64.005ms) - Open Finished: 64.005ms (0.000ns) - First Batch Requested: 64.005ms (0.000ns) - First Batch Returned: 68.005ms (4.000ms) - Last Batch Returned: 68.005ms (0.000ns) - Closed: 68.005ms (0.000ns) - PeakMemoryUsage: 29.00 KB (29696) - RowsReturned: 1 (1) - RowsReturnedRate: 249.00 /sec HDFS_SCAN_NODE (id=1):(Total: 4.000ms, non-child: 4.000ms, % non-child: 100.00%) Table Name: default.pt1 Hdfs split stats (:<# splits>/): 2:1/579.00 B ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:1 Node Lifecycle Event Timeline: 68.005ms - Open Started: 64.005ms (64.005ms) - Open Finished: 64.005ms (0.000ns) - First Batch Requested: 64.005ms (0.000ns) - First Batch Returned: 68.005ms (4.000ms) - Last Batch Returned: 68.005ms (0.000ns) - Closed: 68.005ms (0.000ns) - FooterProcessingTime: (Avg: 0.000ns ; Min: 0.000ns ; Max: 0.000ns ; Number of samples: 1) - InitialRangeActualReservation: (Avg: 16.00 KB (16384) ; Min: 16.00 KB (16384) ; Max: 16.00 KB (16384) ; Number of samples: 1) - InitialRangeIdealReservation: (Avg: 8.00 KB (8192) ; Min: 8.00 KB (8192) ; Max: 8.00 KB (8192) ; Number of samples: 1) - PageIndexProcessingTime: 0.000ns (Number of samples: 0) - ParquetCompressedBytesReadPerColumn: (Avg: 11.00 B (11) ; Min: 11.00 B (11) ; Max: 11.00 B (11) ; Number of samples: 2) - ParquetCompressedPageSize: (Avg: 11.00 B (11) ; Min: 11.00 B (11) ; Max: 11.00 B (11) ; Number of samples: 2) - ParquetRowGroupActualReservation: (Avg: 16.00 KB (16384) ; Min: 16.00 KB (16384) ; Max: 16.00 KB (16384) ; Number of samples: 1) - ParquetRowGroupIdealReservation: (Avg: 16.00 KB (16384) ; Min: 16.00 KB (16384) ; Max: 16.00 KB (16384) ; Number of samples: 1) - ParquetUncompressedBytesReadPerColumn: (Avg: 9.00 B (9) ; Min: 9.00 B (9) ; Max: 9.00 B (9) ; Number of samples: 2) - ParquetUncompressedPageSize: (Avg: 9.00 B (9) ; Min: 9.00 B (9) ; Max: 9.00 B (9) ; Number of samples: 2) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 676.00 B (676) - BytesReadDataNodeCache: 0 - BytesReadLocal: 676.00 B (676) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 676.00 B (676) - CachedFileHandlesHitCount: 3 (3) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0.000ns - MaterializeTupleTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 61.02 KB (62480) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 0.00 /sec - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 25.00 KB (25600) - RowBatchQueueGetWaitTime: 4.000ms - RowBatchQueuePeakMemoryUsage: 41.00 KB (41984) - RowBatchQueuePutWaitTime: 0.000ns - RowBatchesEnqueued: 3 (3) - RowsRead: 1 (1) - RowsReturned: 1 (1) - RowsReturnedRate: 249.00 /sec - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 0.000ns - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 4.000ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 4 (4) - TotalRawHdfsOpenFileTime: 0.000ns - TotalRawHdfsReadTime: 0.000ns - TotalReadThroughput: 0.00 /sec Buffer pool: - AllocTime: 0.000ns - CumulativeAllocationBytes: 24.00 KB (24576) - CumulativeAllocations: 3 (3) - PeakReservation: 16.00 KB (16384) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 16.00 KB (16384) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - SystemAllocTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns CodeGen:(Total: 60.005ms, non-child: 60.005ms, % non-child: 100.00%) - CodegenInvoluntaryContextSwitches: 0 (0) - CodegenTotalWallClockTime: 60.005ms - CodegenSysTime: 4.000ms - CodegenUserTime: 52.000ms - CodegenVoluntaryContextSwitches: 0 (0) - CompileTime: 4.000ms - IrGenerationTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 2.48 MB (2604708) - NumFunctions: 18 (18) - NumInstructions: 420 (420) - OptimizationTime: 32.002ms - PeakMemoryUsage: 210.00 KB (215040) - PrepareTime: 20.001ms