Query (id=394ed7e2a194714b:31ab5b2f00000000) DEBUG MODE WARNING: Query profile created while running a DEBUG build of Impala. Use RELEASE builds to measure query performance. Summary Session ID: 0a4cc8237ba771fb:8a62b8953725f4ad Session Type: BEESWAX Start Time: 2019-10-26 04:26:15.032596000 End Time: 2019-10-26 04:26:25.716512000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 3.4.0-SNAPSHOT DEBUG (build 0f70ade0d78a9eb3afafdfd1a3e36cc8a5563cb4) User: jenkins Connected User: jenkins Delegated User: Network Address: ::1:41320 Default Db: functional_parquet Sql Statement: select id, bigint_col from alltypestiny union all select sum(int_col), bigint_col from alltypes where year=2009 and month=2 group by bigint_col union all select a.id, a.bigint_col from alltypestiny a inner join alltypestiny b on (a.id = b.id) union all select 1000, 2000 Coordinator: impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22000 Query Options (set by configuration): ABORT_ON_ERROR=1,EXEC_SINGLE_NODE_ROWS_THRESHOLD=0,DISABLE_CODEGEN_ROWS_THRESHOLD=0,TIMEZONE=America/Los_Angeles,CLIENT_IDENTIFIER=query_test/test_queries.py::TestQueries::()::test_union[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|table_format:parquet/none Query Options (set by configuration and planner): ABORT_ON_ERROR=1,EXEC_SINGLE_NODE_ROWS_THRESHOLD=0,MT_DOP=0,DISABLE_CODEGEN_ROWS_THRESHOLD=0,TIMEZONE=America/Los_Angeles,CLIENT_IDENTIFIER=query_test/test_queries.py::TestQueries::()::test_union[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|table_format:parquet/none Plan: ---------------- Max Per-Host Resource Reservation: Memory=70.04MB Threads=9 Per-Host Resource Estimates: Memory=338MB WARNING: The following tables are missing relevant table and/or column statistics. functional_parquet.alltypes, functional_parquet.alltypestiny Analyzed query: SELECT id, bigint_col FROM functional_parquet.alltypestiny UNION ALL SELECT sum(CAST(int_col AS BIGINT)), bigint_col FROM functional_parquet.alltypes WHERE `year` = CAST(2009 AS INT) AND `month` = CAST(2 AS INT) GROUP BY bigint_col UNION ALL SELECT a.id, a.bigint_col FROM functional_parquet.alltypestiny a INNER JOIN functional_parquet.alltypestiny b ON (a.id = b.id) UNION ALL SELECT CAST(1000 AS BIGINT), CAST(2000 AS BIGINT) F07:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=70.46KB mem-reservation=0B thread-reservation=1 PLAN-ROOT SINK | output exprs: id, bigint_col | mem-estimate=0B mem-reservation=0B thread-reservation=0 | 11:EXCHANGE [UNPARTITIONED] | mem-estimate=70.46KB mem-reservation=0B thread-reservation=0 | tuple-ids=5 row-size=16B cardinality=2.01K | in pipelines: 01(GETNEXT), 08(GETNEXT), 04(GETNEXT) | F06:PLAN FRAGMENT [RANDOM] hosts=3 instances=3 Per-Host Resources: mem-estimate=129.02MB mem-reservation=35.00MB thread-reservation=2 runtime-filters-memory=1.00MB 00:UNION | constant-operands=1 | mem-estimate=0B mem-reservation=0B thread-reservation=0 | tuple-ids=5 row-size=16B cardinality=2.01K | in pipelines: 01(GETNEXT), 08(GETNEXT), 04(GETNEXT) | |--06:HASH JOIN [INNER JOIN, PARTITIONED] | | hash predicates: a.id = b.id | | fk/pk conjuncts: assumed fk/pk | | runtime filters: RF000[bloom] <- b.id | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 | | tuple-ids=3,4 row-size=16B cardinality=758 | | in pipelines: 04(GETNEXT), 05(OPEN) | | | |--10:EXCHANGE [HASH(b.id)] | | | mem-estimate=18.75KB mem-reservation=0B thread-reservation=0 | | | tuple-ids=4 row-size=4B cardinality=758 | | | in pipelines: 05(GETNEXT) | | | | | F04:PLAN FRAGMENT [RANDOM] hosts=3 instances=3 | | Per-Host Resources: mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=2 | | 05:SCAN HDFS [functional_parquet.alltypestiny b, RANDOM] | | HDFS partitions=4/4 files=4 size=11.92KB | | stored statistics: | | table: rows=unavailable size=unavailable | | partitions: 0/4 rows=unavailable | | columns: unavailable | | extrapolated-rows=disabled max-scan-range-rows=unavailable | | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1 | | tuple-ids=4 row-size=4B cardinality=758 | | in pipelines: 05(GETNEXT) | | | 09:EXCHANGE [HASH(a.id)] | | mem-estimate=38.49KB mem-reservation=0B thread-reservation=0 | | tuple-ids=3 row-size=12B cardinality=758 | | in pipelines: 04(GETNEXT) | | | F03:PLAN FRAGMENT [RANDOM] hosts=3 instances=3 | Per-Host Resources: mem-estimate=33.00MB mem-reservation=1.02MB thread-reservation=2 runtime-filters-memory=1.00MB | 04:SCAN HDFS [functional_parquet.alltypestiny a, RANDOM] | HDFS partitions=4/4 files=4 size=11.92KB | runtime filters: RF000[bloom] -> a.id | stored statistics: | table: rows=unavailable size=unavailable | partitions: 0/4 rows=unavailable | columns: unavailable | extrapolated-rows=disabled max-scan-range-rows=unavailable | mem-estimate=32.00MB mem-reservation=16.00KB thread-reservation=1 | tuple-ids=3 row-size=12B cardinality=758 | in pipelines: 04(GETNEXT) | |--08:AGGREGATE [FINALIZE] | | output: sum:merge(int_col) | | group by: bigint_col | | mem-estimate=128.00MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 | | tuple-ids=2 row-size=16B cardinality=491 | | in pipelines: 08(GETNEXT), 02(OPEN) | | | 07:EXCHANGE [HASH(bigint_col)] | | mem-estimate=17.26KB mem-reservation=0B thread-reservation=0 | | tuple-ids=2 row-size=16B cardinality=491 | | in pipelines: 02(GETNEXT) | | | F01:PLAN FRAGMENT [RANDOM] hosts=1 instances=1 | Per-Host Resources: mem-estimate=160.00MB mem-reservation=34.02MB thread-reservation=2 | 03:AGGREGATE [STREAMING] | | output: sum(CAST(int_col AS BIGINT)) | | group by: bigint_col | | mem-estimate=128.00MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 | | tuple-ids=2 row-size=16B cardinality=491 | | in pipelines: 02(GETNEXT) | | | 02:SCAN HDFS [functional_parquet.alltypes, RANDOM] | partition predicates: `year` = CAST(2009 AS INT), `month` = CAST(2 AS INT) | HDFS partitions=1/24 files=1 size=7.71KB | stored statistics: | table: rows=unavailable size=unavailable | partitions: 0/1 rows=unavailable | columns missing stats: int_col, bigint_col | extrapolated-rows=disabled max-scan-range-rows=unavailable | mem-estimate=32.00MB mem-reservation=16.00KB thread-reservation=1 | tuple-ids=1 row-size=12B cardinality=491 | in pipelines: 02(GETNEXT) | 01:SCAN HDFS [functional_parquet.alltypestiny, RANDOM] HDFS partitions=4/4 files=4 size=11.92KB stored statistics: table: rows=unavailable size=unavailable partitions: 0/4 rows=unavailable columns: unavailable extrapolated-rows=disabled max-scan-range-rows=unavailable mem-estimate=32.00MB mem-reservation=16.00KB thread-reservation=1 tuple-ids=0 row-size=12B cardinality=758 in pipelines: 01(GETNEXT) ---------------- Estimated Per-Host Mem: 354508513 Tables Missing Stats: functional_parquet.alltypes,functional_parquet.alltypestiny Request Pool: default-pool Per Host Min Memory Reservation: impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22001(36.02 MB) impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22002(36.02 MB) impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22000(70.04 MB) Per Host Number of Fragment Instances: impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22001(3) impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22002(3) impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22000(5) Admission result: Admitted immediately Cluster Memory Admitted: 1014.26 MB Executor Group: default ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ---------------------------------------------------------------------------------------------------------------------------------------- F07:ROOT 1 0.000ns 0.000ns 0 0 11:EXCHANGE 1 1.999ms 1.999ms 27 2.01K 40.00 KB 70.46 KB UNPARTITIONED F06:EXCHANGE SENDER 3 2.333ms 2.999ms 7.52 KB 0 00:UNION 3 666.643us 1.999ms 27 2.01K 25.00 KB 0 |--08:AGGREGATE 3 26.332ms 47.998ms 10 491 34.02 MB 128.00 MB FINALIZE | 07:EXCHANGE 3 0.000ns 0.000ns 10 491 16.00 KB 17.26 KB HASH(bigint_col) | F01:EXCHANGE SENDER 1 3.999ms 3.999ms 34.57 KB 0 | 03:AGGREGATE 1 51.998ms 51.998ms 10 491 34.04 MB 128.00 MB STREAMING | 02:SCAN HDFS 1 71.997ms 71.997ms 280 491 57.00 KB 32.00 MB functional_parquet.alltypes |--06:HASH JOIN 3 65.997ms 110.995ms 8 758 1.98 MB 1.94 MB INNER JOIN, PARTITIONED | |--10:EXCHANGE 3 333.321us 999.964us 8 758 32.00 KB 18.75 KB HASH(b.id) | | F04:EXCHANGE SENDER 3 2.666ms 2.999ms 84.78 KB 0 | | 05:SCAN HDFS 3 83.996ms 97.996ms 8 758 41.00 KB 16.00 MB functional_parquet.alltypestiny b | 09:EXCHANGE 3 333.321us 999.964us 8 758 32.00 KB 38.49 KB HASH(a.id) | F03:EXCHANGE SENDER 3 13.999ms 37.998ms 37.53 KB 0 | 04:SCAN HDFS 3 83.663ms 90.996ms 8 758 57.00 KB 32.00 MB functional_parquet.alltypestiny a 01:SCAN HDFS 3 46.664ms 71.997ms 8 758 61.00 KB 32.00 MB functional_parquet.alltypestiny Errors: - AdmissionControlTimeSinceLastUpdate: 119ms (119000000) - ComputeScanRangeAssignmentTimer: 5ms (5999784) - InactiveTotalTime: 0ns (0) - TotalTime: 0ns (0) Query Compilation Metadata of all 2 tables cached: 749.56us (749564) Analysis finished: 1ms (1304090) Authorization finished (noop): 1ms (1323460) Value transfer graph computed: 1ms (1847981) Single node plan created: 3ms (3451031) Runtime filters computed: 3ms (3633581) Distributed plan created: 3ms (3653231) Planning finished: 5ms (5204807) Query Timeline Query submitted: 1ms (1999928) Planning finished: 11ms (11999567) Submit for admission: 31ms (31998844) Completed admission: 65ms (65997616) Ready to start on 3 backends: 95ms (95996533) All 3 execution backends (11 fragment instances) started: 160ms (160994185) First dynamic filter received: 5.96s (5964784563) Rows available: 5.98s (5978784057) First row fetched: 10.54s (10539619327) Last row fetched: 10.54s (10539619327) Released admission control resources: 10.68s (10682614162) Unregister query: 11.17s (11168596609) Frontend - InactiveTotalTime: 0ns (0) - TotalTime: 0ns (0) ImpalaServer - ClientFetchWaitTimer: 463ms (463983241) - InactiveTotalTime: 0ns (0) - NumRowsFetched: 27 (27) - NumRowsFetchedFromCache: 0 (0) - RowMaterializationRate: 5 per second (5) - RowMaterializationTimer: 4.72s (4725829311) - TotalTime: 0ns (0) Execution Profile 394ed7e2a194714b:31ab5b2f00000000 Number of filters: 1 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- 0 6 4 REMOTE false 3 (3) N/A N/A true Backend startup latencies: Count: 3, min / max: 29ms / 42ms, 25th %-ile: 29ms, 50th %-ile: 41ms, 75th %-ile: 41ms, 90th %-ile: 42ms, 95th %-ile: 42ms, 99.9th %-ile: 42ms Slowest backend to start up: impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22001 Final filter table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- 0 6 4 REMOTE false 0 (3) 5s964ms 10s489ms false Per Node Peak Memory Usage: impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22001(38.78 MB) impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22002(38.78 MB) impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22000(73.47 MB) Per Node Bytes Read: impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22001(9.20 KB) impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22002(9.20 KB) impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22000(26.36 KB) Per Node User Time: impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22001(5s359ms) impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22002(5s723ms) impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22000(11s787ms) Per Node System Time: impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22001(46.460ms) impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22002(33.501ms) impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22000(88.907ms) - ExchangeScanRatio: 0.01 (0.014725130890052356) - FiltersReceived: 3 (3) - FinalizationTimer: 0ns (0) - InactiveTotalTime: 0ns (0) - InnerNodeSelectivityRatio: 0.00 (0.0) - NumBackends: 3 (3) - NumCompletedBackends: 3 (3) - NumFragmentInstances: 11 (11) - NumFragments: 5 (5) - TotalBytesRead: 44.8 KiB (45840) - TotalBytesSent: 675 B (675) - TotalCpuTime: 23.04s (23039024000) - TotalInnerBytesSent: 0 B (0) - TotalScanBytesSent: 675 B (675) - TotalTime: 10.64s (10637615788) Per Node Profiles impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22001 - AdmissionSlots: 1 (1) - InactiveTotalTime: 0ns (0) - ScratchBytesRead: 0 B (0) - ScratchBytesWritten: 0 B (0) - ScratchFileUsedBytes: 0 B (0) - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalEncryptionTime: 0ns (0) - TotalReadBlockTime: 0ns (0) - TotalTime: 0ns (0) impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22002 - AdmissionSlots: 1 (1) - InactiveTotalTime: 0ns (0) - ScratchBytesRead: 0 B (0) - ScratchBytesWritten: 0 B (0) - ScratchFileUsedBytes: 0 B (0) - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalEncryptionTime: 0ns (0) - TotalReadBlockTime: 0ns (0) - TotalTime: 0ns (0) impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22000 - AdmissionSlots: 1 (1) - InactiveTotalTime: 0ns (0) - ScratchBytesRead: 0 B (0) - ScratchBytesWritten: 0 B (0) - ScratchFileUsedBytes: 0 B (0) - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalEncryptionTime: 0ns (0) - TotalReadBlockTime: 0ns (0) - TotalTime: 0ns (0) Averaged Fragment F07 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:10s461ms max:10s461ms mean: 10s461ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 (0.0) - BloomFilterBytes: 0 B (0) - ExchangeScanRatio: 0.00 (0.0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 48.2 KiB (49334) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 73.5 MiB (77039702) - RowsProduced: 27 (27) - TotalNetworkReceiveTime: 9.12s (9124670432) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 229 (229) - TotalThreadsTotalWallClockTime: 10.36s (10357625900) - TotalThreadsSysTime: 980.00us (980000) - TotalThreadsUserTime: 578ms (578415000) - TotalThreadsVoluntaryContextSwitches: 6 (6) - TotalTime: 10.38s (10376625213) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 0 B (0) - CumulativeAllocations: 0 (0) - InactiveTotalTime: 0ns (0) - PeakReservation: 0 B (0) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 0 B (0) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 0 B (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 4.56s (4556835414) - ExecTreeExecTime: 4.56s (4556835414) - InactiveTotalTime: 0ns (0) - OpenTime: 5.80s (5800790486) - ExecTreeOpenTime: 4.57s (4569834946) - PrepareTime: 17ms (17999349) - ExecTreePrepareTime: 4ms (4999820) - TotalTime: 0ns (0) PLAN_ROOT_SINK - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsSent: 27 (27) - RowsSentRate: 0 per second (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=11) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 40.0 KiB (40960) - RowsReturned: 27 (27) - RowsReturnedRate: 2 per second (2) - TotalTime: 9.13s (9126670360) Buffer pool - AllocTime: 999.96us (999963) - CumulativeAllocationBytes: 64.0 KiB (65536) - CumulativeAllocations: 8 (8) - InactiveTotalTime: 0ns (0) - PeakReservation: 40.0 KiB (40960) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 40.0 KiB (40960) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 999.96us (999963) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Dequeue - FirstBatchWaitTime: 4.57s (4568834982) - InactiveTotalTime: 0ns (0) - TotalBytesDequeued: 675 B (675) - TotalGetBatchTime: 9.12s (9124670432) - DataWaitTime: 9.12s (9124670432) - TotalTime: 0ns (0) Enqueue - DeserializeRowBatchTime: 999.96us (999963) - InactiveTotalTime: 0ns (0) - TotalBatchesEnqueued: 4 (4) - TotalBatchesReceived: 4 (4) - TotalBytesReceived: 314 B (314) - TotalEarlySenders: 0 (0) - TotalEosReceived: 3 (3) - TotalHasDeferredRPCsTime: 0ns (0) - TotalRPCsDeferred: 0 (0) - TotalTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 229 (229) - CodegenTotalWallClockTime: 1.23s (1228955612) - CodegenSysTime: 979.00us (979000) - CodegenUserTime: 573ms (573794000) - CodegenVoluntaryContextSwitches: 1 (1) - CompileTime: 64ms (64997652) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 1ms (1999928) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 2 (2) - NumInstructions: 36 (36) - OptimizationTime: 192ms (192993029) - PeakMemoryUsage: 18.0 KiB (18432) - PrepareTime: 947ms (947965761) - TotalTime: 1.23s (1228955612) Coordinator Fragment F07 Instance 394ed7e2a194714b:31ab5b2f00000000 (host=impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22000) Last report received time: 2019-10-26 04:26:25.653 - AverageThreadTokens: 0.00 (0.0) - BloomFilterBytes: 0 B (0) - ExchangeScanRatio: 0.00 (0.0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 48.2 KiB (49334) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 73.5 MiB (77039702) - RowsProduced: 27 (27) - TotalNetworkReceiveTime: 9.12s (9124670432) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 229 (229) - TotalThreadsTotalWallClockTime: 10.36s (10357625900) - TotalThreadsSysTime: 980.00us (980000) - TotalThreadsUserTime: 578ms (578415000) - TotalThreadsVoluntaryContextSwitches: 6 (6) - TotalTime: 10.38s (10376625213) Fragment Instance Lifecycle Event Timeline Prepare Finished: 31ms (31998844) Open Finished: 5.83s (5832789330) First Batch Produced: 10.39s (10389624744) First Batch Sent: 10.39s (10389624744) ExecInternal Finished: 10.39s (10392624636) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 0 B (0) - CumulativeAllocations: 0 (0) - InactiveTotalTime: 0ns (0) - PeakReservation: 0 B (0) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 0 B (0) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 0 B (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 4.56s (4556835414) - ExecTreeExecTime: 4.56s (4556835414) - InactiveTotalTime: 0ns (0) - OpenTime: 5.80s (5800790486) - ExecTreeOpenTime: 4.57s (4569834946) - PrepareTime: 17ms (17999349) - ExecTreePrepareTime: 4ms (4999820) - TotalTime: 0ns (0) PLAN_ROOT_SINK - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsSent: 27 (27) - RowsSentRate: 0 per second (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=11) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 9.12s (9124670432) - PeakMemoryUsage: 40.0 KiB (40960) - RowsReturned: 27 (27) - RowsReturnedRate: 2 per second (2) - TotalTime: 9.13s (9126670360) Node Lifecycle Event Timeline Open Started: 1.26s (1262954384) Open Finished: 5.83s (5831789366) First Batch Requested: 5.83s (5832789330) First Batch Returned: 10.39s (10388624780) Last Batch Returned: 10.39s (10389624744) Closed: 10.39s (10390624708) Buffer pool - AllocTime: 999.96us (999963) - CumulativeAllocationBytes: 64.0 KiB (65536) - CumulativeAllocations: 8 (8) - InactiveTotalTime: 0ns (0) - PeakReservation: 40.0 KiB (40960) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 40.0 KiB (40960) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 999.96us (999963) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Dequeue - FirstBatchWaitTime: 4.57s (4568834982) - InactiveTotalTime: 0ns (0) - TotalBytesDequeued: 675 B (675) - TotalGetBatchTime: 9.12s (9124670432) - DataWaitTime: 9.12s (9124670432) - TotalTime: 0ns (0) Enqueue - DeserializeRowBatchTime: 999.96us (999963) - InactiveTotalTime: 0ns (0) - TotalBatchesEnqueued: 4 (4) - TotalBatchesReceived: 4 (4) - TotalBytesReceived: 314 B (314) - TotalEarlySenders: 0 (0) - TotalEosReceived: 3 (3) - TotalHasDeferredRPCsTime: 0ns (0) - TotalRPCsDeferred: 0 (0) - TotalTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 229 (229) - CodegenTotalWallClockTime: 1.23s (1228955612) - CodegenSysTime: 979.00us (979000) - CodegenUserTime: 573ms (573794000) - CodegenVoluntaryContextSwitches: 1 (1) - CompileTime: 64ms (64997652) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 1ms (1999928) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 2 (2) - NumInstructions: 36 (36) - OptimizationTime: 192ms (192993029) - PeakMemoryUsage: 18.0 KiB (18432) - PrepareTime: 947ms (947965761) - TotalTime: 1.23s (1228955612) Averaged Fragment F06 split sizes: min: 2.98 KB, max: 5.96 KB, avg: 3.97 KB, stddev: 1.40 KB completion times: min:5s896ms max:10s518ms mean: 7s544ms stddev:2s107ms execution rates: min:490.00 B/sec max:580.00 B/sec mean:529.00 B/sec stddev:37.63 B/sec num instances: 3 - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 1.0 MiB (1048576) - ExchangeScanRatio: 0.03 (0.027153854265527785) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 35.2 MiB (36916656) - PeakReservation: 35.0 MiB (36700160) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 50.3 MiB (52786007) - RowsProduced: 9 (9) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 33ms (33665450) - TotalStorageWaitTime: 4ms (4666498) - TotalThreadsInvoluntaryContextSwitches: 776 (776) - TotalThreadsTotalWallClockTime: 7.45s (7454397426) - TotalThreadsSysTime: 24ms (24988666) - TotalThreadsUserTime: 4.43s (4432573333) - TotalThreadsVoluntaryContextSwitches: 11 (11) - TotalTime: 7.46s (7456064033) Buffer pool - AllocTime: 333.32us (333321) - CumulativeAllocationBytes: 1.0 MiB (1048576) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.0 MiB (1048576) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 1.0 MiB (1048576) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 1.0 MiB (1048576) - SystemAllocTime: 333.32us (333321) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 130ms (130995268) - ExecTreeExecTime: 95ms (95329890) - InactiveTotalTime: 0ns (0) - OpenTime: 7.28s (7280737032) - ExecTreeOpenTime: 3ms (3666534) - PrepareTime: 42ms (42998447) - ExecTreePrepareTime: 30ms (30332237) - TotalTime: 0ns (0) KrpcDataStreamSender (dst_id=11) - EosSent: 1 (1) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 7.5 KiB (7704) - RowsSent: 9 (9) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0ns (0) - TotalBytesSent: 104 B (104) - TotalTime: 36ms (36332020) - UncompressedRowBatchSize: 225 B (225) UNION_NODE (id=0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 25.0 KiB (25600) - RowsReturned: 9 (9) - RowsReturnedRate: 76 per second (76) - TotalTime: 129ms (129328662) AGGREGATION_NODE (id=8) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.0 MiB (35677312) - RowsReturned: 3 (3) - RowsReturnedRate: 140 per second (140) - TotalTime: 26ms (26332382) GroupingAggregator 0 - BuildTime: 1ms (1999928) - GetResultsTime: 333.32us (333321) - HTResizeTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 34 (34) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.0 MiB (35673216) - PeakReservation: 34.0 MiB (35651584) - PeakUsedReservation: 0 B (0) - RowsRepartitioned: 0 (0) - RowsReturned: 3 (3) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 2ms (2666570) - CumulativeAllocationBytes: 8.7 MiB (9087658) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.7 MiB (9087658) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 2ms (2666570) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Table - HashBuckets: 3,413 (3413) - HashCollisions: 0 (0) - InactiveTotalTime: 0ns (0) - Probes: 3 (3) - Resizes: 0 (0) - TotalTime: 0ns (0) - Travel: 0 (0) EXCHANGE_NODE (id=7) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 16.0 KiB (16384) - RowsReturned: 3 (3) - RowsReturnedRate: 0 per second (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 16.0 KiB (16384) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 16.0 KiB (16384) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 16.0 KiB (16384) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Dequeue - FirstBatchWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBytesDequeued: 83 B (83) - TotalGetBatchTime: 0ns (0) - DataWaitTime: 0ns (0) - TotalTime: 0ns (0) Enqueue - DeserializeRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBatchesEnqueued: 1 (1) - TotalBatchesReceived: 1 (1) - TotalBytesReceived: 55 B (55) - TotalEarlySenders: 0 (0) - TotalEosReceived: 1 (1) - TotalHasDeferredRPCsTime: 0ns (0) - TotalRPCsDeferred: 0 (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=6) - BuildRows: 2 (2) - BuildTime: 20ms (20999241) - InactiveTotalTime: 0ns (0) - LocalTime: 65ms (65997615) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2074880) - ProbeRows: 2 (2) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 2 (2) - RowsReturnedRate: 62 per second (62) - TotalTime: 66ms (66330937) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 234.7 KiB (240298) - CumulativeAllocations: 3 (3) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 234.7 KiB (240298) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=6) - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 2 (2) - HashTablesBuildTime: 1ms (1333285) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 41 (41) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) Hash Table - HashBuckets: 2 (2) - HashCollisions: 0 (0) - InactiveTotalTime: 0ns (0) - Probes: 5 (5) - Resizes: 0 (0) - TotalTime: 0ns (0) - Travel: 0 (0) EXCHANGE_NODE (id=10) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 32.0 KiB (32768) - RowsReturned: 2 (2) - RowsReturnedRate: 1333 per second (1333) - TotalTime: 333.32us (333321) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 32.0 KiB (32768) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 32.0 KiB (32768) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 32.0 KiB (32768) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Dequeue - FirstBatchWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBytesDequeued: 34 B (34) - TotalGetBatchTime: 0ns (0) - DataWaitTime: 0ns (0) - TotalTime: 0ns (0) Enqueue - DeserializeRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBatchesEnqueued: 2 (2) - TotalBatchesReceived: 2 (2) - TotalBytesReceived: 24 B (24) - TotalEarlySenders: 0 (0) - TotalEosReceived: 3 (3) - TotalHasDeferredRPCsTime: 0ns (0) - TotalRPCsDeferred: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=9) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 32.0 KiB (32768) - RowsReturned: 2 (2) - RowsReturnedRate: 1333 per second (1333) - TotalTime: 333.32us (333321) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 32.0 KiB (32768) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 32.0 KiB (32768) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 32.0 KiB (32768) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Dequeue - FirstBatchWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBytesDequeued: 56 B (56) - TotalGetBatchTime: 0ns (0) - DataWaitTime: 0ns (0) - TotalTime: 0ns (0) Enqueue - DeserializeRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBatchesEnqueued: 2 (2) - TotalBatchesReceived: 2 (2) - TotalBytesReceived: 40 B (40) - TotalEarlySenders: 0 (0) - TotalEosReceived: 3 (3) - TotalHasDeferredRPCsTime: 0ns (0) - TotalRPCsDeferred: 0 (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=1) - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 4.1 KiB (4214) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 4.1 KiB (4214) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 4.1 KiB (4214) - CachedFileHandlesHitCount: 4 (4) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 B (0) - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 B (0) - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaterializeTupleTime(*): 333.32us (333321) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 58.3 KiB (59733) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 2.5 MiB/s (2634261) - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 17.3 KiB (17749) - RowBatchQueueGetWaitTime: 17ms (17332707) - RowBatchQueuePeakMemoryUsage: 37.0 KiB (37888) - RowBatchQueuePutWaitTime: 0ns (0) - RowBatchesEnqueued: 4 (4) - RowsRead: 2 (2) - RowsReturned: 2 (2) - RowsReturnedRate: 66 per second (66) - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 4ms (4666498) - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 20ms (20332598) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 11ms (11484000) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 0ns (0) - TotalRawHdfsReadTime(*): 999.96us (999963) - TotalReadThroughput: 0 B/s (0) - TotalTime: 46ms (46664980) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 32.0 KiB (32768) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 21.3 KiB (21845) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 16.0 KiB (16384) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 763 (763) - CodegenTotalWallClockTime: 7.28s (7275737212) - CodegenSysTime: 7ms (7243000) - CodegenUserTime: 4.38s (4382507666) - CodegenVoluntaryContextSwitches: 1 (1) - CompileTime: 407ms (407318621) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 306ms (306655591) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 101 (101) - NumInstructions: 4,052 (4052) - OptimizationTime: 5.90s (5898786946) - PeakMemoryUsage: 2.0 MiB (2074624) - PrepareTime: 647ms (647643274) - TotalTime: 7.28s (7275737212) Fragment F06 Instance 394ed7e2a194714b:31ab5b2f0000000a (host=impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22000) Last report received time: 2019-10-26 04:26:25.710 Hdfs split stats (:<# splits>/): 2:1/2.98 KB 1:1/2.98 KB - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 1.0 MiB (1048576) - ExchangeScanRatio: 0.02 (0.017874090477696933) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 35.2 MiB (36916656) - PeakReservation: 35.0 MiB (36700160) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 73.5 MiB (77039702) - RowsProduced: 11 (11) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 38ms (38998592) - TotalStorageWaitTime: 1ms (1999928) - TotalThreadsInvoluntaryContextSwitches: 859 (859) - TotalThreadsTotalWallClockTime: 10.34s (10343626405) - TotalThreadsSysTime: 25ms (25718000) - TotalThreadsUserTime: 5.78s (5780484000) - TotalThreadsVoluntaryContextSwitches: 15 (15) - TotalTime: 10.36s (10357625900) Fragment Instance Lifecycle Event Timeline Prepare Finished: 103ms (103996244) Open Finished: 10.25s (10254629620) First Batch Produced: 10.35s (10353626045) First Batch Sent: 10.35s (10354626008) ExecInternal Finished: 10.40s (10397624455) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 1.0 MiB (1048576) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.0 MiB (1048576) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 1.0 MiB (1048576) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 1.0 MiB (1048576) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 138ms (138994980) - ExecTreeExecTime: 98ms (98996425) - InactiveTotalTime: 0ns (0) - OpenTime: 10.15s (10150633376) - ExecTreeOpenTime: 4ms (4999819) - PrepareTime: 65ms (65997617) - ExecTreePrepareTime: 40ms (40998519) - TotalTime: 0ns (0) KrpcDataStreamSender (dst_id=11) ExecOption: Unpartitioned Sender Codegen Disabled: not needed - EosSent: 1 (1) - InactiveTotalTime: 38ms (38998592) - PeakMemoryUsage: 7.5 KiB (7704) - RowsSent: 11 (11) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0ns (0) - TotalBytesSent: 113 B (113) - TotalTime: 40ms (40998519) - UncompressedRowBatchSize: 275 B (275) UNION_NODE (id=0) ExecOption: Codegen Enabled - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 25.0 KiB (25600) - RowsReturned: 11 (11) - RowsReturnedRate: 75 per second (75) - TotalTime: 144ms (144994763) Node Lifecycle Event Timeline Open Started: 10.25s (10249629801) Open Finished: 10.25s (10254629620) First Batch Requested: 10.25s (10254629620) First Batch Returned: 10.35s (10353626045) Last Batch Returned: 10.35s (10353626045) Closed: 10.39s (10394624564) AGGREGATION_NODE (id=8) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.0 MiB (35677312) - RowsReturned: 5 (5) - RowsReturnedRate: 104 per second (104) - TotalTime: 47ms (47998267) Node Lifecycle Event Timeline Open Started: 10.28s (10279628717) Open Finished: 10.29s (10293628212) First Batch Requested: 10.29s (10293628212) First Batch Returned: 10.29s (10293628212) Last Batch Returned: 10.30s (10297628067) Closed: 10.30s (10298628031) GroupingAggregator 0 ExecOption: Codegen Enabled - BuildTime: 2ms (2999892) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 20 (20) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.0 MiB (35673216) - PeakReservation: 34.0 MiB (35651584) - PeakUsedReservation: 0 B (0) - RowsRepartitioned: 0 (0) - RowsReturned: 5 (5) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 3ms (3999856) - CumulativeAllocationBytes: 12.0 MiB (12582912) - CumulativeAllocations: 6 (6) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 12.0 MiB (12582912) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 3ms (3999856) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Table - HashBuckets: 5,120 (5120) - HashCollisions: 0 (0) - InactiveTotalTime: 0ns (0) - Probes: 5 (5) - Resizes: 0 (0) - TotalTime: 0ns (0) - Travel: 0 (0) EXCHANGE_NODE (id=7) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 16.0 KiB (16384) - RowsReturned: 5 (5) - RowsReturnedRate: 0 per second (0) - TotalTime: 0ns (0) Node Lifecycle Event Timeline Open Started: 10.28s (10279628717) Open Finished: 10.28s (10279628717) First Batch Requested: 10.29s (10287628428) First Batch Returned: 10.29s (10287628428) Last Batch Returned: 10.29s (10287628428) Closed: 10.29s (10290628320) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 16.0 KiB (16384) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 16.0 KiB (16384) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 16.0 KiB (16384) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Dequeue - FirstBatchWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBytesDequeued: 125 B (125) - TotalGetBatchTime: 0ns (0) - DataWaitTime: 0ns (0) - TotalTime: 0ns (0) Enqueue - DeserializeRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBatchesEnqueued: 1 (1) - TotalBatchesReceived: 1 (1) - TotalBytesReceived: 78 B (78) - TotalEarlySenders: 0 (0) - TotalEosReceived: 1 (1) - TotalHasDeferredRPCsTime: 0ns (0) - TotalRPCsDeferred: 0 (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 2 (2) - BuildTime: 29ms (29998916) - InactiveTotalTime: 0ns (0) - LocalTime: 58ms (58997868) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2074880) - ProbeRows: 2 (2) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 2 (2) - RowsReturnedRate: 33 per second (33) - TotalTime: 58ms (58997868) Node Lifecycle Event Timeline Open Started: 10.30s (10298628031) Open Finished: 10.35s (10351626117) First Batch Requested: 10.35s (10351626117) First Batch Returned: 10.35s (10352626081) Last Batch Returned: 10.35s (10352626081) Closed: 10.35s (10353626045) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 192.0 KiB (196608) - CumulativeAllocations: 3 (3) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 192.0 KiB (196608) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 2 (2) - HashTablesBuildTime: 1ms (1999927) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 50 (50) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) Hash Table - HashBuckets: 2 (2) - HashCollisions: 0 (0) - InactiveTotalTime: 0ns (0) - Probes: 4 (4) - Resizes: 0 (0) - TotalTime: 0ns (0) - Travel: 0 (0) EXCHANGE_NODE (id=10) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 32.0 KiB (32768) - RowsReturned: 2 (2) - RowsReturnedRate: 0 per second (0) - TotalTime: 0ns (0) Node Lifecycle Event Timeline Open Started: 10.31s (10313627489) Open Finished: 10.31s (10313627489) First Batch Requested: 10.34s (10336626659) First Batch Returned: 10.34s (10336626659) Last Batch Returned: 10.34s (10336626659) Closed: 10.35s (10345626333) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 32.0 KiB (32768) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 32.0 KiB (32768) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 32.0 KiB (32768) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Dequeue - FirstBatchWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBytesDequeued: 26 B (26) - TotalGetBatchTime: 0ns (0) - DataWaitTime: 0ns (0) - TotalTime: 0ns (0) Enqueue - DeserializeRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBatchesEnqueued: 2 (2) - TotalBatchesReceived: 2 (2) - TotalBytesReceived: 18 B (18) - TotalEarlySenders: 0 (0) - TotalEosReceived: 3 (3) - TotalHasDeferredRPCsTime: 0ns (0) - TotalRPCsDeferred: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=9) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 32.0 KiB (32768) - RowsReturned: 2 (2) - RowsReturnedRate: 0 per second (0) - TotalTime: 0ns (0) Node Lifecycle Event Timeline Open Started: 10.32s (10322627164) Open Finished: 10.32s (10322627164) First Batch Requested: 10.35s (10351626117) First Batch Returned: 10.35s (10351626117) Last Batch Returned: 10.35s (10351626117) Closed: 10.35s (10353626045) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 32.0 KiB (32768) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 32.0 KiB (32768) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 32.0 KiB (32768) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Dequeue - FirstBatchWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBytesDequeued: 42 B (42) - TotalGetBatchTime: 0ns (0) - DataWaitTime: 0ns (0) - TotalTime: 0ns (0) Enqueue - DeserializeRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBatchesEnqueued: 2 (2) - TotalBatchesReceived: 2 (2) - TotalBytesReceived: 34 B (34) - TotalEarlySenders: 0 (0) - TotalEosReceived: 3 (3) - TotalHasDeferredRPCsTime: 0ns (0) - TotalRPCsDeferred: 0 (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 2:1/2.98 KB 1:1/2.98 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 2 out of 2 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:2 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 6.2 KiB (6322) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 6.2 KiB (6322) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 6.2 KiB (6322) - CachedFileHandlesHitCount: 6 (6) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 B (0) - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 B (0) - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaterializeTupleTime(*): 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 4 (4) - NumRowGroups: 2 (2) - NumRowGroupsWithPageIndex: 2 (2) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 61.0 KiB (62464) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 6.0 MiB/s (6322227) - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 26.0 KiB (26624) - RowBatchQueueGetWaitTime: 21ms (21999205) - RowBatchQueuePeakMemoryUsage: 37.0 KiB (37888) - RowBatchQueuePutWaitTime: 0ns (0) - RowBatchesEnqueued: 6 (6) - RowsRead: 4 (4) - RowsReturned: 4 (4) - RowsReturnedRate: 111 per second (111) - ScanRangesComplete: 2 (2) - ScannerIoWaitTime: 1ms (1999928) - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 21ms (21999205) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 19ms (19819000) - ScannerThreadsVoluntaryContextSwitches: 4 (4) - TotalRawHdfsOpenFileTime(*): 0ns (0) - TotalRawHdfsReadTime(*): 999.96us (999964) - TotalReadThroughput: 0 B/s (0) - TotalTime: 35ms (35998699) Node Lifecycle Event Timeline Open Started: 10.25s (10249629801) Open Finished: 10.25s (10254629620) First Batch Requested: 10.25s (10254629620) First Batch Returned: 10.27s (10272628970) Last Batch Returned: 10.28s (10278628753) Closed: 10.28s (10279628717) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 48.0 KiB (49152) - CumulativeAllocations: 6 (6) - InactiveTotalTime: 0ns (0) - PeakReservation: 32.0 KiB (32768) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 16.0 KiB (16384) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 843 (843) - CodegenTotalWallClockTime: 10.14s (10143633629) - CodegenSysTime: 5ms (5949000) - CodegenUserTime: 5.71s (5712174000) - CodegenVoluntaryContextSwitches: 3 (3) - CompileTime: 589ms (589978691) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 395ms (395985698) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 101 (101) - NumInstructions: 4,052 (4052) - OptimizationTime: 8.25s (8252701926) - PeakMemoryUsage: 2.0 MiB (2074624) - PrepareTime: 885ms (885968000) - TotalTime: 10.14s (10143633629) Instance 394ed7e2a194714b:31ab5b2f00000008 (host=impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22002) Last report received time: 2019-10-26 04:26:21.391 Hdfs split stats (:<# splits>/): 2:1/2.98 KB - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 1.0 MiB (1048576) - ExchangeScanRatio: 0.03 (0.0335336918696615) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 35.2 MiB (36916656) - PeakReservation: 35.0 MiB (36700160) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 38.8 MiB (40659160) - RowsProduced: 8 (8) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 19ms (19999277) - TotalStorageWaitTime: 9ms (9999639) - TotalThreadsInvoluntaryContextSwitches: 768 (768) - TotalThreadsTotalWallClockTime: 6.17s (6170777121) - TotalThreadsSysTime: 24ms (24641000) - TotalThreadsUserTime: 3.89s (3887098000) - TotalThreadsVoluntaryContextSwitches: 12 (12) - TotalTime: 6.17s (6166777267) Fragment Instance Lifecycle Event Timeline Prepare Finished: 58ms (58997869) Open Finished: 6.03s (6027782287) First Batch Produced: 6.15s (6153777736) First Batch Sent: 6.15s (6153777736) ExecInternal Finished: 6.18s (6178776833) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 1.0 MiB (1048576) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.0 MiB (1048576) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 1.0 MiB (1048576) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 1.0 MiB (1048576) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 148ms (148994618) - ExecTreeExecTime: 124ms (124995485) - InactiveTotalTime: 0ns (0) - OpenTime: 5.97s (5968784418) - ExecTreeOpenTime: 2ms (2999892) - PrepareTime: 47ms (47998267) - ExecTreePrepareTime: 40ms (40998519) - TotalTime: 0ns (0) KrpcDataStreamSender (dst_id=11) ExecOption: Unpartitioned Sender Codegen Disabled: not needed - EosSent: 1 (1) - InactiveTotalTime: 20ms (20999241) - PeakMemoryUsage: 7.5 KiB (7704) - RowsSent: 8 (8) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0ns (0) - TotalBytesSent: 106 B (106) - TotalTime: 22ms (22999169) - UncompressedRowBatchSize: 200 B (200) UNION_NODE (id=0) ExecOption: Codegen Enabled - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 25.0 KiB (25600) - RowsReturned: 8 (8) - RowsReturnedRate: 47 per second (47) - TotalTime: 168ms (168993896) Node Lifecycle Event Timeline Open Started: 6.02s (6024782395) Open Finished: 6.03s (6027782287) First Batch Requested: 6.03s (6027782287) First Batch Returned: 6.15s (6152777772) Last Batch Returned: 6.15s (6153777736) Closed: 6.18s (6176776905) AGGREGATION_NODE (id=8) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.0 MiB (35677312) - RowsReturned: 3 (3) - RowsReturnedRate: 176 per second (176) - TotalTime: 16ms (16999386) Node Lifecycle Event Timeline Open Started: 6.09s (6092779939) Open Finished: 6.11s (6105779470) First Batch Requested: 6.11s (6105779470) First Batch Returned: 6.11s (6105779470) Last Batch Returned: 6.11s (6107779397) Closed: 6.11s (6108779361) GroupingAggregator 0 ExecOption: Codegen Enabled - BuildTime: 1ms (1999928) - GetResultsTime: 999.96us (999963) - HTResizeTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 33 (33) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.0 MiB (35673216) - PeakReservation: 34.0 MiB (35651584) - PeakUsedReservation: 0 B (0) - RowsRepartitioned: 0 (0) - RowsReturned: 3 (3) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 2ms (2999892) - CumulativeAllocationBytes: 8.0 MiB (8388608) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.0 MiB (8388608) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 2ms (2999892) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Table - HashBuckets: 3,072 (3072) - HashCollisions: 0 (0) - InactiveTotalTime: 0ns (0) - Probes: 3 (3) - Resizes: 0 (0) - TotalTime: 0ns (0) - Travel: 0 (0) EXCHANGE_NODE (id=7) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 16.0 KiB (16384) - RowsReturned: 3 (3) - RowsReturnedRate: 0 per second (0) - TotalTime: 0ns (0) Node Lifecycle Event Timeline Open Started: 6.09s (6092779939) Open Finished: 6.09s (6092779939) First Batch Requested: 6.10s (6100779650) First Batch Returned: 6.10s (6100779650) Last Batch Returned: 6.10s (6100779650) Closed: 6.10s (6102779578) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 16.0 KiB (16384) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 16.0 KiB (16384) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 16.0 KiB (16384) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Dequeue - FirstBatchWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBytesDequeued: 75 B (75) - TotalGetBatchTime: 0ns (0) - DataWaitTime: 0ns (0) - TotalTime: 0ns (0) Enqueue - DeserializeRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBatchesEnqueued: 1 (1) - TotalBatchesReceived: 1 (1) - TotalBytesReceived: 53 B (53) - TotalEarlySenders: 0 (0) - TotalEosReceived: 1 (1) - TotalHasDeferredRPCsTime: 0ns (0) - TotalRPCsDeferred: 0 (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 2 (2) - BuildTime: 21ms (21999205) - InactiveTotalTime: 0ns (0) - LocalTime: 110ms (110995991) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2074880) - ProbeRows: 2 (2) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 2 (2) - RowsReturnedRate: 18 per second (18) - TotalTime: 110ms (110995991) Node Lifecycle Event Timeline Open Started: 6.11s (6108779361) Open Finished: 6.15s (6151777808) First Batch Requested: 6.15s (6151777808) First Batch Returned: 6.15s (6152777772) Last Batch Returned: 6.15s (6152777772) Closed: 6.15s (6152777772) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 192.0 KiB (196608) - CumulativeAllocations: 3 (3) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 192.0 KiB (196608) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 2 (2) - HashTablesBuildTime: 999.96us (999964) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 50 (50) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) Hash Table - HashBuckets: 2 (2) - HashCollisions: 0 (0) - InactiveTotalTime: 0ns (0) - Probes: 4 (4) - Resizes: 0 (0) - TotalTime: 0ns (0) - Travel: 0 (0) EXCHANGE_NODE (id=10) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 32.0 KiB (32768) - RowsReturned: 2 (2) - RowsReturnedRate: 0 per second (0) - TotalTime: 0ns (0) Node Lifecycle Event Timeline Open Started: 6.13s (6127778675) Open Finished: 6.13s (6127778675) First Batch Requested: 6.13s (6135778386) First Batch Returned: 6.13s (6135778386) Last Batch Returned: 6.13s (6135778386) Closed: 6.15s (6150777844) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 32.0 KiB (32768) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 32.0 KiB (32768) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 32.0 KiB (32768) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Dequeue - FirstBatchWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBytesDequeued: 26 B (26) - TotalGetBatchTime: 0ns (0) - DataWaitTime: 0ns (0) - TotalTime: 0ns (0) Enqueue - DeserializeRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBatchesEnqueued: 2 (2) - TotalBatchesReceived: 2 (2) - TotalBytesReceived: 18 B (18) - TotalEarlySenders: 0 (0) - TotalEosReceived: 3 (3) - TotalHasDeferredRPCsTime: 0ns (0) - TotalRPCsDeferred: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=9) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 32.0 KiB (32768) - RowsReturned: 2 (2) - RowsReturnedRate: 0 per second (0) - TotalTime: 0ns (0) Node Lifecycle Event Timeline Open Started: 6.14s (6136778350) Open Finished: 6.14s (6136778350) First Batch Requested: 6.15s (6151777808) First Batch Returned: 6.15s (6151777808) Last Batch Returned: 6.15s (6151777808) Closed: 6.15s (6152777772) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 32.0 KiB (32768) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 32.0 KiB (32768) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 32.0 KiB (32768) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Dequeue - FirstBatchWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBytesDequeued: 42 B (42) - TotalGetBatchTime: 0ns (0) - DataWaitTime: 0ns (0) - TotalTime: 0ns (0) Enqueue - DeserializeRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBatchesEnqueued: 2 (2) - TotalBatchesReceived: 2 (2) - TotalBytesReceived: 34 B (34) - TotalEarlySenders: 0 (0) - TotalEosReceived: 3 (3) - TotalHasDeferredRPCsTime: 0ns (0) - TotalRPCsDeferred: 0 (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 2:1/2.98 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:1 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 3.1 KiB (3161) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 3.1 KiB (3161) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 3.1 KiB (3161) - CachedFileHandlesHitCount: 3 (3) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 B (0) - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 B (0) - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaterializeTupleTime(*): 999.96us (999963) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 57.0 KiB (58368) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 13.0 KiB (13312) - RowBatchQueueGetWaitTime: 21ms (21999205) - RowBatchQueuePeakMemoryUsage: 37.0 KiB (37888) - RowBatchQueuePutWaitTime: 0ns (0) - RowBatchesEnqueued: 3 (3) - RowsRead: 2 (2) - RowsReturned: 2 (2) - RowsReturnedRate: 27 per second (27) - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 9ms (9999639) - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 30ms (30998880) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 7ms (7847000) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 0ns (0) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 71ms (71997398) Node Lifecycle Event Timeline Open Started: 6.02s (6024782395) Open Finished: 6.03s (6027782287) First Batch Requested: 6.03s (6028782251) First Batch Returned: 6.09s (6091779975) Last Batch Returned: 6.09s (6092779939) Closed: 6.09s (6092779939) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 24.0 KiB (24576) - CumulativeAllocations: 3 (3) - InactiveTotalTime: 0ns (0) - PeakReservation: 16.0 KiB (16384) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 16.0 KiB (16384) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 759 (759) - CodegenTotalWallClockTime: 5.96s (5964784562) - CodegenSysTime: 7ms (7913000) - CodegenUserTime: 3.84s (3843271000) - CodegenVoluntaryContextSwitches: 0 (0) - CompileTime: 422ms (422984722) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 252ms (252990862) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 101 (101) - NumInstructions: 4,052 (4052) - OptimizationTime: 4.71s (4713829745) - PeakMemoryUsage: 2.0 MiB (2074624) - PrepareTime: 555ms (555979919) - TotalTime: 5.96s (5964784562) Instance 394ed7e2a194714b:31ab5b2f00000009 (host=impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22001) Last report received time: 2019-10-26 04:26:21.076 Hdfs split stats (:<# splits>/): 0:1/2.98 KB - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 1.0 MiB (1048576) - ExchangeScanRatio: 0.03 (0.03005378044922493) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 35.2 MiB (36916656) - PeakReservation: 35.0 MiB (36700160) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 38.8 MiB (40659160) - RowsProduced: 8 (8) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 41ms (41998483) - TotalStorageWaitTime: 1ms (1999927) - TotalThreadsInvoluntaryContextSwitches: 701 (701) - TotalThreadsTotalWallClockTime: 5.85s (5848788753) - TotalThreadsSysTime: 24ms (24607000) - TotalThreadsUserTime: 3.63s (3630138000) - TotalThreadsVoluntaryContextSwitches: 7 (7) - TotalTime: 5.84s (5843788933) Fragment Instance Lifecycle Event Timeline Prepare Finished: 41ms (41998483) Open Finished: 5.76s (5764791786) First Batch Produced: 5.83s (5826789547) First Batch Sent: 5.83s (5826789547) ExecInternal Finished: 5.87s (5872787885) Buffer pool - AllocTime: 999.96us (999964) - CumulativeAllocationBytes: 1.0 MiB (1048576) - CumulativeAllocations: 1 (1) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.0 MiB (1048576) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 1.0 MiB (1048576) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 1.0 MiB (1048576) - SystemAllocTime: 999.96us (999964) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 104ms (104996208) - ExecTreeExecTime: 61ms (61997761) - InactiveTotalTime: 0ns (0) - OpenTime: 5.72s (5722793303) - ExecTreeOpenTime: 2ms (2999892) - PrepareTime: 14ms (14999458) - ExecTreePrepareTime: 8ms (8999675) - TotalTime: 0ns (0) KrpcDataStreamSender (dst_id=11) ExecOption: Unpartitioned Sender Codegen Disabled: not needed - EosSent: 1 (1) - InactiveTotalTime: 41ms (41998483) - PeakMemoryUsage: 7.5 KiB (7704) - RowsSent: 8 (8) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0ns (0) - TotalBytesSent: 95 B (95) - TotalTime: 44ms (44998374) - UncompressedRowBatchSize: 200 B (200) UNION_NODE (id=0) ExecOption: Codegen Enabled - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 25.0 KiB (25600) - RowsReturned: 8 (8) - RowsReturnedRate: 108 per second (108) - TotalTime: 73ms (73997328) Node Lifecycle Event Timeline Open Started: 5.76s (5761791894) Open Finished: 5.76s (5764791786) First Batch Requested: 5.76s (5764791786) First Batch Returned: 5.83s (5826789547) Last Batch Returned: 5.83s (5826789547) Closed: 5.87s (5870787957) AGGREGATION_NODE (id=8) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.0 MiB (35677312) - RowsReturned: 2 (2) - RowsReturnedRate: 142 per second (142) - TotalTime: 13ms (13999495) Node Lifecycle Event Timeline Open Started: 5.79s (5787790955) Open Finished: 5.80s (5798790558) First Batch Requested: 5.80s (5798790558) First Batch Returned: 5.80s (5798790558) Last Batch Returned: 5.80s (5800790486) Closed: 5.80s (5800790486) GroupingAggregator 0 ExecOption: Codegen Enabled - BuildTime: 999.96us (999964) - GetResultsTime: 0ns (0) - HTResizeTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 50 (50) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.0 MiB (35673216) - PeakReservation: 34.0 MiB (35651584) - PeakUsedReservation: 0 B (0) - RowsRepartitioned: 0 (0) - RowsReturned: 2 (2) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) Buffer pool - AllocTime: 999.96us (999964) - CumulativeAllocationBytes: 6.0 MiB (6291456) - CumulativeAllocations: 3 (3) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 6.0 MiB (6291456) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 999.96us (999964) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Table - HashBuckets: 2,048 (2048) - HashCollisions: 0 (0) - InactiveTotalTime: 0ns (0) - Probes: 2 (2) - Resizes: 0 (0) - TotalTime: 0ns (0) - Travel: 0 (0) EXCHANGE_NODE (id=7) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 16.0 KiB (16384) - RowsReturned: 2 (2) - RowsReturnedRate: 0 per second (0) - TotalTime: 0ns (0) Node Lifecycle Event Timeline Open Started: 5.79s (5787790955) Open Finished: 5.79s (5787790955) First Batch Requested: 5.79s (5794790702) First Batch Returned: 5.79s (5794790702) Last Batch Returned: 5.79s (5794790702) Closed: 5.79s (5795790666) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 16.0 KiB (16384) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 16.0 KiB (16384) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 16.0 KiB (16384) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Dequeue - FirstBatchWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBytesDequeued: 50 B (50) - TotalGetBatchTime: 0ns (0) - DataWaitTime: 0ns (0) - TotalTime: 0ns (0) Enqueue - DeserializeRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBatchesEnqueued: 1 (1) - TotalBatchesReceived: 1 (1) - TotalBytesReceived: 36 B (36) - TotalEarlySenders: 0 (0) - TotalEosReceived: 1 (1) - TotalHasDeferredRPCsTime: 0ns (0) - TotalRPCsDeferred: 0 (0) - TotalTime: 0ns (0) HASH_JOIN_NODE (id=6) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 4 (4) - BuildTime: 10ms (10999603) - InactiveTotalTime: 0ns (0) - LocalTime: 27ms (27998988) - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 2.0 MiB (2074880) - ProbeRows: 4 (4) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 0ns (0) - RowsReturned: 4 (4) - RowsReturnedRate: 137 per second (137) - TotalTime: 28ms (28998952) Node Lifecycle Event Timeline Open Started: 5.80s (5801790449) Open Finished: 5.83s (5825789583) First Batch Requested: 5.83s (5825789583) First Batch Returned: 5.83s (5825789583) Last Batch Returned: 5.83s (5825789583) Closed: 5.83s (5826789547) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 320.0 KiB (327680) - CumulativeAllocations: 5 (5) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.9 MiB (2031616) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 320.0 KiB (327680) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Join Builder (join_node_id=6) ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 0ns (0) - BuildRowsPartitioned: 4 (4) - HashTablesBuildTime: 999.96us (999964) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 25 (25) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 17.1 KiB (17536) - RepartitionTime: 0ns (0) - SpilledPartitions: 0 (0) - TotalTime: 0ns (0) Hash Table - HashBuckets: 4 (4) - HashCollisions: 0 (0) - InactiveTotalTime: 0ns (0) - Probes: 8 (8) - Resizes: 0 (0) - TotalTime: 0ns (0) - Travel: 0 (0) EXCHANGE_NODE (id=10) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 32.0 KiB (32768) - RowsReturned: 4 (4) - RowsReturnedRate: 4000 per second (4000) - TotalTime: 999.96us (999964) Node Lifecycle Event Timeline Open Started: 5.81s (5809790160) Open Finished: 5.81s (5809790160) First Batch Requested: 5.82s (5816789908) First Batch Returned: 5.82s (5817789872) Last Batch Returned: 5.82s (5817789872) Closed: 5.82s (5822789691) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 32.0 KiB (32768) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 32.0 KiB (32768) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 32.0 KiB (32768) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Dequeue - FirstBatchWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBytesDequeued: 52 B (52) - TotalGetBatchTime: 0ns (0) - DataWaitTime: 0ns (0) - TotalTime: 0ns (0) Enqueue - DeserializeRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBatchesEnqueued: 2 (2) - TotalBatchesReceived: 2 (2) - TotalBytesReceived: 36 B (36) - TotalEarlySenders: 0 (0) - TotalEosReceived: 3 (3) - TotalHasDeferredRPCsTime: 0ns (0) - TotalRPCsDeferred: 0 (0) - TotalTime: 0ns (0) EXCHANGE_NODE (id=9) - ConvertRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 32.0 KiB (32768) - RowsReturned: 4 (4) - RowsReturnedRate: 4000 per second (4000) - TotalTime: 999.96us (999964) Node Lifecycle Event Timeline Open Started: 5.82s (5824789619) Open Finished: 5.82s (5824789619) First Batch Requested: 5.82s (5824789619) First Batch Returned: 5.83s (5825789583) Last Batch Returned: 5.83s (5825789583) Closed: 5.83s (5826789547) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 32.0 KiB (32768) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 32.0 KiB (32768) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 32.0 KiB (32768) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Dequeue - FirstBatchWaitTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBytesDequeued: 84 B (84) - TotalGetBatchTime: 0ns (0) - DataWaitTime: 0ns (0) - TotalTime: 0ns (0) Enqueue - DeserializeRowBatchTime: 0ns (0) - InactiveTotalTime: 0ns (0) - TotalBatchesEnqueued: 2 (2) - TotalBatchesReceived: 2 (2) - TotalBytesReceived: 54 B (54) - TotalEarlySenders: 0 (0) - TotalEosReceived: 3 (3) - TotalHasDeferredRPCsTime: 0ns (0) - TotalRPCsDeferred: 0 (0) - TotalTime: 0ns (0) HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 0:1/2.98 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:1 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 3.1 KiB (3161) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 3.1 KiB (3161) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 3.1 KiB (3161) - CachedFileHandlesHitCount: 3 (3) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 B (0) - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 B (0) - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaterializeTupleTime(*): 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 57.0 KiB (58368) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 1.5 MiB/s (1580557) - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 13.0 KiB (13312) - RowBatchQueueGetWaitTime: 7ms (7999711) - RowBatchQueuePeakMemoryUsage: 37.0 KiB (37888) - RowBatchQueuePutWaitTime: 0ns (0) - RowBatchesEnqueued: 3 (3) - RowsRead: 2 (2) - RowsReturned: 2 (2) - RowsReturnedRate: 62 per second (62) - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 1ms (1999927) - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 7ms (7999711) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 6ms (6786000) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 0ns (0) - TotalRawHdfsReadTime(*): 1ms (1999927) - TotalReadThroughput: 0 B/s (0) - TotalTime: 31ms (31998845) Node Lifecycle Event Timeline Open Started: 5.76s (5761791894) Open Finished: 5.76s (5764791786) First Batch Requested: 5.76s (5764791786) First Batch Returned: 5.79s (5786790991) Last Batch Returned: 5.79s (5787790955) Closed: 5.79s (5787790955) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 24.0 KiB (24576) - CumulativeAllocations: 3 (3) - InactiveTotalTime: 0ns (0) - PeakReservation: 16.0 KiB (16384) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 16.0 KiB (16384) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 688 (688) - CodegenTotalWallClockTime: 5.72s (5718793447) - CodegenSysTime: 7ms (7867000) - CodegenUserTime: 3.59s (3592078000) - CodegenVoluntaryContextSwitches: 0 (0) - CompileTime: 208ms (208992451) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 270ms (270990213) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 101 (101) - NumInstructions: 4,052 (4052) - OptimizationTime: 4.73s (4729829167) - PeakMemoryUsage: 2.0 MiB (2074624) - PrepareTime: 500ms (500981905) - TotalTime: 5.72s (5718793447) Averaged Fragment F01 split sizes: min: 7.71 KB, max: 7.71 KB, avg: 7.71 KB, stddev: 0 completion times: min:5s319ms max:5s319ms mean: 5s319ms stddev:0.000ns execution rates: min:1.45 KB/sec max:1.45 KB/sec mean:1.45 KB/sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - ExchangeScanRatio: 0.02 (0.02050589390962672) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.1 MiB (35772923) - PeakReservation: 34.0 MiB (35667968) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 73.5 MiB (77039702) - RowsProduced: 10 (10) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 39ms (39998555) - TotalStorageWaitTime: 999.96us (999963) - TotalThreadsInvoluntaryContextSwitches: 551 (551) - TotalThreadsTotalWallClockTime: 3.43s (3432876010) - TotalThreadsSysTime: 35ms (35382000) - TotalThreadsUserTime: 2.57s (2567588000) - TotalThreadsVoluntaryContextSwitches: 12 (12) - TotalTime: 3.44s (3440875722) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 0 B (0) - CumulativeAllocations: 0 (0) - InactiveTotalTime: 0ns (0) - PeakReservation: 0 B (0) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 0 B (0) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 0 B (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 120ms (120995630) - ExecTreeExecTime: 79ms (79997111) - InactiveTotalTime: 0ns (0) - OpenTime: 3.29s (3289881175) - ExecTreeOpenTime: 23ms (23999133) - PrepareTime: 28ms (28998953) - ExecTreePrepareTime: 19ms (19999277) - TotalTime: 0ns (0) KrpcDataStreamSender (dst_id=7) - EosSent: 3 (3) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.6 KiB (35400) - RowsSent: 10 (10) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0ns (0) - TotalBytesSent: 167 B (167) - TotalTime: 43ms (43998411) - UncompressedRowBatchSize: 250 B (250) AGGREGATION_NODE (id=3) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.0 MiB (35690624) - RowsReturned: 10 (10) - RowsReturnedRate: 80 per second (80) - TotalTime: 123ms (123995521) GroupingAggregator 0 - BuildTime: 0ns (0) - GetResultsTime: 10ms (10999603) - HTResizeTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.0 MiB (35669120) - PeakReservation: 34.0 MiB (35651584) - PeakUsedReservation: 0 B (0) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 10 (10) - StreamingTime: 3ms (3999856) - TotalTime: 0ns (0) Buffer pool - AllocTime: 3ms (3999856) - CumulativeAllocationBytes: 22.0 MiB (23068672) - CumulativeAllocations: 11 (11) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 22.0 MiB (23068672) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 3ms (3999856) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Table - HashBuckets: 10,240 (10240) - HashCollisions: 0 (0) - InactiveTotalTime: 0ns (0) - Probes: 280 (280) - Resizes: 0 (0) - TotalTime: 0ns (0) - Travel: 0 (0) HDFS_SCAN_NODE (id=2) - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 8.0 KiB (8144) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 8.0 KiB (8144) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 8.0 KiB (8144) - CachedFileHandlesHitCount: 3 (3) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 B (0) - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 B (0) - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaterializeTupleTime(*): 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 57.0 KiB (58368) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 13.0 KiB (13312) - RowBatchQueueGetWaitTime: 28ms (28998952) - RowBatchQueuePeakMemoryUsage: 37.0 KiB (37888) - RowBatchQueuePutWaitTime: 0ns (0) - RowBatchesEnqueued: 3 (3) - RowsRead: 280 (280) - RowsReturned: 280 (280) - RowsReturnedRate: 3889 per second (3889) - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 999.96us (999963) - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 21ms (21999205) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 10ms (10984000) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 0ns (0) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 71ms (71997401) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 24.0 KiB (24576) - CumulativeAllocations: 3 (3) - InactiveTotalTime: 0ns (0) - PeakReservation: 16.0 KiB (16384) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 16.0 KiB (16384) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 508 (508) - CodegenTotalWallClockTime: 3.26s (3263882114) - CodegenSysTime: 3ms (3928000) - CodegenUserTime: 2.51s (2510359000) - CodegenVoluntaryContextSwitches: 1 (1) - CompileTime: 84ms (84996930) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 183ms (183993354) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 51 (51) - NumInstructions: 1,216 (1216) - OptimizationTime: 2.21s (2206920290) - PeakMemoryUsage: 608.0 KiB (622592) - PrepareTime: 778ms (778971865) - TotalTime: 3.26s (3263882114) Fragment F01 Instance 394ed7e2a194714b:31ab5b2f00000001 (host=impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22000) Last report received time: 2019-10-26 04:26:20.510 Hdfs split stats (:<# splits>/): 0:1/7.71 KB - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - ExchangeScanRatio: 0.02 (0.02050589390962672) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.1 MiB (35772923) - PeakReservation: 34.0 MiB (35667968) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 73.5 MiB (77039702) - RowsProduced: 10 (10) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 39ms (39998555) - TotalStorageWaitTime: 999.96us (999963) - TotalThreadsInvoluntaryContextSwitches: 551 (551) - TotalThreadsTotalWallClockTime: 3.43s (3432876010) - TotalThreadsSysTime: 35ms (35382000) - TotalThreadsUserTime: 2.57s (2567588000) - TotalThreadsVoluntaryContextSwitches: 12 (12) - TotalTime: 3.44s (3440875722) Fragment Instance Lifecycle Event Timeline Prepare Finished: 103ms (103996244) Open Finished: 3.39s (3393877419) First Batch Produced: 3.46s (3461874963) First Batch Sent: 3.46s (3461874963) ExecInternal Finished: 3.52s (3519872869) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 0 B (0) - CumulativeAllocations: 0 (0) - InactiveTotalTime: 0ns (0) - PeakReservation: 0 B (0) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 0 B (0) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 0 B (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 120ms (120995630) - ExecTreeExecTime: 79ms (79997111) - InactiveTotalTime: 0ns (0) - OpenTime: 3.29s (3289881175) - ExecTreeOpenTime: 23ms (23999133) - PrepareTime: 28ms (28998953) - ExecTreePrepareTime: 19ms (19999277) - TotalTime: 0ns (0) KrpcDataStreamSender (dst_id=7) ExecOption: Hash Partitioned Sender Codegen Enabled - EosSent: 3 (3) - InactiveTotalTime: 39ms (39998555) - PeakMemoryUsage: 34.6 KiB (35400) - RowsSent: 10 (10) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0ns (0) - TotalBytesSent: 167 B (167) - TotalTime: 43ms (43998411) - UncompressedRowBatchSize: 250 B (250) AGGREGATION_NODE (id=3) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 34.0 MiB (35690624) - RowsReturned: 10 (10) - RowsReturnedRate: 80 per second (80) - TotalTime: 123ms (123995521) Node Lifecycle Event Timeline Open Started: 3.37s (3369878286) Open Finished: 3.39s (3393877419) First Batch Requested: 3.39s (3393877419) First Batch Returned: 3.46s (3461874963) Last Batch Returned: 3.47s (3473874530) Closed: 3.52s (3517872941) GroupingAggregator 0 ExecOption: Streaming Preaggregation, Codegen Enabled - BuildTime: 0ns (0) - GetResultsTime: 10ms (10999603) - HTResizeTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LargestPartitionPercent: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 34.0 MiB (35669120) - PeakReservation: 34.0 MiB (35651584) - PeakUsedReservation: 0 B (0) - ReductionFactorEstimate: 0.00 (0.0) - ReductionFactorThresholdToExpand: 0.00 (0.0) - RowsPassedThrough: 0 (0) - RowsReturned: 10 (10) - StreamingTime: 3ms (3999856) - TotalTime: 0ns (0) Buffer pool - AllocTime: 3ms (3999856) - CumulativeAllocationBytes: 22.0 MiB (23068672) - CumulativeAllocations: 11 (11) - InactiveTotalTime: 0ns (0) - PeakReservation: 34.0 MiB (35651584) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 22.0 MiB (23068672) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 3ms (3999856) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Hash Table - HashBuckets: 10,240 (10240) - HashCollisions: 0 (0) - InactiveTotalTime: 0ns (0) - Probes: 280 (280) - Resizes: 0 (0) - TotalTime: 0ns (0) - Travel: 0 (0) HDFS_SCAN_NODE (id=2) Hdfs split stats (:<# splits>/): 0:1/7.71 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:1 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 8.0 KiB (8144) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 8.0 KiB (8144) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 8.0 KiB (8144) - CachedFileHandlesHitCount: 3 (3) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 B (0) - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 B (0) - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaterializeTupleTime(*): 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 57.0 KiB (58368) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 13.0 KiB (13312) - RowBatchQueueGetWaitTime: 28ms (28998952) - RowBatchQueuePeakMemoryUsage: 37.0 KiB (37888) - RowBatchQueuePutWaitTime: 0ns (0) - RowBatchesEnqueued: 3 (3) - RowsRead: 280 (280) - RowsReturned: 280 (280) - RowsReturnedRate: 3889 per second (3889) - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 999.96us (999963) - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 21ms (21999205) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 10ms (10984000) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 0ns (0) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 71ms (71997401) Node Lifecycle Event Timeline Open Started: 3.37s (3369878286) Open Finished: 3.37s (3374878106) First Batch Requested: 3.39s (3393877419) First Batch Returned: 3.45s (3451875324) Last Batch Returned: 3.46s (3457875108) Closed: 3.46s (3457875108) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 24.0 KiB (24576) - CumulativeAllocations: 3 (3) - InactiveTotalTime: 0ns (0) - PeakReservation: 16.0 KiB (16384) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 16.0 KiB (16384) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 508 (508) - CodegenTotalWallClockTime: 3.26s (3263882114) - CodegenSysTime: 3ms (3928000) - CodegenUserTime: 2.51s (2510359000) - CodegenVoluntaryContextSwitches: 1 (1) - CompileTime: 84ms (84996930) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 183ms (183993354) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 51 (51) - NumInstructions: 1,216 (1216) - OptimizationTime: 2.21s (2206920290) - PeakMemoryUsage: 608.0 KiB (622592) - PrepareTime: 778ms (778971865) - TotalTime: 3.26s (3263882114) Averaged Fragment F03 split sizes: min: 2.98 KB, max: 5.96 KB, avg: 3.97 KB, stddev: 1.40 KB completion times: min:5s132ms max:5s315ms mean: 5s205ms stddev:79.471ms execution rates: min:590.00 B/sec max:1.12 KB/sec mean:777.00 B/sec stddev:262.27 B/sec num instances: 3 - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - ExchangeScanRatio: 0.01 (0.01001792681640831) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.4 MiB (1476128) - PeakReservation: 1.0 MiB (1070421) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 50.3 MiB (52786007) - RowsProduced: 2 (2) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 52ms (52664765) - TotalStorageWaitTime: 8ms (8666354) - TotalThreadsInvoluntaryContextSwitches: 207 (207) - TotalThreadsTotalWallClockTime: 2.35s (2353581659) - TotalThreadsSysTime: 5ms (5953333) - TotalThreadsUserTime: 1.12s (1121824333) - TotalThreadsVoluntaryContextSwitches: 11 (11) - TotalTime: 2.34s (2341582092) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 0 B (0) - CumulativeAllocations: 0 (0) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.0 MiB (1048576) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 0 B (0) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 1.0 MiB (1048576) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 114ms (114329203) - ExecTreeExecTime: 60ms (60331154) - InactiveTotalTime: 0ns (0) - OpenTime: 2.20s (2204920362) - ExecTreeOpenTime: 17ms (17999349) - PrepareTime: 21ms (21332562) - ExecTreePrepareTime: 2ms (2999891) - TotalTime: 0ns (0) KrpcDataStreamSender (dst_id=9) - EosSent: 3 (3) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 37.5 KiB (38432) - RowsSent: 2 (2) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 333.32us (333321) - TotalBytesSent: 40 B (40) - TotalTime: 66ms (66664258) - UncompressedRowBatchSize: 56 B (56) HDFS_SCAN_NODE (id=4) - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 4.1 KiB (4214) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 4.1 KiB (4214) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 4.1 KiB (4214) - CachedFileHandlesHitCount: 4 (4) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 B (0) - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 B (0) - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaterializeTupleTime(*): 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 54.3 KiB (55645) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 2.0 MiB/s (2107409) - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 17.3 KiB (17749) - RowBatchQueueGetWaitTime: 24ms (24999097) - RowBatchQueuePeakMemoryUsage: 31.7 KiB (32426) - RowBatchQueuePutWaitTime: 0ns (0) - RowBatchesEnqueued: 4 (4) - RowsRead: 2 (2) - RowsReturned: 2 (2) - RowsReturnedRate: 30 per second (30) - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 8ms (8666354) - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 34ms (34332093) - ScannerThreadsSysTime: 331.33us (331333) - ScannerThreadsUserTime: 11ms (11848666) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 0ns (0) - TotalRawHdfsReadTime(*): 333.32us (333321) - TotalReadThroughput: 0 B/s (0) - TotalTime: 83ms (83663644) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 32.0 KiB (32768) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 21.3 KiB (21845) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 16.0 KiB (16384) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Filter 0 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 2 (2) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 201 (201) - CodegenTotalWallClockTime: 2.19s (2185921048) - CodegenSysTime: 1ms (1981333) - CodegenUserTime: 1.10s (1099572666) - CodegenVoluntaryContextSwitches: 0 (0) - CompileTime: 109ms (109662706) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 59ms (59997833) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 32 (32) - NumInstructions: 760 (760) - OptimizationTime: 1.30s (1303286260) - PeakMemoryUsage: 380.0 KiB (389120) - PrepareTime: 705ms (705307858) - TotalTime: 2.19s (2185921048) Fragment F03 Instance 394ed7e2a194714b:31ab5b2f00000002 (host=impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22000) Last report received time: 2019-10-26 04:26:20.507 Hdfs split stats (:<# splits>/): 2:1/2.98 KB 1:1/2.98 KB - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - ExchangeScanRatio: 0.01 (0.008541600759253401) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.4 MiB (1476128) - PeakReservation: 1.0 MiB (1081344) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 73.5 MiB (77039702) - RowsProduced: 4 (4) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 57ms (57997905) - TotalStorageWaitTime: 999.96us (999964) - TotalThreadsInvoluntaryContextSwitches: 209 (209) - TotalThreadsTotalWallClockTime: 3.19s (3189884787) - TotalThreadsSysTime: 1ms (1995000) - TotalThreadsUserTime: 1.48s (1481725000) - TotalThreadsVoluntaryContextSwitches: 15 (15) - TotalTime: 3.18s (3184884968) Fragment Instance Lifecycle Event Timeline Prepare Finished: 137ms (137995016) Open Finished: 3.15s (3147886304) First Batch Produced: 3.20s (3201884354) First Batch Sent: 3.20s (3201884354) ExecInternal Finished: 3.28s (3278881573) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 0 B (0) - CumulativeAllocations: 0 (0) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.0 MiB (1048576) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 0 B (0) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 1.0 MiB (1048576) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 127ms (127995377) - ExecTreeExecTime: 67ms (67997544) - InactiveTotalTime: 0ns (0) - OpenTime: 3.01s (3009891288) - ExecTreeOpenTime: 15ms (15999422) - PrepareTime: 45ms (45998339) - ExecTreePrepareTime: 3ms (3999855) - TotalTime: 0ns (0) KrpcDataStreamSender (dst_id=9) ExecOption: Hash Partitioned Sender Codegen Enabled - EosSent: 3 (3) - InactiveTotalTime: 57ms (57997905) - PeakMemoryUsage: 37.5 KiB (38432) - RowsSent: 4 (4) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0ns (0) - TotalBytesSent: 54 B (54) - TotalTime: 95ms (95996533) - UncompressedRowBatchSize: 84 B (84) HDFS_SCAN_NODE (id=4) Hdfs split stats (:<# splits>/): 2:1/2.98 KB 1:1/2.98 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 2 out of 2 Runtime filters: Not all filters arrived (arrived: [], missing [0]), waited for 0. Arrival delay: 3s053ms. Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:2 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 6.2 KiB (6322) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 6.2 KiB (6322) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 6.2 KiB (6322) - CachedFileHandlesHitCount: 6 (6) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 B (0) - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 B (0) - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaterializeTupleTime(*): 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 4 (4) - NumRowGroups: 2 (2) - NumRowGroupsWithPageIndex: 2 (2) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 49.0 KiB (50200) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 6.0 MiB/s (6322227) - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 26.0 KiB (26624) - RowBatchQueueGetWaitTime: 20ms (20999241) - RowBatchQueuePeakMemoryUsage: 21.0 KiB (21504) - RowBatchQueuePutWaitTime: 0ns (0) - RowBatchesEnqueued: 6 (6) - RowsRead: 4 (4) - RowsReturned: 4 (4) - RowsReturnedRate: 43 per second (43) - ScanRangesComplete: 2 (2) - ScannerIoWaitTime: 999.96us (999964) - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 51ms (51998122) - ScannerThreadsSysTime: 994.00us (994000) - ScannerThreadsUserTime: 20ms (20876000) - ScannerThreadsVoluntaryContextSwitches: 5 (5) - TotalRawHdfsOpenFileTime(*): 0ns (0) - TotalRawHdfsReadTime(*): 999.96us (999964) - TotalReadThroughput: 0 B/s (0) - TotalTime: 90ms (90996711) Node Lifecycle Event Timeline Open Started: 3.13s (3131886882) Open Finished: 3.15s (3147886304) First Batch Requested: 3.15s (3147886304) First Batch Returned: 3.20s (3201884354) Last Batch Returned: 3.21s (3215883848) Closed: 3.28s (3276881645) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 48.0 KiB (49152) - CumulativeAllocations: 6 (6) - InactiveTotalTime: 0ns (0) - PeakReservation: 32.0 KiB (32768) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 16.0 KiB (16384) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Filter 0 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 4 (4) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 206 (206) - CodegenTotalWallClockTime: 2.99s (2992891902) - CodegenSysTime: 1ms (1001000) - CodegenUserTime: 1.45s (1447586000) - CodegenVoluntaryContextSwitches: 1 (1) - CompileTime: 130ms (130995269) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 79ms (79997111) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 32 (32) - NumInstructions: 760 (760) - OptimizationTime: 1.72s (1718937915) - PeakMemoryUsage: 380.0 KiB (389120) - PrepareTime: 1.05s (1054961896) - TotalTime: 2.99s (2992891902) Instance 394ed7e2a194714b:31ab5b2f00000004 (host=impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22001) Last report received time: 2019-10-26 04:26:20.346 Hdfs split stats (:<# splits>/): 0:1/2.98 KB - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - ExchangeScanRatio: 0.01 (0.010756089844985764) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.4 MiB (1476128) - PeakReservation: 1.0 MiB (1064960) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 38.8 MiB (40659160) - RowsProduced: 2 (2) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 64ms (64997654) - TotalStorageWaitTime: 7ms (7999711) - TotalThreadsInvoluntaryContextSwitches: 155 (155) - TotalThreadsTotalWallClockTime: 1.99s (1988928163) - TotalThreadsSysTime: 12ms (12926000) - TotalThreadsUserTime: 925ms (925542000) - TotalThreadsVoluntaryContextSwitches: 10 (10) - TotalTime: 1.98s (1977928560) Fragment Instance Lifecycle Event Timeline Prepare Finished: 62ms (62997724) Open Finished: 1.92s (1917930728) First Batch Produced: 1.96s (1962929102) First Batch Sent: 1.96s (1962929102) ExecInternal Finished: 2.03s (2032926574) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 0 B (0) - CumulativeAllocations: 0 (0) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.0 MiB (1048576) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 0 B (0) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 1.0 MiB (1048576) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 112ms (112995918) - ExecTreeExecTime: 45ms (45998338) - InactiveTotalTime: 0ns (0) - OpenTime: 1.85s (1854933004) - ExecTreeOpenTime: 34ms (34998736) - PrepareTime: 8ms (8999674) - ExecTreePrepareTime: 2ms (2999892) - TotalTime: 0ns (0) KrpcDataStreamSender (dst_id=9) ExecOption: Hash Partitioned Sender Codegen Enabled - EosSent: 3 (3) - InactiveTotalTime: 64ms (64997654) - PeakMemoryUsage: 37.5 KiB (38432) - RowsSent: 2 (2) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 999.96us (999963) - TotalBytesSent: 34 B (34) - TotalTime: 67ms (67997543) - UncompressedRowBatchSize: 42 B (42) HDFS_SCAN_NODE (id=4) Hdfs split stats (:<# splits>/): 0:1/2.98 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Runtime filters: Not all filters arrived (arrived: [], missing [0]), waited for 0. Arrival delay: 1s863ms. Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:1 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 3.1 KiB (3161) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 3.1 KiB (3161) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 3.1 KiB (3161) - CachedFileHandlesHitCount: 3 (3) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 B (0) - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 B (0) - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaterializeTupleTime(*): 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 57.0 KiB (58368) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 13.0 KiB (13312) - RowBatchQueueGetWaitTime: 15ms (15999422) - RowBatchQueuePeakMemoryUsage: 37.0 KiB (37888) - RowBatchQueuePutWaitTime: 0ns (0) - RowBatchesEnqueued: 3 (3) - RowsRead: 2 (2) - RowsReturned: 2 (2) - RowsReturnedRate: 22 per second (22) - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 7ms (7999711) - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 20ms (20999241) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 7ms (7169000) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 0ns (0) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 86ms (86996858) Node Lifecycle Event Timeline Open Started: 1.88s (1882931992) Open Finished: 1.92s (1917930728) First Batch Requested: 1.92s (1918930692) First Batch Returned: 1.96s (1962929102) Last Batch Returned: 1.96s (1963929066) Closed: 2.03s (2031926610) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 24.0 KiB (24576) - CumulativeAllocations: 3 (3) - InactiveTotalTime: 0ns (0) - PeakReservation: 16.0 KiB (16384) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 16.0 KiB (16384) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Filter 0 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 2 (2) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 144 (144) - CodegenTotalWallClockTime: 1.82s (1818934304) - CodegenSysTime: 3ms (3965000) - CodegenUserTime: 908ms (908803000) - CodegenVoluntaryContextSwitches: 0 (0) - CompileTime: 91ms (91996678) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 23ms (23999133) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 32 (32) - NumInstructions: 760 (760) - OptimizationTime: 1.20s (1200956623) - PeakMemoryUsage: 380.0 KiB (389120) - PrepareTime: 494ms (494982123) - TotalTime: 1.82s (1818934304) Instance 394ed7e2a194714b:31ab5b2f00000003 (host=impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22002) Last report received time: 2019-10-26 04:26:20.306 Hdfs split stats (:<# splits>/): 2:1/2.98 KB - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - ExchangeScanRatio: 0.01 (0.010756089844985764) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 1.4 MiB (1476128) - PeakReservation: 1.0 MiB (1064960) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 38.8 MiB (40659160) - RowsProduced: 2 (2) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 34ms (34998736) - TotalStorageWaitTime: 16ms (16999387) - TotalThreadsInvoluntaryContextSwitches: 259 (259) - TotalThreadsTotalWallClockTime: 1.88s (1881932028) - TotalThreadsSysTime: 2ms (2939000) - TotalThreadsUserTime: 958ms (958206000) - TotalThreadsVoluntaryContextSwitches: 9 (9) - TotalTime: 1.86s (1861932750) Fragment Instance Lifecycle Event Timeline Prepare Finished: 32ms (32998808) Open Finished: 1.78s (1782935603) First Batch Produced: 1.85s (1848933219) First Batch Sent: 1.85s (1848933219) ExecInternal Finished: 1.89s (1886931847) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 0 B (0) - CumulativeAllocations: 0 (0) - InactiveTotalTime: 0ns (0) - PeakReservation: 1.0 MiB (1048576) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 0 B (0) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 1.0 MiB (1048576) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 101ms (101996316) - ExecTreeExecTime: 66ms (66997580) - InactiveTotalTime: 0ns (0) - OpenTime: 1.75s (1749936795) - ExecTreeOpenTime: 2ms (2999891) - PrepareTime: 8ms (8999675) - ExecTreePrepareTime: 1ms (1999928) - TotalTime: 0ns (0) KrpcDataStreamSender (dst_id=9) ExecOption: Hash Partitioned Sender Codegen Enabled - EosSent: 3 (3) - InactiveTotalTime: 34ms (34998736) - PeakMemoryUsage: 37.5 KiB (38432) - RowsSent: 2 (2) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0ns (0) - TotalBytesSent: 34 B (34) - TotalTime: 35ms (35998700) - UncompressedRowBatchSize: 42 B (42) HDFS_SCAN_NODE (id=4) Hdfs split stats (:<# splits>/): 2:1/2.98 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Runtime filters: Not all filters arrived (arrived: [], missing [0]), waited for 0. Arrival delay: 1s757ms. Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:1 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 3.1 KiB (3161) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 3.1 KiB (3161) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 3.1 KiB (3161) - CachedFileHandlesHitCount: 3 (3) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 B (0) - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 B (0) - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaterializeTupleTime(*): 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 2 (2) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 57.0 KiB (58368) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 13.0 KiB (13312) - RowBatchQueueGetWaitTime: 37ms (37998628) - RowBatchQueuePeakMemoryUsage: 37.0 KiB (37888) - RowBatchQueuePutWaitTime: 0ns (0) - RowBatchesEnqueued: 3 (3) - RowsRead: 2 (2) - RowsReturned: 2 (2) - RowsReturnedRate: 27 per second (27) - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 16ms (16999387) - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 29ms (29998917) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 7ms (7501000) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 0ns (0) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 72ms (72997363) Node Lifecycle Event Timeline Open Started: 1.78s (1779935712) Open Finished: 1.78s (1782935603) First Batch Requested: 1.78s (1782935603) First Batch Returned: 1.85s (1848933219) Last Batch Returned: 1.85s (1849933183) Closed: 1.89s (1885931883) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 24.0 KiB (24576) - CumulativeAllocations: 3 (3) - InactiveTotalTime: 0ns (0) - PeakReservation: 16.0 KiB (16384) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 16.0 KiB (16384) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Filter 0 (1.00 MB) - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - InactiveTotalTime: 0ns (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 2 (2) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) - TotalTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 253 (253) - CodegenTotalWallClockTime: 1.75s (1745936940) - CodegenSysTime: 978.00us (978000) - CodegenUserTime: 942ms (942329000) - CodegenVoluntaryContextSwitches: 0 (0) - CompileTime: 105ms (105996172) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 75ms (75997256) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 32 (32) - NumInstructions: 760 (760) - OptimizationTime: 989ms (989964244) - PeakMemoryUsage: 380.0 KiB (389120) - PrepareTime: 565ms (565979557) - TotalTime: 1.75s (1745936940) Averaged Fragment F04 split sizes: min: 2.98 KB, max: 5.96 KB, avg: 3.97 KB, stddev: 1.40 KB completion times: min:5s118ms max:5s323ms mean: 5s201ms stddev:88.114ms execution rates: min:590.00 B/sec max:1.12 KB/sec mean:777.33 B/sec stddev:260.82 B/sec num instances: 3 - AverageThreadTokens: 1.14 (1.138888888888889) - BloomFilterBytes: 0 B (0) - ExchangeScanRatio: 0.01 (0.005802707930367505) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 307.8 KiB (315168) - PeakReservation: 10.7 KiB (10922) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 50.3 MiB (52786007) - RowsProduced: 2 (2) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 64ms (64664331) - TotalStorageWaitTime: 11ms (11332924) - TotalThreadsInvoluntaryContextSwitches: 175 (175) - TotalThreadsTotalWallClockTime: 2.14s (2140589352) - TotalThreadsSysTime: 13ms (13226666) - TotalThreadsUserTime: 1.02s (1020320000) - TotalThreadsVoluntaryContextSwitches: 11 (11) - TotalTime: 2.12s (2117923504) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 0 B (0) - CumulativeAllocations: 0 (0) - InactiveTotalTime: 0ns (0) - PeakReservation: 0 B (0) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 0 B (0) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 0 B (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 139ms (139994943) - ExecTreeExecTime: 73ms (73997327) - InactiveTotalTime: 0ns (0) - OpenTime: 1.97s (1967928921) - ExecTreeOpenTime: 3ms (3333212) - PrepareTime: 8ms (8999675) - ExecTreePrepareTime: 3ms (3333212) - TotalTime: 0ns (0) KrpcDataStreamSender (dst_id=10) - EosSent: 3 (3) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 84.8 KiB (86816) - RowsSent: 2 (2) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0ns (0) - TotalBytesSent: 24 B (24) - TotalTime: 67ms (67330901) - UncompressedRowBatchSize: 34 B (34) HDFS_SCAN_NODE (id=5) - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.33 (0.3333333333333333) - BytesRead: 4.0 KiB (4136) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 4.0 KiB (4136) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 4.0 KiB (4136) - CachedFileHandlesHitCount: 2 (2) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 B (0) - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 B (0) - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaterializeTupleTime(*): 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 1 (1) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 41.0 KiB (41984) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 6.7 KiB (6826) - RowBatchQueueGetWaitTime: 27ms (27332346) - RowBatchQueuePeakMemoryUsage: 29.0 KiB (29696) - RowBatchQueuePutWaitTime: 0ns (0) - RowBatchesEnqueued: 4 (4) - RowsRead: 2 (2) - RowsReturned: 2 (2) - RowsReturnedRate: 30 per second (30) - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 11ms (11332924) - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 32ms (32665487) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 10ms (10024000) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 333.32us (333321) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 513 B/s (513) - TotalTime: 83ms (83996965) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 21.3 KiB (21845) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 10.7 KiB (10922) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.0 KiB (8192) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 165 (165) - CodegenTotalWallClockTime: 1.96s (1959929210) - CodegenSysTime: 987.67us (987666) - CodegenUserTime: 1.00s (1000125333) - CodegenVoluntaryContextSwitches: 0 (0) - CompileTime: 51ms (51664801) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 36ms (36998663) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 20 (20) - NumInstructions: 446 (446) - OptimizationTime: 1.24s (1243955070) - PeakMemoryUsage: 223.0 KiB (228352) - PrepareTime: 616ms (616311073) - TotalTime: 1.96s (1960262532) Fragment F04 Instance 394ed7e2a194714b:31ab5b2f00000005 (host=impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22000) Last report received time: 2019-10-26 04:26:20.515 Hdfs split stats (:<# splits>/): 2:1/2.98 KB 1:1/2.98 KB - AverageThreadTokens: 1.17 (1.1666666666666667) - BloomFilterBytes: 0 B (0) - ExchangeScanRatio: 0.01 (0.005802707930367505) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 307.8 KiB (315168) - PeakReservation: 16.0 KiB (16384) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 73.5 MiB (77039702) - RowsProduced: 4 (4) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 85ms (85996893) - TotalStorageWaitTime: 23ms (23999133) - TotalThreadsInvoluntaryContextSwitches: 191 (191) - TotalThreadsTotalWallClockTime: 2.83s (2826897897) - TotalThreadsSysTime: 24ms (24832000) - TotalThreadsUserTime: 1.38s (1379299000) - TotalThreadsVoluntaryContextSwitches: 15 (15) - TotalTime: 2.78s (2780899559) Fragment Instance Lifecycle Event Timeline Prepare Finished: 136ms (136995052) Open Finished: 2.73s (2729901401) First Batch Produced: 2.80s (2798898909) First Batch Sent: 2.80s (2798898909) ExecInternal Finished: 2.91s (2908894936) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 0 B (0) - CumulativeAllocations: 0 (0) - InactiveTotalTime: 0ns (0) - PeakReservation: 0 B (0) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 0 B (0) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 0 B (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 174ms (174993679) - ExecTreeExecTime: 85ms (85996894) - InactiveTotalTime: 0ns (0) - OpenTime: 2.59s (2592906349) - ExecTreeOpenTime: 3ms (3999856) - PrepareTime: 11ms (11999567) - ExecTreePrepareTime: 3ms (3999855) - TotalTime: 0ns (0) KrpcDataStreamSender (dst_id=10) ExecOption: Hash Partitioned Sender Codegen Enabled - EosSent: 3 (3) - InactiveTotalTime: 85ms (85996893) - PeakMemoryUsage: 84.8 KiB (86816) - RowsSent: 4 (4) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0ns (0) - TotalBytesSent: 36 B (36) - TotalTime: 88ms (88996785) - UncompressedRowBatchSize: 52 B (52) HDFS_SCAN_NODE (id=5) Hdfs split stats (:<# splits>/): 2:1/2.98 KB 1:1/2.98 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 2 out of 2 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:2 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 6.1 KiB (6204) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 6.1 KiB (6204) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 6.1 KiB (6204) - CachedFileHandlesHitCount: 4 (4) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 B (0) - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 B (0) - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaterializeTupleTime(*): 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 2 (2) - NumRowGroups: 2 (2) - NumRowGroupsWithPageIndex: 2 (2) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 41.0 KiB (41984) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 10.0 KiB (10240) - RowBatchQueueGetWaitTime: 45ms (45998339) - RowBatchQueuePeakMemoryUsage: 29.0 KiB (29696) - RowBatchQueuePutWaitTime: 0ns (0) - RowBatchesEnqueued: 6 (6) - RowsRead: 4 (4) - RowsReturned: 4 (4) - RowsReturnedRate: 40 per second (40) - ScanRangesComplete: 2 (2) - ScannerIoWaitTime: 23ms (23999133) - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 58ms (58997869) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 17ms (17814000) - ScannerThreadsVoluntaryContextSwitches: 5 (5) - TotalRawHdfsOpenFileTime(*): 999.96us (999964) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 97ms (97996461) Node Lifecycle Event Timeline Open Started: 2.73s (2725901545) Open Finished: 2.73s (2729901401) First Batch Requested: 2.73s (2730901365) First Batch Returned: 2.80s (2798898909) Last Batch Returned: 2.82s (2817898223) Closed: 2.91s (2906895008) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 32.0 KiB (32768) - CumulativeAllocations: 4 (4) - InactiveTotalTime: 0ns (0) - PeakReservation: 16.0 KiB (16384) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.0 KiB (8192) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 185 (185) - CodegenTotalWallClockTime: 2.59s (2586906565) - CodegenSysTime: 0ns (0) - CodegenUserTime: 1.35s (1348341000) - CodegenVoluntaryContextSwitches: 2 (2) - CompileTime: 48ms (48998231) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 65ms (65997616) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 20 (20) - NumInstructions: 446 (446) - OptimizationTime: 1.55s (1554943838) - PeakMemoryUsage: 223.0 KiB (228352) - PrepareTime: 895ms (895967639) - TotalTime: 2.59s (2586906565) Instance 394ed7e2a194714b:31ab5b2f00000006 (host=impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22002) Last report received time: 2019-10-26 04:26:20.292 Hdfs split stats (:<# splits>/): 2:1/2.98 KB - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - ExchangeScanRatio: 0.01 (0.005802707930367505) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 307.8 KiB (315168) - PeakReservation: 8.0 KiB (8192) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 38.8 MiB (40659160) - RowsProduced: 2 (2) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 54ms (54998014) - TotalStorageWaitTime: 999.96us (999964) - TotalThreadsInvoluntaryContextSwitches: 191 (191) - TotalThreadsTotalWallClockTime: 1.79s (1792935243) - TotalThreadsSysTime: 5ms (5921000) - TotalThreadsUserTime: 878ms (878339000) - TotalThreadsVoluntaryContextSwitches: 9 (9) - TotalTime: 1.79s (1787935423) Fragment Instance Lifecycle Event Timeline Prepare Finished: 58ms (58997869) Open Finished: 1.73s (1730937481) First Batch Produced: 1.78s (1780935675) First Batch Sent: 1.78s (1780935675) ExecInternal Finished: 1.84s (1838933581) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 0 B (0) - CumulativeAllocations: 0 (0) - InactiveTotalTime: 0ns (0) - PeakReservation: 0 B (0) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 0 B (0) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 0 B (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 105ms (105996172) - ExecTreeExecTime: 50ms (50998158) - InactiveTotalTime: 0ns (0) - OpenTime: 1.67s (1671939612) - ExecTreeOpenTime: 2ms (2999891) - PrepareTime: 8ms (8999675) - ExecTreePrepareTime: 3ms (3999855) - TotalTime: 0ns (0) KrpcDataStreamSender (dst_id=10) ExecOption: Hash Partitioned Sender Codegen Enabled - EosSent: 3 (3) - InactiveTotalTime: 54ms (54998014) - PeakMemoryUsage: 84.8 KiB (86816) - RowsSent: 2 (2) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0ns (0) - TotalBytesSent: 18 B (18) - TotalTime: 56ms (56997942) - UncompressedRowBatchSize: 26 B (26) HDFS_SCAN_NODE (id=5) Hdfs split stats (:<# splits>/): 2:1/2.98 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:1 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 3.0 KiB (3102) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 3.0 KiB (3102) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 3.0 KiB (3102) - CachedFileHandlesHitCount: 2 (2) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 B (0) - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 B (0) - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaterializeTupleTime(*): 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 1 (1) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 41.0 KiB (41984) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 5.0 KiB (5120) - RowBatchQueueGetWaitTime: 15ms (15999422) - RowBatchQueuePeakMemoryUsage: 29.0 KiB (29696) - RowBatchQueuePutWaitTime: 0ns (0) - RowBatchesEnqueued: 3 (3) - RowsRead: 2 (2) - RowsReturned: 2 (2) - RowsReturnedRate: 31 per second (31) - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 999.96us (999964) - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 14ms (14999459) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 6ms (6454000) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 0ns (0) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 0 B/s (0) - TotalTime: 62ms (62997722) Node Lifecycle Event Timeline Open Started: 1.73s (1727937590) Open Finished: 1.73s (1730937481) First Batch Requested: 1.73s (1730937481) First Batch Returned: 1.78s (1780935675) Last Batch Returned: 1.78s (1781935639) Closed: 1.84s (1836933653) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 16.0 KiB (16384) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.0 KiB (8192) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.0 KiB (8192) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 178 (178) - CodegenTotalWallClockTime: 1.67s (1666939793) - CodegenSysTime: 982.00us (982000) - CodegenUserTime: 861ms (861585000) - CodegenVoluntaryContextSwitches: 0 (0) - CompileTime: 67ms (67997544) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 17ms (17999350) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 20 (20) - NumInstructions: 446 (446) - OptimizationTime: 1.12s (1118959586) - PeakMemoryUsage: 223.0 KiB (228352) - PrepareTime: 455ms (455983530) - TotalTime: 1.67s (1667939757) Instance 394ed7e2a194714b:31ab5b2f00000007 (host=impala-ec2-centos74-r4-4xlarge-ondemand-0fda.vpc.cloudera.com:22001) Last report received time: 2019-10-26 04:26:20.342 Hdfs split stats (:<# splits>/): 0:1/2.98 KB - AverageThreadTokens: 1.25 (1.25) - BloomFilterBytes: 0 B (0) - ExchangeScanRatio: 0.01 (0.005802707930367505) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 307.8 KiB (315168) - PeakReservation: 8.0 KiB (8192) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 38.8 MiB (40659160) - RowsProduced: 2 (2) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 52ms (52998086) - TotalStorageWaitTime: 8ms (8999675) - TotalThreadsInvoluntaryContextSwitches: 143 (143) - TotalThreadsTotalWallClockTime: 1.80s (1801934917) - TotalThreadsSysTime: 8ms (8927000) - TotalThreadsUserTime: 803ms (803322000) - TotalThreadsVoluntaryContextSwitches: 10 (10) - TotalTime: 1.78s (1784935531) Fragment Instance Lifecycle Event Timeline Prepare Finished: 76ms (76997219) Open Finished: 1.72s (1715938023) First Batch Produced: 1.80s (1799934990) First Batch Sent: 1.80s (1799934990) ExecInternal Finished: 1.86s (1856932931) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 0 B (0) - CumulativeAllocations: 0 (0) - InactiveTotalTime: 0ns (0) - PeakReservation: 0 B (0) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 0 B (0) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - ReservationLimit: 0 B (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) Fragment Instance Lifecycle Timings - ExecTime: 138ms (138994980) - ExecTreeExecTime: 84ms (84996930) - InactiveTotalTime: 0ns (0) - OpenTime: 1.64s (1638940804) - ExecTreeOpenTime: 2ms (2999891) - PrepareTime: 5ms (5999783) - ExecTreePrepareTime: 1ms (1999928) - TotalTime: 0ns (0) KrpcDataStreamSender (dst_id=10) ExecOption: Hash Partitioned Sender Codegen Enabled - EosSent: 3 (3) - InactiveTotalTime: 52ms (52998086) - PeakMemoryUsage: 84.8 KiB (86816) - RowsSent: 2 (2) - RpcFailure: 0 (0) - RpcRetry: 0 (0) - SerializeBatchTime: 0ns (0) - TotalBytesSent: 18 B (18) - TotalTime: 55ms (55997978) - UncompressedRowBatchSize: 26 B (26) HDFS_SCAN_NODE (id=5) Hdfs split stats (:<# splits>/): 0:1/2.98 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% File Formats: PARQUET/SNAPPY:1 - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 1.00 (1.0) - BytesRead: 3.0 KiB (3102) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 3.0 KiB (3102) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 3.0 KiB (3102) - CachedFileHandlesHitCount: 2 (2) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DataCacheHitBytes: 0 B (0) - DataCacheHitCount: 0 (0) - DataCacheMissBytes: 0 B (0) - DataCacheMissCount: 0 (0) - DataCachePartialHitCount: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaterializeTupleTime(*): 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumColumns: 1 (1) - NumDictFilteredRowGroups: 0 (0) - NumDisksAccessed: 1 (1) - NumPages: 1 (1) - NumRowGroups: 1 (1) - NumRowGroupsWithPageIndex: 1 (1) - NumScannerThreadMemUnavailable: 0 (0) - NumScannerThreadReservationsDenied: 0 (0) - NumScannerThreadsStarted: 1 (1) - NumScannersWithNoReads: 0 (0) - NumStatsFilteredPages: 0 (0) - NumStatsFilteredRowGroups: 0 (0) - PeakMemoryUsage: 41.0 KiB (41984) - PeakScannerThreadConcurrency: 1 (1) - PerReadThreadRawHdfsThroughput: 0 B/s (0) - RemoteScanRanges: 0 (0) - RowBatchBytesEnqueued: 5.0 KiB (5120) - RowBatchQueueGetWaitTime: 19ms (19999278) - RowBatchQueuePeakMemoryUsage: 29.0 KiB (29696) - RowBatchQueuePutWaitTime: 0ns (0) - RowBatchesEnqueued: 3 (3) - RowsRead: 2 (2) - RowsReturned: 2 (2) - RowsReturnedRate: 21 per second (21) - ScanRangesComplete: 1 (1) - ScannerIoWaitTime: 8ms (8999675) - ScannerThreadWorklessLoops: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 23ms (23999133) - ScannerThreadsSysTime: 0ns (0) - ScannerThreadsUserTime: 5ms (5804000) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 0ns (0) - TotalRawHdfsReadTime(*): 0ns (0) - TotalReadThroughput: 1.5 KiB/s (1541) - TotalTime: 90ms (90996713) Node Lifecycle Event Timeline Open Started: 1.71s (1712938132) Open Finished: 1.72s (1715938023) First Batch Requested: 1.72s (1715938023) First Batch Returned: 1.80s (1799934990) Last Batch Returned: 1.80s (1800934953) Closed: 1.85s (1855932967) Buffer pool - AllocTime: 0ns (0) - CumulativeAllocationBytes: 16.0 KiB (16384) - CumulativeAllocations: 2 (2) - InactiveTotalTime: 0ns (0) - PeakReservation: 8.0 KiB (8192) - PeakUnpinnedBytes: 0 B (0) - PeakUsedReservation: 8.0 KiB (8192) - ReadIoBytes: 0 B (0) - ReadIoOps: 0 (0) - ReadIoWaitTime: 0ns (0) - SystemAllocTime: 0ns (0) - TotalTime: 0ns (0) - WriteIoBytes: 0 B (0) - WriteIoOps: 0 (0) - WriteIoWaitTime: 0ns (0) CodeGen - CodegenInvoluntaryContextSwitches: 134 (134) - CodegenTotalWallClockTime: 1.62s (1625941274) - CodegenSysTime: 1ms (1981000) - CodegenUserTime: 790ms (790450000) - CodegenVoluntaryContextSwitches: 0 (0) - CompileTime: 37ms (37998628) - InactiveTotalTime: 0ns (0) - IrGenerationTime: 26ms (26999024) - LoadTime: 0ns (0) - ModuleBitcodeSize: 2.4 MiB (2560172) - NumFunctions: 20 (20) - NumInstructions: 446 (446) - OptimizationTime: 1.06s (1057961788) - PeakMemoryUsage: 223.0 KiB (228352) - PrepareTime: 496ms (496982050) - TotalTime: 1.62s (1625941274)