Details
-
Bug
-
Status: Resolved
-
Minor
-
Resolution: Cannot Reproduce
-
Impala 2.6.0
-
None
Description
Summary line reports one filter arriving while both filters were received.
HDFS_SCAN_NODE (id=6):(Total: 9s443ms, non-child: 9s443ms, % non-child: 100.00%)
ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 14
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 2:39/8.86 GB 5:42/9.29 GB 4:28/6.50 GB 1:29/6.45 GB 6:32/7.38 GB 3:27/6.07 GB 10:34/7.63 GB 7:26/5.75 GB 0:33/7.24 GB 9:37/8.06 GB 8:22/4.86 GB
Runtime filters: Only following filters arrived: 8, waited 8s991ms
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:335 PARQUET/SNAPPY:28
BytesRead(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 73.18 MB, 73.18 MB, 73.18 MB, 73.18 MB, 73.18 MB, 73.18 MB, 73.18 MB, 93.31 MB, 134.68 MB, 161.83 MB, 161.83 MB, 161.83 MB, 161.83 MB, 184.49 MB, 206.65 MB, 233.75 MB, 233.75 MB, 233.75 MB, 261.02 MB, 261.02 MB, 288.24 MB, 309.87 MB, 309.87 MB, 309.87 MB, 337.08 MB, 337.08 MB, 337.08 MB, 337.08 MB, 337.08 MB
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 3.26
- BytesRead: 337.08 MB (353454758)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 337.08 MB (353454758)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 337.08 MB (353454758)
- DecompressionTime: 784.603ms
- MaxCompressedTextFileLength: 0
- NumColumns: 2 (2)
- NumDisksAccessed: 6 (6)
- NumRowGroups: 14 (14)
- NumScannerThreadsStarted: 4 (4)
- PeakMemoryUsage: 134.29 MB (140810128)
- PerReadThreadRawHdfsThroughput: 2.00 GB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 90.33M (90325159)
- RowsReturned: 90.32M (90321194)
- RowsReturnedRate: 9.56 M/sec
- ScanRangesComplete: 349 (349)
- ScannerThreadsInvoluntaryContextSwitches: 2.85K (2851)
- ScannerThreadsTotalWallClockTime: 1m16s
- MaterializeTupleTime(*): 50s026ms
- ScannerThreadsSysTime: 887.864ms
- ScannerThreadsUserTime: 6s995ms
- ScannerThreadsVoluntaryContextSwitches: 176.78K (176783)
- TotalRawHdfsReadTime(*): 164.437ms
- TotalReadThroughput: 13.76 MB/sec
Filter 7 (1.00 MB):
- Rows processed: 229.36K (229362)
- Rows rejected: 3.96K (3965)
- Rows total: 229.38K (229376)
Filter 8 (1.00 MB):
- Files processed: 349 (349)
- Files rejected: 335 (335)
- Files total: 349 (349)
- RowGroups processed: 88.21K (88214)
- RowGroups rejected: 0 (0)
- RowGroups total: 88.21K (88214)
- Rows processed: 229.36K (229362)
- Rows rejected: 0 (0)
- Rows total: 229.38K (229376)
- Splits processed: 14 (14)
- Splits rejected: 0 (0)
- Splits total: 14 (14)
Ditto in final routing table
ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 6 3 0 20 LOCAL true 0 (20) N/A N/A 5 4 0 20 LOCAL false 0 (20) N/A N/A 8 9 6 20 LOCAL true 0 (20) N/A N/A 7 10 6 20 LOCAL false 0 (20) N/A N/A 10 15 12 20 LOCAL true 0 (20) N/A N/A 9 16 12 20 LOCAL false 0 (20) N/A N/A 12 21 18 20 LOCAL true 0 (20) N/A N/A 11 22 18 20 LOCAL false 0 (20) N/A N/A 14 27 24 20 LOCAL true 0 (20) N/A N/A 13 28 24 20 LOCAL false 0 (20) N/A N/A 16 33 30 20 LOCAL true 0 (20) N/A N/A 15 34 30 20 LOCAL false 0 (20) N/A N/A 4 36 2 17 REMOTE false 0 (20) 25s414ms 34s022ms 3 37 8, 2 17, 17 REMOTE, REMOTE false, false 2 (20) 33s973ms N/A 2 38 14, 8, 2 17, 17, 17 REMOTE, REMOTE, REMOTE false, false, false 0 (20) 16s435ms 16s461ms 1 39 20, 14, 8, 2 17, 17, 17, 17 REMOTE, REMOTE, REMOTE, REMOTE false, false, false, false 0 (20) 15s442ms 15s473ms 0 40 26, 20, 14, 8, 2 17, 17, 17, 17, 17 REMOTE, REMOTE, REMOTE, REMOTE, REMOTE false, false, false, false, false 0 (20) 9s203ms 9s232ms