Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
1.13.0
-
None
Description
Data set:
The data is generated used the attached file: DRILL_6118_data_source.csv
Data gen commands:
create table dfs.tmp.`DRILL_6118_parquet_partitioned_by_folders/d1` (c1, c2, c3, c4, c5) as select cast(columns[0] as int) c1, columns[1] c2, columns[2] c3, columns[3] c4, columns[4] c5 from dfs.tmp.`DRILL_6118_data_source.csv` where columns[0] in (1, 3); create table dfs.tmp.`DRILL_6118_parquet_partitioned_by_folders/d2` (c1, c2, c3, c4, c5) as select cast(columns[0] as int) c1, columns[1] c2, columns[2] c3, columns[3] c4, columns[4] c5 from dfs.tmp.`DRILL_6118_data_source.csv` where columns[0]=2; create table dfs.tmp.`DRILL_6118_parquet_partitioned_by_folders/d3` (c1, c2, c3, c4, c5) as select cast(columns[0] as int) c1, columns[1] c2, columns[2] c3, columns[3] c4, columns[4] c5 from dfs.tmp.`DRILL_6118_data_source.csv` where columns[0]>3;
Steps:
- Execute the following query:
explain plan for select * from (select * from (select * from dfs.tmp.`DRILL_6118_parquet_partitioned_by_folders`)) where c1<3
Expected result:
numFiles=2, numRowGroups=2, only files from the folders d1 and d2 should be scanned.
Actual result:
Filter push down doesn't work:
numFiles=3, numRowGroups=3, scanning from all files
Attachments
Issue Links
- relates to
-
DRILL-6118 Handle item star columns during project / filter push down and directory pruning
-
- Closed
-
- links to
