[DRILL-6199] Filter push down doesn't work with more than one nested subqueries

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.13.0
• Fix Version/s: 1.14.0
• Labels: ready-to-commit
• Assignee: Arina Ielchiieva
• Reporter: Anton Gozhiy
• Reviewer: Chunhui Shi
• Created: 01/Mar/18 09:58
• Updated: 22/Mar/18 14:32
• Resolved: 20/Mar/18 21:24

Description

Data set:
The data is generated used the attached file: DRILL_6118_data_source.csv
Data gen commands:

create table dfs.tmp.`DRILL_6118_parquet_partitioned_by_folders/d1` (c1, c2, c3, c4, c5) as select cast(columns[0] as int) c1, columns[1] c2, columns[2] c3, columns[3] c4, columns[4] c5 from dfs.tmp.`DRILL_6118_data_source.csv` where columns[0] in (1, 3);
create table dfs.tmp.`DRILL_6118_parquet_partitioned_by_folders/d2` (c1, c2, c3, c4, c5) as select cast(columns[0] as int) c1, columns[1] c2, columns[2] c3, columns[3] c4, columns[4] c5 from dfs.tmp.`DRILL_6118_data_source.csv` where columns[0]=2;
create table dfs.tmp.`DRILL_6118_parquet_partitioned_by_folders/d3` (c1, c2, c3, c4, c5) as select cast(columns[0] as int) c1, columns[1] c2, columns[2] c3, columns[3] c4, columns[4] c5 from dfs.tmp.`DRILL_6118_data_source.csv` where columns[0]>3;

Steps:

1. Execute the following query:

   explain plan for select * from (select * from (select * from dfs.tmp.`DRILL_6118_parquet_partitioned_by_folders`)) where c1<3
   

Expected result:
numFiles=2, numRowGroups=2, only files from the folders d1 and d2 should be scanned.

Actual result:
Filter push down doesn't work:
numFiles=3, numRowGroups=3, scanning from all files

Issue Links

• relates to: DRILL-6118 Handle item star columns during project / filter push down and directory pruning
• links to: GitHub Pull Request #1152