[DRILL-1386] Enable Partition Pruning for filesystem queries

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.5.0
• Fix Version/s: 0.6.0
• Component/s: Query Planning & Optimization, Storage - JSON, Storage - Parquet, Storage - Text & CSV
• Assignee: DrillCommitter
• Reporter: Jacques Nadeau
• Created: 08/Sep/14 17:58
• Updated: 25/Sep/14 21:45
• Resolved: 25/Sep/14 21:45

Description

If you have files:

/files/2012/Jan/log.csv
/files/2013/Jan/log.csv

And write the query "select * from `/files/` where dir0 = 2012 and dir1='Jan' "

Drill will read both files and then filter out all of the second file. Drill should recognize that dir0=2012 and dir1='Jan' can be pushed down into the storage layer to prune the files read as part of the query.