[DRILL-4857] When no partition pruning occurs with metadata caching there's a performance regression

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.7.0
• Fix Version/s: 1.8.0
• Component/s: Metadata, Query Planning & Optimization
• Assignee: Aman Sinha
• Reporter: Aman Sinha
• Reviewer: dgu-atmapr
• Created: 19/Aug/16 19:34
• Updated: 13/Sep/16 18:20
• Resolved: 24/Aug/16 16:36

Description

After DRILL-4530, we see the (expected) performance improvements in planning time with metadata cache for cases where partition pruning got applied. However, in cases where it did not get applied and for sufficiently large number of files (tested with up to 400K files), there's performance regression. Part of this was addressed by DRILL-4846. This JIRA is to track some remaining fixes to address the regression.

Issue Links

• links to: GitHub Pull Request #575