[DRILL-6664] Parquet reader should not allow batches with more than 64k rows

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Done
• Fix Version/s: 1.15.0
• Assignee: Salim Achouche
• Reporter: Salim Achouche
• Reviewer: Boaz Ben-Zvi
• Created: 03/Aug/18 20:13
• Updated: 04/Aug/18 06:33
• Resolved: 04/Aug/18 06:33

Description

The Drill configuration allows the Parquet reader to handle batches larger than 64. We should limit this setting to 64k as several operators assume a maximum batch size of 64k.

NOTE - This Jira is precautionary as the default is 32k rows maximum

Issue Links

• links to: GitHub Pull Request #1420