[DRILL-6340] Output Batch Control in Project using the RecordBatchSizer

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.14.0
• Component/s: Execution - Relational Operators
• Labels: ready-to-commit
• Assignee: Karthikeyan Manivannan
• Reporter: Karthikeyan Manivannan
• Reviewer: Boaz Ben-Zvi
• Created: 17/Apr/18 23:02
• Updated: 29/Jun/18 21:08
• Resolved: 22/Jun/18 21:20

Description

This bug is for tracking the changes required to implement Output Batch Sizing in Project using the RecordBatchSizer. The challenge in doing this mainly lies in dealing with expressions that produce variable-length columns. The following doc talks about some of the design approaches for dealing with such variable-length columns.

https://docs.google.com/document/d/1h0WsQsen6xqqAyyYSrtiAniQpVZGmQNQqC1I2DJaxAA/edit?usp=sharing

Issue Links

• causes: DRILL-6529 Project Batch Sizing causes two LargeFileCompilation tests to timeout
• links to: GitHub Pull Request #1302