[DRILL-5815] Provide option to set query memory as percent of total

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.11.0
• Fix Version/s: 1.12.0
• Labels: doc-impacting ready-to-commit
• Assignee: Paul Rogers
• Reporter: Paul Rogers
• Reviewer: Boaz Ben-Zvi
• Created: 25/Sep/17 02:20
• Updated: 16/Oct/17 19:50
• Resolved: 16/Oct/17 19:50

Description

Drill provides a parameter to set the memory per query as a static number which defaults to 2 GB. This number is a wonderful setting for the default Drillbit configuration of 8 GB heap; it allows 2-3 concurrent queries. But, as Drillbit memory increases, the default becomes a bit constraining. While users can change the setting, they seldom do.

In addition, provide an option that sets memory as a percent of total memory. If the allocation is 10%, say, and total memory is 128 GB, then each query gets ~13GB, which is a big improvement.

The existing option acts as a floor: the query must receive at least that much memory.

DOCUMENTATION
New option should be documented - planner.memory.percent_per_query
Default - 0.05 (which is equivalent to 5 %)
To disable feature set to 0.

More information can be found in pull request description.

Issue Links

• links to: GitHub Pull Request #960

Sub-Tasks

• Doc new parameter: Open