[IMPALA-4833] Use scheduling information to make per-node memory reservation tight

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.9.0
• Fix Version/s: Impala 2.10.0
• Component/s: Backend
• Labels: resource-management
• Target Version: Product Backlog
• Assignee: Matthew Jacobs
• Reporter: Tim Armstrong
• Created: 27/Jan/17 17:33
• Updated: 14/Aug/17 16:14
• Resolved: 14/Aug/17 16:14

Description

Following on from IMPALA-3748, we should consider combining the resource estimates with the schedule to accurate compute the minimum buffer requirement per daemon. Different daemons may have different resource requirements because not every fragment runs on every daemon (e.g. unpartitioned fragments, fragments with fewer scan ranges than daemons).

Issue Links

• is blocked by: IMPALA-3748 Compute memory reservation in planner and claim atomically in Prepare()