[IMPALA-3671] Add query option to limit scratch space usage

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 2.7.0
• Fix Version/s: Impala 2.8.0
• Component/s: Backend
• Labels: documentation resource-management
• Target Version: Product Backlog
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 03/Jun/16 21:31
• Updated: 15/Dec/16 00:36
• Resolved: 24/Sep/16 04:53

Description

The immediate motivation for this is to enable better testing of graceful failures when spilling is disabled (e.g. IMPALA-3670). Currently we only have control over this via startup options, so we have to implement these as custom cluster tests, but there's no reason in principle we need to start a fresh cluster.

The idea would be to add a query option 'scratch_limit' or similar, that limits the amount of scratch directory space that can be used. This would be useful to prevent runaway queries or to prevent queries from spilling when that is not desired.