[IMPALA-2079] Don't fail when write to scratch dir results in error.

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 2.1, Impala 2.2
• Fix Version/s: Impala 2.9.0
• Component/s: Backend
• Labels: query-lifecycle ramp-up resource-management
• Target Version: Product Backlog
• Assignee: Tim Armstrong
• Reporter: Alan Choi
• Created: 18/Jun/15 18:08
• Updated: 19/Oct/17 02:55
• Resolved: 05/Jan/17 15:31

Description

Currently, a query fails when it tries to spill data to a disk and that write results in an error. This can happen for multiple reasons: a) no write permission, b) bad disk.

The fix should do two things:

• Check for write permissions on startup
• Don't fail queries that hit an error at runtime (e.g. disk full) if there is another functional scratch disk that can be used instead.

Issue Links

• is blocked by: IMPALA-3200 Replace BufferedBlockMgr with new buffer pool
• is duplicated by: IMPALA-2298 Skip full disks when writing tmp files
• is duplicated by: IMPALA-6078 IMPALA query fails if a scratch directory goes bad with a disk failure
• is related to: IMPALA-3866 Improve error reporting for scratch write errors
• relates to: IMPALA-4683 Intelligently blacklist scratch disks with physical I/O errors