[IMPALA-2162] Scratch files can be clobbered with multiple impalads per node

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: Impala 2.2
• Target Version: Product Backlog
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 29/Jul/15 18:38
• Updated: 29/May/20 16:59
• Resolved: 09/Jan/16 18:03

Description

Bug occurs if multiple impalads are running on a node with a shared scratch directory, and one impalad starts while another has spilled data to disk in the shared scratch.

Currently impalad deletes and recreates each scratch directory (e.g. /tmp/impala-scratch) in order to remove old scratch files and ensure it has the correct write permissions for the directory.

It also is unlikely to cause observable problems if impalads are started at around the same time - the scratch directory will be deleted and recreated multiple times, but scratch files will not be clobbered unless one of the impalads had spilled to disk.

Issue Links

• is related to: IMPALA-9798 TestScratchDir.test_multiple_dirs fails to start impalad