[IMPALA-7138] Fix detection and handling of Device Mapper volumes

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 3.0, Impala 2.12.0
• Fix Version/s: Impala 4.0.0
• Component/s: Backend
• Labels: resource-management
• Target Version: Product Backlog
• Epic Color: ghx-label-4
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 06/Jun/18 23:41
• Updated: 02/Apr/20 17:50
• Resolved: 02/Apr/20 17:50

Description

I've heard a couple of instances of Impala doing the wrong thing when multiple scratch directories are present on a single DeviceMapper logical volume. E.g. in the following setup, we only use one directory as a disk.

>> mount list
/dev/mapper/xx01-lv_data01 on /data01 type ext4 (rw,relatime,data=ordered)
/dev/mapper/xx02-lv_data02 on /data02 type ext4 (rw,relatime,data=ordered)
/dev/mapper/xx03-lv_data03 on /data03 type ext4 (rw,relatime,data=ordered)
/dev/mapper/xx04-lv_data04 on /data04 type ext4 (rw,relatime,data=ordered)
/dev/mapper/xx05-lv_data05 on /data05 type ext4 (rw,relatime,data=ordered)
/dev/mapper/xx06-lv_data06 on /data06 type ext4 (rw,relatime,data=ordered)
/dev/mapper/xx07-lv_data07 on /data07 type ext4 (rw,relatime,data=ordered)
/dev/mapper/xx08-lv_data08 on /data08 type ext4 (rw,relatime,data=ordered)
/dev/mapper/xx09-lv_data09 on /data09 type ext4 (rw,relatime,data=ordered)
/dev/mapper/xx10-lv_data10 on /data10 type ext4 (rw,relatime,data=ordered)
/dev/mapper/xx11-lv_data11 on /data11 type ext4 (rw,relatime,data=ordered)
/dev/mapper/xx12-lv_data12 on /data12 type ext4 (rw,relatime,data=ordered)

Scratch dirs are:
/data01/impala/impalad,/data02/impala/impalad,/data03/impala/impalad,/data04/impala/impalad,/data05/impala/impalad,/data06/impala/impalad,/data07/impala/impalad,/data08/impala/impalad,/data09/impala/impalad,/data10/impala/impalad,/data11/impala/impalad,/data12/impala/impalad


~~~ tmp-file-mgr.cc:122] Using scratch directory /data01/impala/impalad/impala-scratch on disk13. 

A workaround is to set --allow_multiple_scratch_dirs_per_device=true

There are a few questions here:

1. Does the deduplication logic even make sense to have?
2. Do we need to fix how we treat Device mapper volumes?

Issue Links

• relates to: IMPALA-8188 Some SSDs are not properly detected as non-rotational

Sub-Tasks

• Consider changing --allow_multiple_scratch_dirs_per_device default: Resolved