[IMPALA-7200] Local filesystem dataload fails due to missing FILESYSTEM_PREFIX

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 3.1.0
• Fix Version/s: Impala 3.1.0
• Component/s: Infrastructure
• Labels: broken-build
• Target Version: Impala 3.1.0
• Epic Color: ghx-label-8
• Assignee: Joe McDonnell
• Reporter: Joe McDonnell
• Created: 22/Jun/18 22:00
• Updated: 25/Jun/18 16:51
• Resolved: 25/Jun/18 16:51

Description

A recent local filesystem run failed during dataload with the following output:

14:27:43 Running custom post-load steps (logging to /data/jenkins/workspace/impala-asf-master-core-local/repos/Impala/logs/data_loading/custom-post-load-steps.log)... 
14:27:49     FAILED (Took: 0 min 6 sec)
14:27:49     'custom-post-load-steps' failed. Tail of log:
14:27:49 Log for command 'custom-post-load-steps'
14:27:49 put: `/test-warehouse/': No such file or directory: `file:///test-warehouse'

Local filesystem tests rely on the FILESYSTEM_PREFIX being added to add HDFS calls. The fix for IMPALA-3307 included this line in testdata/bin/create-load-data.sh:

# IMPALA-3307: Upload test time-zone database
hadoop fs -Ddfs.block.size=1048576 -put -f ${IMPALA_HOME}/testdata/tzdb /test-warehouse/

This needs to use ${FILESYSTEM_PREFIX}/test-warehouse rather than /test-warehouse. Local runs on /tmp. This is likely to impact s3 as well.

Issue Links

• is broken by: IMPALA-3307 add support for IANA time zone database