Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7200

Local filesystem dataload fails due to missing FILESYSTEM_PREFIX

    XMLWordPrintableJSON

    Details

      Description

      A recent local filesystem run failed during dataload with the following output:

      14:27:43 Running custom post-load steps (logging to /data/jenkins/workspace/impala-asf-master-core-local/repos/Impala/logs/data_loading/custom-post-load-steps.log)... 
      14:27:49     FAILED (Took: 0 min 6 sec)
      14:27:49     'custom-post-load-steps' failed. Tail of log:
      14:27:49 Log for command 'custom-post-load-steps'
      14:27:49 put: `/test-warehouse/': No such file or directory: `file:///test-warehouse'

      Local filesystem tests rely on the FILESYSTEM_PREFIX being added to add HDFS calls. The fix for IMPALA-3307 included this line in testdata/bin/create-load-data.sh:

      # IMPALA-3307: Upload test time-zone database
      hadoop fs -Ddfs.block.size=1048576 -put -f ${IMPALA_HOME}/testdata/tzdb /test-warehouse/

      This needs to use ${FILESYSTEM_PREFIX}/test-warehouse rather than /test-warehouse. Local runs on /tmp. This is likely to impact s3 as well.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                joemcdonnell Joe McDonnell
                Reporter:
                joemcdonnell Joe McDonnell
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: