[IMPALA-2669] S3: Allow Frontend.loadTableData() to work with "other" filesystems

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: Impala 2.3.0
• Component/s: Frontend
• Labels: s3
• Target Version: Impala 2.5.0
• Assignee: Sailesh Mukil
• Reporter: Sailesh Mukil
• Created: 16/Nov/15 19:18
• Updated: 02/Feb/16 07:52
• Resolved: 02/Feb/16 07:52

Description

Frontend.loadTableData() expects that the underlying filesystem we use is HDFS. Change this to make it filesystem agnostic.

The bigger problem here is the staging of temporary files before renaming them to their final destination. Since S3 does not support renames, we need to choose one of the 3 options as mentioned in the parent JIRA or work out a better solution.