[HUDI-1681] Support object storage for Flink writer

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Fix Version/s: 0.8.0
• Component/s: flink
• Labels: pull-request-available
• Assignee: Danny Chen
• Reporter: Danny Chen
• Created: 11/Mar/21 12:00
• Updated: 12/Mar/21 08:40
• Resolved: 12/Mar/21 08:40

Description

In order to support object storage, we need these changes:

• Use the Hadoop filesystem so that we can find the plugin filesystem
• Do not fetch file size until the file handle is closed
• Do not close the opened filesystem because we want to use the filesystem cache

Issue Links

• links to: GitHub Pull Request #2662