[FLINK-18568] Add Support for Azure Data Lake Store Gen 2 in File Sink

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.12.0
• Component/s: API / DataStream, Connectors / Common
• Labels: auto-deprioritized-major stale-assigned
• Assignee: ramkrishna.s.vasudevan
• Reporter: Israel Ekpo
• Created: 10/Jul/20 11:38
• Updated: 18/Aug/23 22:35

Description

The objective of this improvement is to add support for Azure Data Lake Store Gen 2 (ADLS Gen2) [2] as one of the supported filesystems for the FileSink [1]

[1] https://nightlies.apache.org/flink/flink-docs-stable/docs/connectors/datastream/filesystem/#file-sink
[2] https://hadoop.apache.org/docs/current/hadoop-azure/abfs.html

Issue Links

• is related to: FLINK-17444 StreamingFileSink Azure HadoopRecoverableWriter class missing.
• requires: FLINK-18562 Add Support for Azure Data Lake Store Gen 2 in Flink File System

Sub-Tasks

• Correct the packaging of HadoopRecoverableWriter and related issues: Closed
• Introduce Azure Data Lake Gen2 APIs in the Hadoop Recoverable path: Closed
• Improvement in truncate code in the new ABFSOutputstream: Open