[FLINK-14907] Support to Configure Azure Storage Key via Environment Variable

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.12.0
• Component/s: FileSystems
• Labels: pull-request-available
• Assignee: Konstantin Knauf
• Reporter: Konstantin Knauf
• Created: 21/Nov/19 16:30
• Updated: 21/Apr/21 14:45
• Resolved: 21/Apr/21 14:45

Description

From a security perspective it would often be preferable pass the Azure storage key via an env to the Flink Azure filesystem. Currently we only support to set it in the Flink configuration [1].

The least intrusive way, I think, would be to include a very simple org.apache.hadoop.fs.azure.KeyProvider to flink-azure-fs-hadoop, which reads the account key from an env variable called AZURE_STORAGE_KEY.

It could then be enabled in the flink-conf.yaml via:

fs.azure.account.keyprovider.<account_name>.blob.core.windows.net: org.apache.flink.fs.azurefs.<NewProviderClass>

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/filesystems/azure.html

Issue Links

• links to: GitHub Pull Request #10376