[FLINK-26570] Remote module configuration interpolation

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Stateful Functions
• Labels: pull-request-available stale-assigned
• Assignee: Fil Karnicki
• Reporter: Fil Karnicki
• Created: 10/Mar/22 06:45
• Updated: 20/Mar/24 19:37

Description

Add the ability for users to provide placeholders in module.yaml, e.g.

kind: com.foo.bar/test
spec:
  something: ${REPLACE_ME}
  transport:
    password: ${REPLACE_ME_WITH_A_SECRET}
    array:
      - ${REPLACE_ME}
      - sthElse 

These placeholders would be resolved in 

org.apache.flink.statefun.flink.core.jsonmodule.RemoteModule#bindComponent

using 

ParameterTool.fromSystemProperties().mergeWith(ParameterTool.fromMap(globalConfiguration) 

by traversing the ComponentJsonObject.specJsonNode() and replacing values that contain placeholders with values from the combined system+globalConfig map

Issue Links

• links to: GitHub Pull Request #308
• links to: GitHub Pull Request #309