[FLINK-36073] ApplicationMode with the K8s operator does not support downloading jars via filesystem plugins

• Type: Bug
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.0.0, 1.19.2, 1.20.1
• Component/s: Runtime / Coordination
• Assignee: Ferenc Csaky
• Reporter: Robert Metzger
• Created: 15/Aug/24 15:35
• Updated: 12/Sep/24 15:45

Description

As discussed in this ticket https://issues.apache.org/jira/browse/FLINK-28915 we can not define a FlinkDeployment spec with

spec:
  job:
    jarURI: s3://myDevBucket/myjob.jar

It fails with:

org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.main(StandaloneApplicationClusterEntryPoint.java:89) [flink-dist-1.19-x.jar:1.19-x]
Caused by: java.net.MalformedURLException: unknown protocol: s3
	at java.net.URL.<init>(URL.java:652) ~[?:?]
	at java.net.URL.<init>(URL.java:541) ~[?:?]
	at java.net.URL.<init>(URL.java:488) ~[?:?]
	at org.apache.flink.configuration.ConfigUtils.decodeListFromConfig(ConfigUtils.java:133) ~[flink-dist-1.19-x.jar:1.19-x]
	at org.apache.flink.client.program.DefaultPackagedProgramRetriever.getClasspathsFromConfiguration(DefaultPackagedProgramRetriever.java:273) ~[flink-dist-1.19-x.jar:1.19-x]
	at org.apache.flink.client.program.DefaultPackagedProgramRetriever.create(DefaultPackagedProgramRetriever.java:121) ~[flink-dist-1.19-x.jar:1.19-x]
	... 4 more

even though I have defined:

  podTemplate:
    spec:
      containers:
        - name: flink-main-container
          env:
            - name: ENABLE_BUILT_IN_PLUGINS
              value: "flink-s3-fs-presto-1.19.0.jar"

Issue Links

• is related to: FLINK-28915 Make application mode could support remote DFS schema(e.g. S3, OSS, HDFS, etc.)