[FLINK-21024] Dynamic properties get exposed to job's main method if user parameters are passed

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.12.1
• Fix Version/s: 1.12.2, 1.13.0
• Component/s: Runtime / Configuration
• Labels: pull-request-available starter
• Assignee: Xintong Song
• Reporter: Matthias Pohl
• Created: 19/Jan/21 08:42
• Updated: 28/May/21 07:12
• Resolved: 21/Jan/21 10:53

Description

A bug was identified in the user ML by Alexey exposing dynamic properties into the job user code.

I was able to reproduce this issue by slightly adapting the WordCount example (org.apache.flink.streaming.examples.wordcount.WordCount2 in attached WordCount.jar ).

Initiating a standalone job without using the --input parameter will result in printing an empty array:

./bin/standalone-job.sh start --job-classname org.apache.flink.streaming.examples.wordcount.WordCount2

The corresponding *.out file looks like this:

[]
Executing WordCount2 example with default input data set.
Use --input to specify file input.
Printing result to stdout. Use --output to specify output path.

In contrast, initiating the standalone job using the --input parameter will expose the dynamic properties:

./bin/standalone-job.sh start --job-classname org.apache.flink.streaming.examples.wordcount.WordCount2 --input /opt/flink/config/flink-conf.yaml

Resulting in the following output:

[--input, /opt/flink/config/flink-conf.yaml, -D, jobmanager.memory.off-heap.size=134217728b, -D, jobmanager.memory.jvm-overhead.min=201326592b, -D, jobmanager.memory.jvm-metaspace.size=268435456b, -D, jobmanager.memory.heap.size=1073741824b, -D, jobmanager.memory.jvm-overhead.max=201326592b]
Printing result to stdout. Use --output to specify output path.

Interestingly, this cannot be reproduced on a local standalone session cluster.

Issue Links

• links to: GitHub Pull Request #14697