Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-21024

Dynamic properties get exposed to job's main method if user parameters are passed

    XMLWordPrintableJSON

    Details

      Description

      A bug was identified in the user ML by Alexey exposing dynamic properties into the job user code.

      I was able to reproduce this issue by slightly adapting the WordCount example (org.apache.flink.streaming.examples.wordcount.WordCount2 in attached WordCount.jar ).

      Initiating a standalone job without using the --input parameter will result in printing an empty array:

      ./bin/standalone-job.sh start --job-classname org.apache.flink.streaming.examples.wordcount.WordCount2
      

      The corresponding *.out file looks like this:

      []
      Executing WordCount2 example with default input data set.
      Use --input to specify file input.
      Printing result to stdout. Use --output to specify output path.
      

      In contrast, initiating the standalone job using the --input parameter will expose the dynamic properties:

      ./bin/standalone-job.sh start --job-classname org.apache.flink.streaming.examples.wordcount.WordCount2 --input /opt/flink/config/flink-conf.yaml
      

      Resulting in the following output:

      [--input, /opt/flink/config/flink-conf.yaml, -D, jobmanager.memory.off-heap.size=134217728b, -D, jobmanager.memory.jvm-overhead.min=201326592b, -D, jobmanager.memory.jvm-metaspace.size=268435456b, -D, jobmanager.memory.heap.size=1073741824b, -D, jobmanager.memory.jvm-overhead.max=201326592b]
      Printing result to stdout. Use --output to specify output path.
      

      Interestingly, this cannot be reproduced on a local standalone session cluster.

        Attachments

        1. WordCount.jar
          15 kB
          Matthias

          Issue Links

            Activity

              People

              • Assignee:
                xintongsong Xintong Song
                Reporter:
                mapohl Matthias
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: