Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-18171

Do not take client side config options to cluster

    XMLWordPrintableJSON

    Details

      Description

      Follow the discussion in this PR[1], some client side config options should not be taken into the cluster. Once it takes effect, it will cause some issue(e.g. FLINK-18149).

      For K8s deployment, we have explicitly remove KubernetesConfigOptions.KUBE_CONFIG_FILE and DeploymentOptionsInternal.CONF_DIR

      For Yarn deployment, we have at least two options DeploymentOptionsInternal.CONF_DIR and YarnConfigOptionsInternal.APPLICATION_LOG_CONFIG_FILE could be removed in client side.

       

      Benefit from this, we could avoid some unexpected configuration loading or secret issues. Also we could avoid the logs in the jobmanager confusing the users since it is a client local path.

      2020-06-05 14:38:38,656 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: $internal.deployment.config-dir, /home/danrtsey.wy/flink-1.11-SNAPSHOT/conf
      2020-06-05 14:38:38,656 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: $internal.yarn.log-config-file, /home/danrtsey.wy/flink-1.11-SNAPSHOT/conf/log4j.properties
      

       

      [1]https://github.com/apache/flink/pull/12501#pullrequestreview-425452351

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              fly_in_gis Yang Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: