Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Duplicate
-
1.18.0
-
None
-
None
Description
When run job by K8s or Yarn Application Mode use org.apache.flink.configuration.GlobalConfiguration#loadYAMLResource method parse config。
private static Configuration loadYAMLResource(File file) { final Configuration config = new Configuration(); try (BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(file)))) { String line; int lineNo = 0; while ((line = reader.readLine()) != null) { lineNo++; // 1. check for comments String[] comments = line.split("#", 2); String conf = comments[0].trim(); // 2. get key and value if (conf.length() > 0) { String[] kv = conf.split(": ", 2); // skip line with no valid key-value pair if (kv.length == 1) { LOG.warn( "Error while trying to split key and value in configuration file " + file + ":" + lineNo + ": Line is not a key-value pair (missing space after ':'?)"); continue; } String key = kv[0].trim(); String value = kv[1].trim(); // sanity check if (key.length() == 0 || value.length() == 0) { LOG.warn( "Error after splitting key and value in configuration file " + file + ":" + lineNo + ": Key or value was empty"); continue; } config.setString(key, value); } } } catch (IOException e) { throw new RuntimeException("Error parsing YAML configuration.", e); } return config; }
if config value contains '#' like
$internal.application.program-args: '{test:test#jsonstring}'
the following code is not correct
line.split("#", 2)
To fix this,i think we should import snakeyaml to fix complex situation
Attachments
Issue Links
- Blocked
-
FLINK-33297 FLIP-366: Support standard YAML for FLINK configuration
- Closed