Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16013

List and map config options could not be parsed correctly

    XMLWordPrintableJSON

    Details

      Description

      Currently, if a config option is List type and written to a flink-conf.yaml, it could not be parsed correctly when reloaded from yaml resource. The root cause is we use List#toString to save into the yaml resource. However, when we want to parse a List from a string, we use semicolon to split the value.

       

      Also the Map, Duration type have the same problem.

       

      The following is a unit test to reproduce this problem.

      public void testWriteConfigurationAndReload() throws IOException {
       final File flinkConfDir = temporaryFolder.newFolder().getAbsoluteFile();
       final Configuration flinkConfig = new Configuration();
       final ConfigOption<List<String>> listConfigOption = ConfigOptions
        .key("test-list-string-key")
        .stringType()
        .asList()
        .noDefaultValue();
       final List<String> values = Arrays.asList("value1", "value2", "value3");
       flinkConfig.set(listConfigOption, values);
       assertThat(values, Matchers.containsInAnyOrder(flinkConfig.get(listConfigOption).toArray()));
      
       BootstrapTools.writeConfiguration(flinkConfig, new File(flinkConfDir, "flink-conf.yaml"));
       final Configuration loadedFlinkConfig = GlobalConfiguration.loadConfiguration(flinkConfDir.getAbsolutePath());
       assertThat(values, Matchers.containsInAnyOrder(loadedFlinkConfig.get(listConfigOption).toArray()));
      }
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                fly_in_gis Yang Wang
                Reporter:
                fly_in_gis Yang Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m