Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.1.0
    • None
    • None
    • None

    Description

      We need Ozone reconfigure framework to support reload new config value dynamically

      Attachments

        Issue Links

          Activity

            maobaolong Baolong Mao added a comment -

            Hi elek, I used to work well with HDFS reconfig framework, it helps us to solve many problem without restart the process, something like change the interval of redundancyMonitor, resize some queue size, anyway, it helps us a lot.

            But, after some thought of Ozone, i think it hard to have a reconfig framework like HDFS by some following reasons.

            • The configuration keys of HDFS come from the specific config files in the classpath, like core-default.xml, hdfs-default.xml, core-site.xml, hdfs-site.xml, after i modify these files, I can use hdfs reconfig admin shell command to reload these files and able to get this diffs keys, and fire the event to the listeners. This can work because each process of HDFS, like namenode, datanode maintains one configuration.
            • The configuration keys of Ozone come from the specific config files in the classpath, like ozone-site.xml, but, we can specify other config file by `-conf` arguments, we can also specify the config key by `-Dkey=value`, further more, Ozone invent a java annotation based configuration, it is easy to used. But it is hard to get the diffs between the changed config files and the config key in mem.

            Do you have some idea to make reconfig come true to Ozone?

            maobaolong Baolong Mao added a comment - Hi elek , I used to work well with HDFS reconfig framework, it helps us to solve many problem without restart the process, something like change the interval of redundancyMonitor, resize some queue size, anyway, it helps us a lot. But, after some thought of Ozone, i think it hard to have a reconfig framework like HDFS by some following reasons. The configuration keys of HDFS come from the specific config files in the classpath, like core-default.xml, hdfs-default.xml, core-site.xml, hdfs-site.xml, after i modify these files, I can use hdfs reconfig admin shell command to reload these files and able to get this diffs keys, and fire the event to the listeners. This can work because each process of HDFS, like namenode, datanode maintains one configuration. The configuration keys of Ozone come from the specific config files in the classpath, like ozone-site.xml, but, we can specify other config file by `-conf` arguments, we can also specify the config key by `-Dkey=value`, further more, Ozone invent a java annotation based configuration, it is easy to used. But it is hard to get the diffs between the changed config files and the config key in mem. Do you have some idea to make reconfig come true to Ozone?
            elek Marton Elek added a comment -

            I don't think it's impossible. I see two independent question:

            1. How the interface should look like with annotation based configuration
            2. How the changes can be identified.

            About 1.

            I think an ideal interface is an async event It can be connected with the event queue and the new conf can be delivered in an async way.

            public class ReplicationManager {
              private ReplicationManagerConfig config;
            
              public ReplicationManager(ReplicationManagerConfig config) {
                 this.config = config
                //init 
              }
            
            
               public void onConfigChanged(ReplicationManagerConfig newConfig){
                  if (newConfig.getThreadNo() != ccurrentConfig.getThreadNo()) {
                         // restart thread pool
                  }
              }
            }
            

            About 2:

            Under the hood we use plain old Hadoop configuration, I supposed the configuration reload supposed to be working. As far as I understood the problem is with the CLI override parameters (which exists in Hadoop, too. BTW).

            But at most of the places we use `ConfigurationSource` interface which would make it easy to read configuration from multiple sources. For example:

            ConfigSource hadoop = new LegacyHadoopConf();
            ConfigSource cli = ConfigsFromCliParams(); 
            
            ConfigSource ozoneConfiguration = new CompositeConfigurationSource(cli, hadoop);
            

            With this approach the composite configuration can read configuration from mutiple sources but hadoop configuration can be reloaded.

            (It requires a few adjustments on Hadoop RPC side as we have direct cast there, but it's possible to solve...)

            elek Marton Elek added a comment - I don't think it's impossible. I see two independent question: 1. How the interface should look like with annotation based configuration 2. How the changes can be identified. About 1. I think an ideal interface is an async event It can be connected with the event queue and the new conf can be delivered in an async way. public class ReplicationManager { private ReplicationManagerConfig config; public ReplicationManager(ReplicationManagerConfig config) { this .config = config //init } public void onConfigChanged(ReplicationManagerConfig newConfig){ if (newConfig.getThreadNo() != ccurrentConfig.getThreadNo()) { // restart thread pool } } } About 2: Under the hood we use plain old Hadoop configuration, I supposed the configuration reload supposed to be working. As far as I understood the problem is with the CLI override parameters (which exists in Hadoop, too. BTW). But at most of the places we use `ConfigurationSource` interface which would make it easy to read configuration from multiple sources. For example: ConfigSource hadoop = new LegacyHadoopConf(); ConfigSource cli = ConfigsFromCliParams(); ConfigSource ozoneConfiguration = new CompositeConfigurationSource(cli, hadoop); With this approach the composite configuration can read configuration from mutiple sources but hadoop configuration can be reloaded. (It requires a few adjustments on Hadoop RPC side as we have direct cast there, but it's possible to solve...)

            People

              maobaolong Baolong Mao
              maobaolong Baolong Mao
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: