Hadoop Common
  1. Hadoop Common
  2. HADOOP-7001

Allow configuration changes without restarting configured nodes

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0
    • Component/s: conf
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently, changing the configuration on a node (e.g., the name node) requires that we restart the node. We propose a change that would allow us to make configuration changes without restarting. Nodes that support configuration changes at run time should implement the following interface:

      interface ChangeableConfigured extends Configured

      { void changeConfiguration(Configuration newConf) throws ConfigurationChangeException; }

      The contract of changeConfiguration is as follows:
      The node will compare newConf to the existing configuration. For each configuration property that is set to a different value than in the current configuration, the node will either adjust its behaviour to conform to the new configuration or throw a ConfigurationChangeException if this change is not possible at run time. If a configuration property is set in the current configuration but is unset in newConf, the node should use its default value for this property. After a successful invocation of changeConfiguration, the behaviour of the configured node should be indistinguishable from the behaviour of a node that was configured with newConf at creation.

      It should be easy to change existing nodes to implement this interface. We can start by throwing the exception for all changes and then gradually start supporting more and more changes at run time. (We might even consider replacing Configured with ChangeableConfigured entirely, but I think the proposal above afford greater flexibility).

      1. HADOOP-7001.5.patch
        31 kB
        Patrick Kling
      2. HADOOP-7001.4.patch
        31 kB
        Patrick Kling
      3. HADOOP-7001.3.patch
        37 kB
        Patrick Kling
      4. HADOOP-7001.2.patch
        32 kB
        Patrick Kling
      5. HADOOP-7001.patch
        32 kB
        Patrick Kling
      6. reconfigurable.patch
        9 kB
        Patrick Kling

        Issue Links

          Activity

          Patrick Kling created issue -
          Allen Wittenauer made changes -
          Field Original Value New Value
          Link This issue is related to HADOOP-5670 [ HADOOP-5670 ]
          dhruba borthakur made changes -
          Assignee Patrick Kling [ pkling ]
          Patrick Kling made changes -
          Attachment reconfigurable.patch [ 12457308 ]
          Patrick Kling made changes -
          Attachment reconfigurable.patch [ 12457308 ]
          Patrick Kling made changes -
          Attachment reconfigurable.patch [ 12457309 ]
          Patrick Kling made changes -
          Attachment HADOOP-7001.patch [ 12458002 ]
          Patrick Kling made changes -
          Link This issue blocks HDFS-1477 [ HDFS-1477 ]
          Patrick Kling made changes -
          Attachment HADOOP-7001.2.patch [ 12458031 ]
          Patrick Kling made changes -
          Attachment HADOOP-7001.3.patch [ 12458601 ]
          Patrick Kling made changes -
          Attachment HADOOP-7001.4.patch [ 12458671 ]
          Patrick Kling made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Patrick Kling made changes -
          Attachment HADOOP-7001.5.patch [ 12459978 ]
          dhruba borthakur made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.23.0 [ 12315569 ]
          Resolution Fixed [ 1 ]
          dhruba borthakur made changes -
          Component/s conf [ 12310711 ]
          Konstantin Boudnik made changes -
          Affects Version/s 0.23.0 [ 12315569 ]
          Patrick Kling made changes -
          Link This issue is related to HADOOP-7049 [ HADOOP-7049 ]
          Wang Xu made changes -
          Link This issue blocks HDFS-1362 [ HDFS-1362 ]
          Wang Xu made changes -
          Link This issue blocks HDFS-1362 [ HDFS-1362 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Jeff Hammerbacher made changes -
          Link This issue relates to HBASE-3909 [ HBASE-3909 ]
          su wenbo made changes -
          Affects Version/s 0.23.0 [ 12315569 ]
          su wenbo made changes -
          Description Currently, changing the configuration on a node (e.g., the name node) requires that we restart the node. We propose a change that would allow us to make configuration changes without restarting. Nodes that support configuration changes at run time should implement the following interface:

          interface ChangeableConfigured extends Configured {
             void changeConfiguration(Configuration newConf) throws ConfigurationChangeException;
          }

          The contract of changeConfiguration is as follows:
          The node will compare newConf to the existing configuration. For each configuration property that is set to a different value than in the current configuration, the node will either adjust its behaviour to conform to the new configuration or throw a ConfigurationChangeException if this change is not possible at run time. If a configuration property is set in the current configuration but is unset in newConf, the node should use its default value for this property. After a successful invocation of changeConfiguration, the behaviour of the configured node should be indistinguishable from the behaviour of a node that was configured with newConf at creation.

          It should be easy to change existing nodes to implement this interface. We can start by throwing the exception for all changes and then gradually start supporting more and more changes at run time. (We might even consider replacing Configured with ChangeableConfigured entirely, but I think the proposal above afford greater flexibility).


          Currently, changing the configuration on a node (e.g., the name node) requires that we restart the node. We propose a change that would allow us to make configuration changes without restarting. Nodes that support configuration changes at run time should implement the following interface:

          interface ChangeableConfigured extends Configured {
             void changeConfiguration(Configuration newConf) throws ConfigurationChangeException;
          }

          The contract of changeConfiguration is as follows:
          The node will compare newConf to the existing configuration. For each configuration property that is set to a different value than in the current configuration, the node will either adjust its behaviour to conform to the new configuration or throw a ConfigurationChangeException if this change is not possible at run time. If a configuration property is set in the current configuration but is unset in newConf, the node should use its default value for this property. After a successful invocation of changeConfiguration, the behaviour of the configured node should be indistinguishable from the behaviour of a node that was configured with newConf at creation.

          It should be easy to change existing nodes to implement this interface. We can start by throwing the exception for all changes and then gradually start supporting more and more changes at run time. (We might even consider replacing Configured with ChangeableConfigured entirely, but I think the proposal above afford greater flexibility).


          Gavin made changes -
          Link This issue blocks HDFS-1477 [ HDFS-1477 ]
          Gavin made changes -
          Link This issue is depended upon by HDFS-1477 [ HDFS-1477 ]
          Gavin made changes -
          Link This issue blocks HDFS-1362 [ HDFS-1362 ]
          Gavin made changes -
          Link This issue is depended upon by HDFS-1362 [ HDFS-1362 ]
          Allen Wittenauer made changes -
          Link This issue duplicates HADOOP-5772 [ HADOOP-5772 ]
          Allen Wittenauer made changes -
          Link This issue duplicates MAPREDUCE-442 [ MAPREDUCE-442 ]

            People

            • Assignee:
              Patrick Kling
              Reporter:
              Patrick Kling
            • Votes:
              0 Vote for this issue
              Watchers:
              25 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development