Hadoop Common
  1. Hadoop Common
  2. HADOOP-5772

Implement a 'refreshable' configuration system with right access-controls etc.

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: conf
    • Labels:
      None

      Description

      We have various bits and pieces of code to refresh certain configuration files, various components to restrict access to who can actually refresh the configs etc.

      I propose we start thinking about a simple system to support this as a first-class citizen in Hadoop Core...

        Activity

        Hide
        steve_l added a comment -

        Incidentally, as I struggle to bring up a cluster where no machine knows their hostname/IPAddr until they have opened their IPC and http ports, it would be very good for the worker nodes TT and DN to be tweaked to reread the conf file every time they try looking up the address:port of the master nodes. That would allow the namenode and job tracker to restart on different hosts and have all the workers bond to them once their configurations are updated.

        Show
        steve_l added a comment - Incidentally, as I struggle to bring up a cluster where no machine knows their hostname/IPAddr until they have opened their IPC and http ports, it would be very good for the worker nodes TT and DN to be tweaked to reread the conf file every time they try looking up the address:port of the master nodes. That would allow the namenode and job tracker to restart on different hosts and have all the workers bond to them once their configurations are updated.
        Hide
        steve_l added a comment -

        I think reloading is separate from distributed sourcing of configuration . You can have a distributed source of configuration that isn't reloaded/read-on-demand; you can even reload stuff from the local (or NFS-mounted) FS

        Show
        steve_l added a comment - I think reloading is separate from distributed sourcing of configuration . You can have a distributed source of configuration that isn't reloaded/read-on-demand; you can even reload stuff from the local (or NFS-mounted) FS
        Hide
        Arun C Murthy added a comment -

        Sure, alternatively we could use this jira to track just the 'refreshable' part of the configuration alongwith access-control etc.

        Show
        Arun C Murthy added a comment - Sure, alternatively we could use this jira to track just the 'refreshable' part of the configuration alongwith access-control etc.
        Hide
        Jeff Hammerbacher added a comment -

        Hey Arun,

        https://issues.apache.org/jira/browse/HADOOP-5670 seems to address this issue as well. Should we consolidate discussion into one ticket?

        Later,
        Jeff

        Show
        Jeff Hammerbacher added a comment - Hey Arun, https://issues.apache.org/jira/browse/HADOOP-5670 seems to address this issue as well. Should we consolidate discussion into one ticket? Later, Jeff

          People

          • Assignee:
            Unassigned
            Reporter:
            Arun C Murthy
          • Votes:
            1 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:

              Development