Hadoop Common
  1. Hadoop Common
  2. HADOOP-5772

Implement a 'refreshable' configuration system with right access-controls etc.

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: conf
    • Labels:
      None

      Description

      We have various bits and pieces of code to refresh certain configuration files, various components to restrict access to who can actually refresh the configs etc.

      I propose we start thinking about a simple system to support this as a first-class citizen in Hadoop Core...

        Issue Links

          Activity

          Arun C Murthy created issue -
          Hide
          Jeff Hammerbacher added a comment -

          Hey Arun,

          https://issues.apache.org/jira/browse/HADOOP-5670 seems to address this issue as well. Should we consolidate discussion into one ticket?

          Later,
          Jeff

          Show
          Jeff Hammerbacher added a comment - Hey Arun, https://issues.apache.org/jira/browse/HADOOP-5670 seems to address this issue as well. Should we consolidate discussion into one ticket? Later, Jeff
          Hide
          Arun C Murthy added a comment -

          Sure, alternatively we could use this jira to track just the 'refreshable' part of the configuration alongwith access-control etc.

          Show
          Arun C Murthy added a comment - Sure, alternatively we could use this jira to track just the 'refreshable' part of the configuration alongwith access-control etc.
          Hide
          Steve Loughran added a comment -

          I think reloading is separate from distributed sourcing of configuration . You can have a distributed source of configuration that isn't reloaded/read-on-demand; you can even reload stuff from the local (or NFS-mounted) FS

          Show
          Steve Loughran added a comment - I think reloading is separate from distributed sourcing of configuration . You can have a distributed source of configuration that isn't reloaded/read-on-demand; you can even reload stuff from the local (or NFS-mounted) FS
          Hide
          Steve Loughran added a comment -

          Incidentally, as I struggle to bring up a cluster where no machine knows their hostname/IPAddr until they have opened their IPC and http ports, it would be very good for the worker nodes TT and DN to be tweaked to reread the conf file every time they try looking up the address:port of the master nodes. That would allow the namenode and job tracker to restart on different hosts and have all the workers bond to them once their configurations are updated.

          Show
          Steve Loughran added a comment - Incidentally, as I struggle to bring up a cluster where no machine knows their hostname/IPAddr until they have opened their IPC and http ports, it would be very good for the worker nodes TT and DN to be tweaked to reread the conf file every time they try looking up the address:port of the master nodes. That would allow the namenode and job tracker to restart on different hosts and have all the workers bond to them once their configurations are updated.
          Allen Wittenauer made changes -
          Field Original Value New Value
          Link This issue is duplicated by HADOOP-7001 [ HADOOP-7001 ]
          Allen Wittenauer made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Duplicate [ 3 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          1904d 1h 31m 1 Allen Wittenauer 22/Jul/14 20:37

            People

            • Assignee:
              Unassigned
              Reporter:
              Arun C Murthy
            • Votes:
              1 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development