Implement a 'refreshable' configuration system with right access-controls etc.

• Key: HADOOP-5772
• Type: New Feature
• Status: Open
• Priority: Major
• Assignee: Unassigned
• Reporter: Arun C Murthy
• Component/s: conf

Description

We have various bits and pieces of code to refresh certain configuration files, various components to restrict access to who can actually refresh the configs etc.

I propose we start thinking about a simple system to support this as a first-class citizen in Hadoop Core...

Comments

[Jeff Hammerbacher]

Hey Arun,

https://issues.apache.org/jira/browse/HADOOP-5670 seems to address this issue as well. Should we consolidate discussion into one ticket?

Later,
Jeff

[Arun C Murthy]

Sure, alternatively we could use this jira to track just the 'refreshable' part of the configuration alongwith access-control etc.

[Steve Loughran]

I think reloading is separate from distributed sourcing of configuration . You can have a distributed source of configuration that isn't reloaded/read-on-demand; you can even reload stuff from the local (or NFS-mounted) FS

[Steve Loughran]

Incidentally, as I struggle to bring up a cluster where no machine knows their hostname/IPAddr until they have opened their IPC and http ports, it would be very good for the worker nodes TT and DN to be tweaked to reread the conf file every time they try looking up the address:port of the master nodes. That would allow the namenode and job tracker to restart on different hosts and have all the workers bond to them once their configurations are updated.