Improve how Hadoop gets configured
The current approach of a two-level XML configuration file works; it offers
- default values in (easily overridden) configuration files, rather than just Java source
- A way to override the default values
- conversion from string to float, double.
- with the ${property} evaluation, there is some ability to reference system values for some limited adaptation.
- errors show up at parse time (except for value parse problems)
- A serialization format to exchange configuration with other nodes
- the possibility of updating a local (in-memory) configuration
But it has limits
[1] Requires a change to the XML files to be pushed out to every node
[2] Differences between configurations can cause obscure bugs
[3] No support for complex hierarchical configurations
[4] No easy way to cross-reference data other than copy and paste.
[5] No way for a deployed instance to update configuration data for other instances to query
[6] Value type checking/dereferencing failure is not signalled by a custom error; there is no explicit exception on any of the get/set operations.
[7] No consistency with names.
[8] Not easily managed by different configuration architectures/tools
This issue is to group/track the problems, then discuss solutions.
Improvement
Open
Major
Improve how Hadoop gets configured
