Renaming of configuration keys

• Key: HDFS-531
• Type: Improvement
• Status: Open
• Priority: Major
• Assignee: Jitendra Nath Pandey
• Reporter: Jitendra Nath Pandey
• Fix Version/s: 0.21.0

Description

Keys in configuration files should be standardized so that key names reflect the components they are used in.
For example:
dfs.backup.address should be renamed to dfs.namenode.backup.address
dfs.data.dir should be renamed to dfs.datanode.data.dir

This change will impact both hdfs and common sources.
Following convention is proposed:
1. Any key related hdfs should begin with 'dfs'.
2. Any key related to namenode, datanode or client should begin with dfs.namenode, dfs.datanode or dfs.client respectively.

Comments

[Koji Noguchi]

Are we going to keep backward compatibility with 'deprecated' warning for at least one version?

[Jitendra Nath Pandey]

Yes, backward compatibility will be maintained.
We are planning to use backward compatibility mechanism proposed in following jira -
https://issues.apache.org/jira/browse/HADOOP-6105

[Jitendra Nath Pandey]

For backward compatibility, this jira depends upon HADOOP-6105

[Jitendra Nath Pandey]

The attached file 'changed_config_keys.txt' lists of keys which are proposed to be changed along with the proposed new names.

[Tom White]

• It seems wrong that there are HDFS properties in Common. Rather than dfs.networktopology.* how about net.topology.* to reflect the fact they are in the o.a.h.net package? Also, why not keep hadoop.tmp.dir instead of renaming it dfs.tmp.dir?
• The distributed cache is moving to MapReduce so filecache.local.cache.size, should have a mapred prefix. See also MAPREDUCE-711.
• My preference would be that the HDFS prefix is hdfs, not dfs, to be consistent with the protocol scheme (hdfs://) and package names (o.a.h.hdfs). But this is a big change.

[Owen O'Malley]

I agree with all of Tom's comments.

1. all topology stuff belongs in net.topology.
2. leave hadoop.tmp.dir alone
3. distributed cache should be mapred
4. I think we should pass on the dfs to hdfs. It would be nice, but I don't think it is worth it.

I think dfs.max.replication.streams and dfs.heartbeat.recheck.interval should have a more specific context? dfs.namenode.*?

[dhruba borthakur]

> dfs.max.replication.streams and dfs.heartbeat.recheck.interval should have a more specific context? dfs.namenode.*?

I like this idea. Does it mean that we have dfs.namenode.* dfs.client.* and dfs.datanode.* types of property names?

[Jitendra Nath Pandey]

Updated list of changed keys is attached incorporating the suggestions.

[Jitendra Nath Pandey]

unchanged_keys.txt file contains the keys that are assumed to be rightly named and will not be changed.