Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None

      Description

      Keys in configuration files should be standardized so that key names reflect the components they are used in.
      For example:
      dfs.backup.address should be renamed to dfs.namenode.backup.address
      dfs.data.dir should be renamed to dfs.datanode.data.dir

      This change will impact both hdfs and common sources.
      Following convention is proposed:
      1. Any key related hdfs should begin with 'dfs'.
      2. Any key related to namenode, datanode or client should begin with dfs.namenode, dfs.datanode or dfs.client respectively.

      1. unchanged_keys.txt
        2 kB
        Jitendra Nath Pandey
      2. changed_config_keys.2.txt
        4 kB
        Jitendra Nath Pandey
      3. changed_config_keys.txt
        3 kB
        Jitendra Nath Pandey

        Issue Links

          Activity

          Hide
          Koji Noguchi added a comment -

          Are we going to keep backward compatibility with 'deprecated' warning for at least one version?

          Show
          Koji Noguchi added a comment - Are we going to keep backward compatibility with 'deprecated' warning for at least one version?
          Hide
          Jitendra Nath Pandey added a comment -

          Yes, backward compatibility will be maintained.
          We are planning to use backward compatibility mechanism proposed in following jira -
          https://issues.apache.org/jira/browse/HADOOP-6105

          Show
          Jitendra Nath Pandey added a comment - Yes, backward compatibility will be maintained. We are planning to use backward compatibility mechanism proposed in following jira - https://issues.apache.org/jira/browse/HADOOP-6105
          Hide
          Jitendra Nath Pandey added a comment -

          For backward compatibility, this jira depends upon HADOOP-6105

          Show
          Jitendra Nath Pandey added a comment - For backward compatibility, this jira depends upon HADOOP-6105
          Hide
          Jitendra Nath Pandey added a comment -

          The attached file 'changed_config_keys.txt' lists of keys which are proposed to be changed along with the proposed new names.

          Show
          Jitendra Nath Pandey added a comment - The attached file 'changed_config_keys.txt' lists of keys which are proposed to be changed along with the proposed new names.
          Hide
          Tom White added a comment -
          • It seems wrong that there are HDFS properties in Common. Rather than dfs.networktopology.* how about net.topology.* to reflect the fact they are in the o.a.h.net package? Also, why not keep hadoop.tmp.dir instead of renaming it dfs.tmp.dir?
          • The distributed cache is moving to MapReduce so filecache.local.cache.size, should have a mapred prefix. See also MAPREDUCE-711.
          • My preference would be that the HDFS prefix is hdfs, not dfs, to be consistent with the protocol scheme (hdfs://) and package names (o.a.h.hdfs). But this is a big change.
          Show
          Tom White added a comment - It seems wrong that there are HDFS properties in Common. Rather than dfs.networktopology.* how about net.topology.* to reflect the fact they are in the o.a.h.net package? Also, why not keep hadoop.tmp.dir instead of renaming it dfs.tmp.dir? The distributed cache is moving to MapReduce so filecache.local.cache.size, should have a mapred prefix. See also MAPREDUCE-711 . My preference would be that the HDFS prefix is hdfs, not dfs, to be consistent with the protocol scheme (hdfs://) and package names (o.a.h.hdfs). But this is a big change.
          Hide
          Owen O'Malley added a comment -

          I agree with all of Tom's comments.

          1. all topology stuff belongs in net.topology.
          2. leave hadoop.tmp.dir alone
          3. distributed cache should be mapred
          4. I think we should pass on the dfs to hdfs. It would be nice, but I don't think it is worth it.

          I think dfs.max.replication.streams and dfs.heartbeat.recheck.interval should have a more specific context? dfs.namenode.*?

          Show
          Owen O'Malley added a comment - I agree with all of Tom's comments. 1. all topology stuff belongs in net.topology. 2. leave hadoop.tmp.dir alone 3. distributed cache should be mapred 4. I think we should pass on the dfs to hdfs. It would be nice, but I don't think it is worth it. I think dfs.max.replication.streams and dfs.heartbeat.recheck.interval should have a more specific context? dfs.namenode.*?
          Hide
          dhruba borthakur added a comment -

          > dfs.max.replication.streams and dfs.heartbeat.recheck.interval should have a more specific context? dfs.namenode.*?

          I like this idea. Does it mean that we have dfs.namenode.* dfs.client.* and dfs.datanode.* types of property names?

          Show
          dhruba borthakur added a comment - > dfs.max.replication.streams and dfs.heartbeat.recheck.interval should have a more specific context? dfs.namenode.*? I like this idea. Does it mean that we have dfs.namenode.* dfs.client.* and dfs.datanode.* types of property names?
          Hide
          Jitendra Nath Pandey added a comment -

          Updated list of changed keys is attached incorporating the suggestions.

          Show
          Jitendra Nath Pandey added a comment - Updated list of changed keys is attached incorporating the suggestions.
          Hide
          Jitendra Nath Pandey added a comment -

          unchanged_keys.txt file contains the keys that are assumed to be rightly named and will not be changed.

          Show
          Jitendra Nath Pandey added a comment - unchanged_keys.txt file contains the keys that are assumed to be rightly named and will not be changed.
          Hide
          Jitendra Nath Pandey added a comment -

          Resolving this as fixed, because related jiras are committed.

          Show
          Jitendra Nath Pandey added a comment - Resolving this as fixed, because related jiras are committed.

            People

            • Assignee:
              Jitendra Nath Pandey
              Reporter:
              Jitendra Nath Pandey
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development