Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1381

HDFS javadocs hard-code references to dfs.namenode.name.dir and dfs.datanode.data.dir parameters

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.20.1
    • 0.23.0
    • test
    • Updated the JavaDocs to appropriately represent the new Configuration Keys that are used in the code. The docs did not match the code.
    • newbie

    Description

      The javadoc for MiniDFSCluster makes repeated references to setting dfs.name.dir and dfs.data.dir. These should be replaced with references to DFSConfigKeys' DFS_NAMENODE_NAME_DIR_KEY and DFS_DATANODE_DATA_DIR_KEY, respectively. The old values are deprecated in DFSConfigKeys, but we should switch to the new values where ever we can.

      Also, a quick search the code shows that TestDFSStorageStateRecovery.java and UpgradeUtilities.java should be updated as well.

      Attachments

        1. HDFS-1381-take2.txt
          12 kB
          Jim Plush
        2. HDFS-1381-take1.txt
          11 kB
          Jim Plush

        Activity

          atm Aaron Myers added a comment -

          Update: though these configs no longer show up in MiniDFSCluster, they're still in a few places they shouldn't be.

          atm Aaron Myers added a comment - Update: though these configs no longer show up in MiniDFSCluster , they're still in a few places they shouldn't be.
          jimplush Jim Plush added a comment -

          literally replaced the mentioned the name.dir and data.dir javadoc mentions with their fully qualified ConfigKey names DFSConfigKeys.DFS_NAMENODE_NAME_DIR_KEY and DFSConfigKeys.DFS_DATANODE_DATA_DIR_KEY

          jimplush Jim Plush added a comment - literally replaced the mentioned the name.dir and data.dir javadoc mentions with their fully qualified ConfigKey names DFSConfigKeys.DFS_NAMENODE_NAME_DIR_KEY and DFSConfigKeys.DFS_DATANODE_DATA_DIR_KEY
          jimplush Jim Plush added a comment -

          Updated the JavaDocs to represent the keys mentioned in the bug report.

          jimplush Jim Plush added a comment - Updated the JavaDocs to represent the keys mentioned in the bug report.
          atm Aaron Myers added a comment -

          Hey Jim, patch looks great. Thanks a lot for doing this.

          Only thing is that I'd recommend using the @link annotation with the '#' symbol to separate members so that when Javadocs are generated they'll include hyperlinks to the right place. This will also let Eclipse and other IDEs refactor the comments correctly if we ever change the names of these variables.

          For example, instead of this:

          DFSConfigKeys.DFS_NAMENODE_NAME_DIR_KEY
          

          do this:

          {@link DFSConfigKeys#DFS_NAMENODE_NAME_DIR_KEY}
          

          You should be able to find examples of this throughout the code base.

          atm Aaron Myers added a comment - Hey Jim, patch looks great. Thanks a lot for doing this. Only thing is that I'd recommend using the @link annotation with the '#' symbol to separate members so that when Javadocs are generated they'll include hyperlinks to the right place. This will also let Eclipse and other IDEs refactor the comments correctly if we ever change the names of these variables. For example, instead of this: DFSConfigKeys.DFS_NAMENODE_NAME_DIR_KEY do this: {@link DFSConfigKeys#DFS_NAMENODE_NAME_DIR_KEY} You should be able to find examples of this throughout the code base.
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12483782/HDFS-1381-take1.txt
          against trunk revision 1139397.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/834//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/834//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/834//console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483782/HDFS-1381-take1.txt against trunk revision 1139397. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/834//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/834//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/834//console This message is automatically generated.
          jimplush Jim Plush added a comment -

          updating with Aaron's recommended changes for using @link tags to the Class#Member

          jimplush Jim Plush added a comment - updating with Aaron's recommended changes for using @link tags to the Class#Member
          atm Aaron Myers added a comment -

          +1, the patch looks good to me. Will commit pending Hudson results.

          atm Aaron Myers added a comment - +1, the patch looks good to me. Will commit pending Hudson results.
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12483792/HDFS-1381-take2.txt
          against trunk revision 1139397.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/836//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/836//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/836//console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483792/HDFS-1381-take2.txt against trunk revision 1139397. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/836//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/836//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/836//console This message is automatically generated.
          atm Aaron Myers added a comment -

          Updating JIRA title to more accurately reflect the change which this patch will introduce.

          atm Aaron Myers added a comment - Updating JIRA title to more accurately reflect the change which this patch will introduce.
          atm Aaron Myers added a comment -

          I've just committed this. Thanks a lot for the contribution, Jim!

          atm Aaron Myers added a comment - I've just committed this. Thanks a lot for the contribution, Jim!
          hudson Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #760 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/760/)
          HDFS-1381. HDFS javadocs hard-code references to dfs.namenode.name.dir and dfs.datanode.data.dir parameters (Jim Plush via atm)

          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139715
          Files :

          • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDFSStorageStateRecovery.java
          • /hadoop/common/trunk/hdfs/CHANGES.txt
          • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/MiniDFSCluster.java
          • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java
          hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #760 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/760/ ) HDFS-1381 . HDFS javadocs hard-code references to dfs.namenode.name.dir and dfs.datanode.data.dir parameters (Jim Plush via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139715 Files : /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDFSStorageStateRecovery.java /hadoop/common/trunk/hdfs/CHANGES.txt /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/MiniDFSCluster.java /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java
          hudson Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #707 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/707/)
          HDFS-1381. HDFS javadocs hard-code references to dfs.namenode.name.dir and dfs.datanode.data.dir parameters (Jim Plush via atm)

          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139715
          Files :

          • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDFSStorageStateRecovery.java
          • /hadoop/common/trunk/hdfs/CHANGES.txt
          • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/MiniDFSCluster.java
          • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java
          hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #707 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/707/ ) HDFS-1381 . HDFS javadocs hard-code references to dfs.namenode.name.dir and dfs.datanode.data.dir parameters (Jim Plush via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139715 Files : /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDFSStorageStateRecovery.java /hadoop/common/trunk/hdfs/CHANGES.txt /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/MiniDFSCluster.java /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java

          People

            jimplush Jim Plush
            jghoman Jakob Homan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: