Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1381

HDFS javadocs hard-code references to dfs.namenode.name.dir and dfs.datanode.data.dir parameters

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.23.0
    • Component/s: test
    • Labels:
    • Release Note:
      Updated the JavaDocs to appropriately represent the new Configuration Keys that are used in the code. The docs did not match the code.
    • Tags:
      newbie

      Description

      The javadoc for MiniDFSCluster makes repeated references to setting dfs.name.dir and dfs.data.dir. These should be replaced with references to DFSConfigKeys' DFS_NAMENODE_NAME_DIR_KEY and DFS_DATANODE_DATA_DIR_KEY, respectively. The old values are deprecated in DFSConfigKeys, but we should switch to the new values where ever we can.

      Also, a quick search the code shows that TestDFSStorageStateRecovery.java and UpgradeUtilities.java should be updated as well.

      1. HDFS-1381-take2.txt
        12 kB
        Jim Plush
      2. HDFS-1381-take1.txt
        11 kB
        Jim Plush

        Activity

        Hide
        Aaron T. Myers added a comment -

        Update: though these configs no longer show up in MiniDFSCluster, they're still in a few places they shouldn't be.

        Show
        Aaron T. Myers added a comment - Update: though these configs no longer show up in MiniDFSCluster , they're still in a few places they shouldn't be.
        Hide
        Jim Plush added a comment -

        literally replaced the mentioned the name.dir and data.dir javadoc mentions with their fully qualified ConfigKey names DFSConfigKeys.DFS_NAMENODE_NAME_DIR_KEY and DFSConfigKeys.DFS_DATANODE_DATA_DIR_KEY

        Show
        Jim Plush added a comment - literally replaced the mentioned the name.dir and data.dir javadoc mentions with their fully qualified ConfigKey names DFSConfigKeys.DFS_NAMENODE_NAME_DIR_KEY and DFSConfigKeys.DFS_DATANODE_DATA_DIR_KEY
        Hide
        Jim Plush added a comment -

        Updated the JavaDocs to represent the keys mentioned in the bug report.

        Show
        Jim Plush added a comment - Updated the JavaDocs to represent the keys mentioned in the bug report.
        Hide
        Aaron T. Myers added a comment -

        Hey Jim, patch looks great. Thanks a lot for doing this.

        Only thing is that I'd recommend using the @link annotation with the '#' symbol to separate members so that when Javadocs are generated they'll include hyperlinks to the right place. This will also let Eclipse and other IDEs refactor the comments correctly if we ever change the names of these variables.

        For example, instead of this:

        DFSConfigKeys.DFS_NAMENODE_NAME_DIR_KEY
        

        do this:

        {@link DFSConfigKeys#DFS_NAMENODE_NAME_DIR_KEY}
        

        You should be able to find examples of this throughout the code base.

        Show
        Aaron T. Myers added a comment - Hey Jim, patch looks great. Thanks a lot for doing this. Only thing is that I'd recommend using the @link annotation with the '#' symbol to separate members so that when Javadocs are generated they'll include hyperlinks to the right place. This will also let Eclipse and other IDEs refactor the comments correctly if we ever change the names of these variables. For example, instead of this: DFSConfigKeys.DFS_NAMENODE_NAME_DIR_KEY do this: {@link DFSConfigKeys#DFS_NAMENODE_NAME_DIR_KEY} You should be able to find examples of this throughout the code base.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12483782/HDFS-1381-take1.txt
        against trunk revision 1139397.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 9 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/834//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/834//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/834//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483782/HDFS-1381-take1.txt against trunk revision 1139397. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/834//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/834//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/834//console This message is automatically generated.
        Hide
        Jim Plush added a comment -

        updating with Aaron's recommended changes for using @link tags to the Class#Member

        Show
        Jim Plush added a comment - updating with Aaron's recommended changes for using @link tags to the Class#Member
        Hide
        Aaron T. Myers added a comment -

        +1, the patch looks good to me. Will commit pending Hudson results.

        Show
        Aaron T. Myers added a comment - +1, the patch looks good to me. Will commit pending Hudson results.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12483792/HDFS-1381-take2.txt
        against trunk revision 1139397.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 9 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/836//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/836//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/836//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483792/HDFS-1381-take2.txt against trunk revision 1139397. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/836//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/836//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/836//console This message is automatically generated.
        Hide
        Aaron T. Myers added a comment -

        Updating JIRA title to more accurately reflect the change which this patch will introduce.

        Show
        Aaron T. Myers added a comment - Updating JIRA title to more accurately reflect the change which this patch will introduce.
        Hide
        Aaron T. Myers added a comment -

        I've just committed this. Thanks a lot for the contribution, Jim!

        Show
        Aaron T. Myers added a comment - I've just committed this. Thanks a lot for the contribution, Jim!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #760 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/760/)
        HDFS-1381. HDFS javadocs hard-code references to dfs.namenode.name.dir and dfs.datanode.data.dir parameters (Jim Plush via atm)

        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139715
        Files :

        • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDFSStorageStateRecovery.java
        • /hadoop/common/trunk/hdfs/CHANGES.txt
        • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #760 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/760/ ) HDFS-1381 . HDFS javadocs hard-code references to dfs.namenode.name.dir and dfs.datanode.data.dir parameters (Jim Plush via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139715 Files : /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDFSStorageStateRecovery.java /hadoop/common/trunk/hdfs/CHANGES.txt /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/MiniDFSCluster.java /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #707 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/707/)
        HDFS-1381. HDFS javadocs hard-code references to dfs.namenode.name.dir and dfs.datanode.data.dir parameters (Jim Plush via atm)

        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139715
        Files :

        • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDFSStorageStateRecovery.java
        • /hadoop/common/trunk/hdfs/CHANGES.txt
        • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #707 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/707/ ) HDFS-1381 . HDFS javadocs hard-code references to dfs.namenode.name.dir and dfs.datanode.data.dir parameters (Jim Plush via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139715 Files : /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDFSStorageStateRecovery.java /hadoop/common/trunk/hdfs/CHANGES.txt /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/MiniDFSCluster.java /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java

          People

          • Assignee:
            Jim Plush
            Reporter:
            Jakob Homan
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development