Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3226

Allow GetConf tool to print arbitrary keys

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.0-alpha
    • Component/s: tools
    • Labels:
      None

      Description

      Currently the "hdfs getconf" tool can only print out certain keys, like the list of NNs, etc. It would be handy to allow it to fetch an arbitrary configuration. For example, users may wish to write shell scripts that interact with their hadoop cluster, and it is useful to be able to fetch configs like the name of the superuser, or the state of whether HA is enabled.

      1. hdfs-3226.txt
        10 kB
        Todd Lipcon

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1045 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1045/)
          HDFS-3234. Accidentally left log message in GetConf after HDFS-3226. Contributed by Todd Lipcon. (Revision 1311541)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311541
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1045 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1045/ ) HDFS-3234 . Accidentally left log message in GetConf after HDFS-3226 . Contributed by Todd Lipcon. (Revision 1311541) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311541 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1010 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1010/)
          HDFS-3234. Accidentally left log message in GetConf after HDFS-3226. Contributed by Todd Lipcon. (Revision 1311541)

          Result = FAILURE
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311541
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1010 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1010/ ) HDFS-3234 . Accidentally left log message in GetConf after HDFS-3226 . Contributed by Todd Lipcon. (Revision 1311541) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311541 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2108 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2108/)
          HDFS-3234. Accidentally left log message in GetConf after HDFS-3226. Contributed by Todd Lipcon. (Revision 1311541)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311541
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2108 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2108/ ) HDFS-3234 . Accidentally left log message in GetConf after HDFS-3226 . Contributed by Todd Lipcon. (Revision 1311541) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311541 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2033 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2033/)
          HDFS-3234. Accidentally left log message in GetConf after HDFS-3226. Contributed by Todd Lipcon. (Revision 1311541)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311541
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2033 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2033/ ) HDFS-3234 . Accidentally left log message in GetConf after HDFS-3226 . Contributed by Todd Lipcon. (Revision 1311541) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311541 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2045 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2045/)
          HDFS-3234. Accidentally left log message in GetConf after HDFS-3226. Contributed by Todd Lipcon. (Revision 1311541)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311541
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2045 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2045/ ) HDFS-3234 . Accidentally left log message in GetConf after HDFS-3226 . Contributed by Todd Lipcon. (Revision 1311541) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311541 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1042 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1042/)
          HDFS-3226. Allow GetConf tool to print arbitrary keys. Contributed by Todd Lipcon. (Revision 1310648)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1310648
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1042 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1042/ ) HDFS-3226 . Allow GetConf tool to print arbitrary keys. Contributed by Todd Lipcon. (Revision 1310648) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1310648 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1007 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1007/)
          HDFS-3226. Allow GetConf tool to print arbitrary keys. Contributed by Todd Lipcon. (Revision 1310648)

          Result = FAILURE
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1310648
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1007 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1007/ ) HDFS-3226 . Allow GetConf tool to print arbitrary keys. Contributed by Todd Lipcon. (Revision 1310648) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1310648 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2036 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2036/)
          HDFS-3226. Allow GetConf tool to print arbitrary keys. Contributed by Todd Lipcon. (Revision 1310648)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1310648
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2036 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2036/ ) HDFS-3226 . Allow GetConf tool to print arbitrary keys. Contributed by Todd Lipcon. (Revision 1310648) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1310648 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2025 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2025/)
          HDFS-3226. Allow GetConf tool to print arbitrary keys. Contributed by Todd Lipcon. (Revision 1310648)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1310648
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2025 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2025/ ) HDFS-3226 . Allow GetConf tool to print arbitrary keys. Contributed by Todd Lipcon. (Revision 1310648) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1310648 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2100 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2100/)
          HDFS-3226. Allow GetConf tool to print arbitrary keys. Contributed by Todd Lipcon. (Revision 1310648)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1310648
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2100 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2100/ ) HDFS-3226 . Allow GetConf tool to print arbitrary keys. Contributed by Todd Lipcon. (Revision 1310648) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1310648 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/GetConf.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java
          Hide
          Todd Lipcon added a comment -

          The failed test is unrelated. Thanks for the review, Eli. Committed to trunk and branch-2.

          Show
          Todd Lipcon added a comment - The failed test is unrelated. Thanks for the review, Eli. Committed to trunk and branch-2.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12521735/hdfs-3226.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hdfs.TestDatanodeBlockScanner

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2219//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2219//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521735/hdfs-3226.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestDatanodeBlockScanner +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2219//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2219//console This message is automatically generated.
          Hide
          Eli Collins added a comment -

          +1 pending jenkins

          Show
          Eli Collins added a comment - +1 pending jenkins
          Hide
          Todd Lipcon added a comment -

          Attached patch adds: hdfs getconf -confKey foo which will read and print the given key. I also made it always trim the results, since there are no configs where whitespace around the edge is semantically important, and trimming in bash is a pain.

          Show
          Todd Lipcon added a comment - Attached patch adds: hdfs getconf -confKey foo which will read and print the given key. I also made it always trim the results, since there are no configs where whitespace around the edge is semantically important, and trimming in bash is a pain.

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development