Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1727

fsck command can display command usage if user passes any illegal argument

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.20.1, 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In fsck command if user passes the arguments like
      ./hadoop fsck -test -files -blocks -racks
      In this case it will take / and will display whole DFS information regarding to files,blocks,racks.

      But here, we are hiding the user mistake. Instead of this, we can display the command usage if user passes any invalid argument like above.

      If user passes illegal optional arguments like
      ./hadoop fsck /test -listcorruptfileblocks instead of
      ./hadoop fsck /test -list-corruptfileblocks also we can display the proper command usage

      1. HDFS-1727.patch
        4 kB
        sravankorumilli
      2. HDFS-1727.1.patch
        3 kB
        sravankorumilli
      3. HDFS-1727_3.patch
        3 kB
        sravankorumilli

        Activity

        Uma Maheswara Rao G created issue -
        Uma Maheswara Rao G made changes -
        Field Original Value New Value
        Summary fsck command can display usage if user passes any other arguments with '-' ( other than -move, -delete, -files , -openforwrite, -blocks , -locations, -racks). fsck command can display command usage if user passes any illegal argument
        Assignee Uma Maheswara Rao G [ umamaheswararao ]
        Description In fsck command if user passes the arguments like
           ./hadoop fsck -test -files -blocks -racks
           In this case it will take / and will display whole DFS information regarding to files,blocks,racks.

        But here, we are hiding the user mistake. Instead of this, we can display the usage if user passes any invalid argument like above.
        In fsck command if user passes the arguments like
        ./hadoop fsck -test -files -blocks -racks
        In this case it will take / and will display whole DFS information regarding to files,blocks,racks.

        But here, we are hiding the user mistake. Instead of this, we can display the command usage if user passes any invalid argument like above.

        If user passes illegal optional arguments like
        ./hadoop fsck /test -listcorruptfileblocks instead of
        ./hadoop fsck /test -list-corruptfileblocks also we can display the proper command usage
        Uma Maheswara Rao G made changes -
        Assignee Uma Maheswara Rao G [ umamaheswararao ]
        sravankorumilli made changes -
        Attachment HDFS-1727.patch [ 12480252 ]
        sravankorumilli made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 0.20.1 [ 12314048 ]
        Affects Version/s 0.23.0 [ 12315571 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12480252/HDFS-1727.patch
        against trunk revision 1126795.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:
        org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
        org.apache.hadoop.hdfs.TestFSOutputSummer

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/617//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/617//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/617//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12480252/HDFS-1727.patch against trunk revision 1126795. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestFSOutputSummer +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/617//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/617//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/617//console This message is automatically generated.
        Hide
        sravankorumilli added a comment -

        I have verified, these test failures are not related to this patch.

        Show
        sravankorumilli added a comment - I have verified, these test failures are not related to this patch.
        Hide
        Todd Lipcon added a comment -

        A few comments:

        • I think the null == dir check needs an else clause that prints an error that only one path may be passed. That is to say, if you run "hadoop fsck /foo /bar", it should print an error rather than ignoring /bar.
        • To make the tests run faster, I think it would be better for the two test cases to be combined and share the same MiniDFSCluster.
        • The test case would be easier to follow if you used an argument like "-asdfjisadfijs" or "-thisIsNotAValidFlag" instead of "-listcorruptfileblocks".
        • The error message could be formatted nicer. Instead of "Invalid arguments passed 'foo'" I would use the same formatting as the FsShell errors - ie "fsck: Illegal option 'foo'"
        Show
        Todd Lipcon added a comment - A few comments: I think the null == dir check needs an else clause that prints an error that only one path may be passed. That is to say, if you run "hadoop fsck /foo /bar", it should print an error rather than ignoring /bar. To make the tests run faster, I think it would be better for the two test cases to be combined and share the same MiniDFSCluster. The test case would be easier to follow if you used an argument like "-asdfjisadfijs" or "-thisIsNotAValidFlag" instead of "-listcorruptfileblocks". The error message could be formatted nicer. Instead of "Invalid arguments passed 'foo'" I would use the same formatting as the FsShell errors - ie "fsck: Illegal option 'foo'"
        Hide
        Todd Lipcon added a comment -

        One more comment: please remove the "@throws Exception" from the test javadoc. It doesn't give any extra information, so just clutters the code.

        Show
        Todd Lipcon added a comment - One more comment: please remove the "@throws Exception" from the test javadoc. It doesn't give any extra information, so just clutters the code.
        Hide
        sravankorumilli added a comment -

        Hi Todd,
        Thanks for your comments. I prepared the patch as per your comments please find the updated patch.

        Show
        sravankorumilli added a comment - Hi Todd, Thanks for your comments. I prepared the patch as per your comments please find the updated patch.
        sravankorumilli made changes -
        Attachment HDFS-1727.1.patch [ 12480517 ]
        Uma Maheswara Rao G made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Uma Maheswara Rao G made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12480517/HDFS-1727.1.patch
        against trunk revision 1127823.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:
        org.apache.hadoop.cli.TestHDFSCLI
        org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery

        +1 contrib tests. The patch passed contrib unit tests.

        -1 system test framework. The patch failed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/635//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/635//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/635//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12480517/HDFS-1727.1.patch against trunk revision 1127823. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery +1 contrib tests. The patch passed contrib unit tests. -1 system test framework. The patch failed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/635//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/635//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/635//console This message is automatically generated.
        Hide
        Uma Maheswara Rao G added a comment -

        I verified these test failures are not related to this.

        I have verified the logs the system test framework compilation failure is also not related to my changes. This is the Hudson report when I ran the patch verification locally in my CI.

        [exec] +1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] +1 tests included. The patch appears to include 4 new or modified tests.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
        [exec]
        [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
        [exec]
        [exec] +1 system test framework. The patch passed system test framework compile.

        Show
        Uma Maheswara Rao G added a comment - I verified these test failures are not related to this. I have verified the logs the system test framework compilation failure is also not related to my changes. This is the Hudson report when I ran the patch verification locally in my CI. [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 4 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile.
        Hide
        sravankorumilli added a comment -

        Sorry, The above comments are given by me but Umamaheswara Rao has logged in my machine i did not check that.

        Show
        sravankorumilli added a comment - Sorry, The above comments are given by me but Umamaheswara Rao has logged in my machine i did not check that.
        Hide
        Todd Lipcon added a comment -

        Thanks for updating the patch. In the case that you pass two paths, though, it should print a more informative error – e.g "fsck: can only operate on on path at a time", rather than the same "Illegal option" error.

        Show
        Todd Lipcon added a comment - Thanks for updating the patch. In the case that you pass two paths, though, it should print a more informative error – e.g "fsck: can only operate on on path at a time", rather than the same "Illegal option" error.
        sravankorumilli made changes -
        Attachment HDFS-1727_3.patch [ 12480941 ]
        Uma Maheswara Rao G made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Uma Maheswara Rao G made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12480941/HDFS-1727_3.patch
        against trunk revision 1128987.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/663//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/663//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/663//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12480941/HDFS-1727_3.patch against trunk revision 1128987. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/663//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/663//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/663//console This message is automatically generated.
        Todd Lipcon made changes -
        Assignee sravankorumilli [ sravankorumilli ]
        Hide
        Todd Lipcon added a comment -

        +1, looks good to me. How would you like your name to show up in CHANGES.txt? On JIRA it is just showing your username.

        Show
        Todd Lipcon added a comment - +1, looks good to me. How would you like your name to show up in CHANGES.txt? On JIRA it is just showing your username.
        Hide
        sravankorumilli added a comment -

        Thanks Todd, I would prefer Sravan Kumar .K

        Show
        sravankorumilli added a comment - Thanks Todd, I would prefer Sravan Kumar .K
        Hide
        Todd Lipcon added a comment -

        Committed to trunk. Thanks, Sravan!

        Show
        Todd Lipcon added a comment - Committed to trunk. Thanks, Sravan!
        Todd Lipcon made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Fix Version/s 0.23.0 [ 12315571 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #697 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/697/)
        HDFS-1727. fsck command should display command usage if user passes any illegal argument. Contributed by Sravan Kumar.

        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1129831
        Files :

        • /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/tools/DFSck.java
        • /hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
        • /hadoop/hdfs/trunk/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #697 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/697/ ) HDFS-1727 . fsck command should display command usage if user passes any illegal argument. Contributed by Sravan Kumar. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1129831 Files : /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/tools/DFSck.java /hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestFsck.java /hadoop/hdfs/trunk/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #685 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/685/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #685 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/685/ )
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            sravankorumilli
            Reporter:
            Uma Maheswara Rao G
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development