Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1727

fsck command can display command usage if user passes any illegal argument

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.20.1, 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In fsck command if user passes the arguments like
      ./hadoop fsck -test -files -blocks -racks
      In this case it will take / and will display whole DFS information regarding to files,blocks,racks.

      But here, we are hiding the user mistake. Instead of this, we can display the command usage if user passes any invalid argument like above.

      If user passes illegal optional arguments like
      ./hadoop fsck /test -listcorruptfileblocks instead of
      ./hadoop fsck /test -list-corruptfileblocks also we can display the proper command usage

      1. HDFS-1727.patch
        4 kB
        sravankorumilli
      2. HDFS-1727.1.patch
        3 kB
        sravankorumilli
      3. HDFS-1727_3.patch
        3 kB
        sravankorumilli

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        7d 1h 14m 2 Uma Maheswara Rao G 31/May/11 16:29
        Open Open Patch Available Patch Available
        78d 3h 16m 3 Uma Maheswara Rao G 31/May/11 16:30
        Patch Available Patch Available Resolved Resolved
        2h 40m 1 Todd Lipcon 31/May/11 19:10
        Resolved Resolved Closed Closed
        167d 6h 42m 1 Arun C Murthy 15/Nov/11 00:52
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #685 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/685/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #685 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/685/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #697 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/697/)
        HDFS-1727. fsck command should display command usage if user passes any illegal argument. Contributed by Sravan Kumar.

        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1129831
        Files :

        • /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/tools/DFSck.java
        • /hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
        • /hadoop/hdfs/trunk/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #697 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/697/ ) HDFS-1727 . fsck command should display command usage if user passes any illegal argument. Contributed by Sravan Kumar. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1129831 Files : /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/tools/DFSck.java /hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestFsck.java /hadoop/hdfs/trunk/CHANGES.txt
        Todd Lipcon made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Fix Version/s 0.23.0 [ 12315571 ]
        Resolution Fixed [ 1 ]
        Hide
        Todd Lipcon added a comment -

        Committed to trunk. Thanks, Sravan!

        Show
        Todd Lipcon added a comment - Committed to trunk. Thanks, Sravan!
        Hide
        sravankorumilli added a comment -

        Thanks Todd, I would prefer Sravan Kumar .K

        Show
        sravankorumilli added a comment - Thanks Todd, I would prefer Sravan Kumar .K
        Hide
        Todd Lipcon added a comment -

        +1, looks good to me. How would you like your name to show up in CHANGES.txt? On JIRA it is just showing your username.

        Show
        Todd Lipcon added a comment - +1, looks good to me. How would you like your name to show up in CHANGES.txt? On JIRA it is just showing your username.
        Todd Lipcon made changes -
        Assignee sravankorumilli [ sravankorumilli ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12480941/HDFS-1727_3.patch
        against trunk revision 1128987.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/663//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/663//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/663//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12480941/HDFS-1727_3.patch against trunk revision 1128987. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/663//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/663//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/663//console This message is automatically generated.
        Uma Maheswara Rao G made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Uma Maheswara Rao G made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        sravankorumilli made changes -
        Attachment HDFS-1727_3.patch [ 12480941 ]
        Hide
        Todd Lipcon added a comment -

        Thanks for updating the patch. In the case that you pass two paths, though, it should print a more informative error – e.g "fsck: can only operate on on path at a time", rather than the same "Illegal option" error.

        Show
        Todd Lipcon added a comment - Thanks for updating the patch. In the case that you pass two paths, though, it should print a more informative error – e.g "fsck: can only operate on on path at a time", rather than the same "Illegal option" error.
        Hide
        sravankorumilli added a comment -

        Sorry, The above comments are given by me but Umamaheswara Rao has logged in my machine i did not check that.

        Show
        sravankorumilli added a comment - Sorry, The above comments are given by me but Umamaheswara Rao has logged in my machine i did not check that.
        Hide
        Uma Maheswara Rao G added a comment -

        I verified these test failures are not related to this.

        I have verified the logs the system test framework compilation failure is also not related to my changes. This is the Hudson report when I ran the patch verification locally in my CI.

        [exec] +1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] +1 tests included. The patch appears to include 4 new or modified tests.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
        [exec]
        [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
        [exec]
        [exec] +1 system test framework. The patch passed system test framework compile.

        Show
        Uma Maheswara Rao G added a comment - I verified these test failures are not related to this. I have verified the logs the system test framework compilation failure is also not related to my changes. This is the Hudson report when I ran the patch verification locally in my CI. [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 4 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12480517/HDFS-1727.1.patch
        against trunk revision 1127823.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:
        org.apache.hadoop.cli.TestHDFSCLI
        org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery

        +1 contrib tests. The patch passed contrib unit tests.

        -1 system test framework. The patch failed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/635//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/635//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/635//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12480517/HDFS-1727.1.patch against trunk revision 1127823. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery +1 contrib tests. The patch passed contrib unit tests. -1 system test framework. The patch failed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/635//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/635//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/635//console This message is automatically generated.
        Uma Maheswara Rao G made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Uma Maheswara Rao G made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        sravankorumilli made changes -
        Attachment HDFS-1727.1.patch [ 12480517 ]
        Hide
        sravankorumilli added a comment -

        Hi Todd,
        Thanks for your comments. I prepared the patch as per your comments please find the updated patch.

        Show
        sravankorumilli added a comment - Hi Todd, Thanks for your comments. I prepared the patch as per your comments please find the updated patch.
        Hide
        Todd Lipcon added a comment -

        One more comment: please remove the "@throws Exception" from the test javadoc. It doesn't give any extra information, so just clutters the code.

        Show
        Todd Lipcon added a comment - One more comment: please remove the "@throws Exception" from the test javadoc. It doesn't give any extra information, so just clutters the code.
        Hide
        Todd Lipcon added a comment -

        A few comments:

        • I think the null == dir check needs an else clause that prints an error that only one path may be passed. That is to say, if you run "hadoop fsck /foo /bar", it should print an error rather than ignoring /bar.
        • To make the tests run faster, I think it would be better for the two test cases to be combined and share the same MiniDFSCluster.
        • The test case would be easier to follow if you used an argument like "-asdfjisadfijs" or "-thisIsNotAValidFlag" instead of "-listcorruptfileblocks".
        • The error message could be formatted nicer. Instead of "Invalid arguments passed 'foo'" I would use the same formatting as the FsShell errors - ie "fsck: Illegal option 'foo'"
        Show
        Todd Lipcon added a comment - A few comments: I think the null == dir check needs an else clause that prints an error that only one path may be passed. That is to say, if you run "hadoop fsck /foo /bar", it should print an error rather than ignoring /bar. To make the tests run faster, I think it would be better for the two test cases to be combined and share the same MiniDFSCluster. The test case would be easier to follow if you used an argument like "-asdfjisadfijs" or "-thisIsNotAValidFlag" instead of "-listcorruptfileblocks". The error message could be formatted nicer. Instead of "Invalid arguments passed 'foo'" I would use the same formatting as the FsShell errors - ie "fsck: Illegal option 'foo'"
        Hide
        sravankorumilli added a comment -

        I have verified, these test failures are not related to this patch.

        Show
        sravankorumilli added a comment - I have verified, these test failures are not related to this patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12480252/HDFS-1727.patch
        against trunk revision 1126795.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:
        org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
        org.apache.hadoop.hdfs.TestFSOutputSummer

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/617//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/617//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/617//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12480252/HDFS-1727.patch against trunk revision 1126795. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestFSOutputSummer +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/617//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/617//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/617//console This message is automatically generated.
        sravankorumilli made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 0.20.1 [ 12314048 ]
        Affects Version/s 0.23.0 [ 12315571 ]
        sravankorumilli made changes -
        Attachment HDFS-1727.patch [ 12480252 ]
        Uma Maheswara Rao G made changes -
        Assignee Uma Maheswara Rao G [ umamaheswararao ]
        Uma Maheswara Rao G made changes -
        Field Original Value New Value
        Summary fsck command can display usage if user passes any other arguments with '-' ( other than -move, -delete, -files , -openforwrite, -blocks , -locations, -racks). fsck command can display command usage if user passes any illegal argument
        Assignee Uma Maheswara Rao G [ umamaheswararao ]
        Description In fsck command if user passes the arguments like
           ./hadoop fsck -test -files -blocks -racks
           In this case it will take / and will display whole DFS information regarding to files,blocks,racks.

        But here, we are hiding the user mistake. Instead of this, we can display the usage if user passes any invalid argument like above.
        In fsck command if user passes the arguments like
        ./hadoop fsck -test -files -blocks -racks
        In this case it will take / and will display whole DFS information regarding to files,blocks,racks.

        But here, we are hiding the user mistake. Instead of this, we can display the command usage if user passes any invalid argument like above.

        If user passes illegal optional arguments like
        ./hadoop fsck /test -listcorruptfileblocks instead of
        ./hadoop fsck /test -list-corruptfileblocks also we can display the proper command usage
        Uma Maheswara Rao G created issue -

          People

          • Assignee:
            sravankorumilli
            Reporter:
            Uma Maheswara Rao G
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development