Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.19.0
    • Component/s: fs, util
    • Labels:
      None
    • Environment:

      branch-0.18

    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Added a check to fsck options to make sure -files is not the first option so as to resolve conflicts with GenericOptionsParser.

      Description

      Somehow depending on the order of options, GenericOptionsParser throws an error.

      This fail.

      [knoguchi@gsgw2001 tmp]$ ~/branch-0.18/bin/hadoop fsck
      Usage: DFSck <path> [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]]
      
      [knoguchi@tmp]$ ~/branch-0.18/bin/hadoop fsck -files -blocks -locations /user/knoguchi
      java.io.FileNotFoundException: File -blocks does not exist.
              at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:278)
              at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:233)
              at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:315)
              at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:134)
              at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:119)
              at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59)
              at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
              at org.apache.hadoop.dfs.DFSck.main(DFSck.java:137)
      
      ...Status: HEALTHY
       Total size:    2769 B
       Total dirs:    4
       Total files:   3
       Total blocks (validated):      3 (avg. block size 923 B)
      ...
      

      This works.

      [knoguchi@tmp]$ ~/branch-0.18/bin/hadoop fsck -blocks -locations -files /user/knoguchi
      /user/knoguchi/.Trash <dir>
      /user/knoguchi/hod-logs <dir>
      /user/knoguchi/hod-logs/____ <dir>
      /user/knoguchi/hod-logs/____/aaa.tar.gz 1024 bytes, 1 block(s):  OK
      0. blk_-5724352734215884188_0 len=1024 repl=3 [11.111.11.11:11111, 22.22.22.22:11111, 33.33.33.33:11111]
      ...
      /user/knoguchi/mapredsystem <dir>
      Status: HEALTHY
       Total size:    2769 B
       Total dirs:    4
       Total files:   3
       Total blocks (validated):      3 (avg. block size 923 B)
       
      1. HADOOP-3911.patch
        3 kB
        Lohit Vijayarenu

        Activity

        Hide
        knoguchi Koji Noguchi added a comment -

        Lohit told me that path has to come before the options.

        Before version 0.18, it used to be lenient with order of arguments, however, usage clearly states the correct command.

        Closing as invalid.

        Show
        knoguchi Koji Noguchi added a comment - Lohit told me that path has to come before the options. Before version 0.18, it used to be lenient with order of arguments, however, usage clearly states the correct command. Closing as invalid.
        Hide
        nidaley Nigel Daley added a comment -

        But Koji, this is still a bug if a Java stack trace is printed. No command-line commands should spill stack traces.

        Show
        nidaley Nigel Daley added a comment - But Koji, this is still a bug if a Java stack trace is printed. No command-line commands should spill stack traces.
        Hide
        knoguchi Koji Noguchi added a comment -

        > But Koji, this is still a bug if a Java stack trace is printed. No command-line commands should spill stack traces.
        >
        True.
        Ok, reopening the Jira.
        I'll change the priority to minor next.

        Show
        knoguchi Koji Noguchi added a comment - > But Koji, this is still a bug if a Java stack trace is printed. No command-line commands should spill stack traces. > True. Ok, reopening the Jira. I'll change the priority to minor next.
        Hide
        lohit Lohit Vijayarenu added a comment -

        Yes, this has to be fixed. Nothing seems to be changed in terms of options parsing of fsck. Either it was broken from beginning or something changed in GenericOptionsParser.

        Show
        lohit Lohit Vijayarenu added a comment - Yes, this has to be fixed. Nothing seems to be changed in terms of options parsing of fsck. Either it was broken from beginning or something changed in GenericOptionsParser.
        Hide
        lohit Lohit Vijayarenu added a comment - - edited

        This was not happenning in 0.18 becuase GenericOptionsParser did not consider -files to be a generic option. What seems to be happening here is that -files which was used by fsck is being treated as an option for GenericOptionsParser. It might not be good idea to change fsck's -files option now. I think making a check that is not the first argument works. Here is a patch for that.

        Show
        lohit Lohit Vijayarenu added a comment - - edited This was not happenning in 0.18 becuase GenericOptionsParser did not consider -files to be a generic option. What seems to be happening here is that -files which was used by fsck is being treated as an option for GenericOptionsParser. It might not be good idea to change fsck's -files option now. I think making a check that is not the first argument works. Here is a patch for that.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12390048/HADOOP-3911.patch
        against trunk revision 695755.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3270/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3270/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3270/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3270/console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12390048/HADOOP-3911.patch against trunk revision 695755. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3270/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3270/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3270/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3270/console This message is automatically generated.
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        +1 patch looks good.

        need release note.

        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - +1 patch looks good. need release note.
        Hide
        lohit Lohit Vijayarenu added a comment -

        Thanks for review Nicolas,
        I just committed this.

        Show
        lohit Lohit Vijayarenu added a comment - Thanks for review Nicolas, I just committed this.
        Hide
        hudson Hudson added a comment -
        Show
        hudson Hudson added a comment - Integrated in Hadoop-trunk #611 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/611/ )

          People

          • Assignee:
            lohit Lohit Vijayarenu
            Reporter:
            knoguchi Koji Noguchi
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development