Hadoop Common
  1. Hadoop Common
  2. HADOOP-943

fsck to show the filename of the corrupted file

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      When fsck / shows some corrupted file/block information,

      *******************************
      CORRUPT FILES: 1
      MISSING BLOCKS: 1
      MISSING SIZE: ___ B
      ********************************

      it doesn't show the filename.

      I can run fsck / -files again, but this would take a lot longer than the plain fsck.

        Activity

        Hide
        Koji Noguchi added a comment -

        It'll be nice if this can also print ouf the info(file/block) for under-replicated blocks.

        Show
        Koji Noguchi added a comment - It'll be nice if this can also print ouf the info(file/block) for under-replicated blocks.
        Hide
        dhruba borthakur added a comment -

        In the current implementation, underReplicatedBlocks is actually the distance of all blocks from their target replication factor. For example, if there were two blocks whose target replication factor was 3 and each of those blocks had one replica, then fsck will display underReplicatedBlocks as 4.

        My change will display underReplicatedBlocks as 2 in the above case.

        Show
        dhruba borthakur added a comment - In the current implementation, underReplicatedBlocks is actually the distance of all blocks from their target replication factor. For example, if there were two blocks whose target replication factor was 3 and each of those blocks had one replica, then fsck will display underReplicatedBlocks as 4. My change will display underReplicatedBlocks as 2 in the above case.
        Hide
        dhruba borthakur added a comment -

        When fsck detects under-replicated blocks or corrupted blocks, it prints out the corresponding filename.

        Show
        dhruba borthakur added a comment - When fsck detects under-replicated blocks or corrupted blocks, it prints out the corresponding filename.
        Hide
        Milind Bhandarkar added a comment -

        +1 patch reviewed.

        Dhruba,

        Can we eliminate the printing of '.' in fsck ? That would simplify processing the report mechanically. It is tricky though, because the http client may timeout on a large fsck printing all its output at the end.

        For -files, -blocks, -locations, can we make the fsck output more machine-processable (say limiting info about each file to a single line?)

        Show
        Milind Bhandarkar added a comment - +1 patch reviewed. Dhruba, Can we eliminate the printing of '.' in fsck ? That would simplify processing the report mechanically. It is tricky though, because the http client may timeout on a large fsck printing all its output at the end. For -files, -blocks, -locations, can we make the fsck output more machine-processable (say limiting info about each file to a single line?)
        Hide
        dhruba borthakur added a comment -

        I would like to defer the requirement "make the output of fsck machine readable" to a separate JIRA issue.

        Show
        dhruba borthakur added a comment - I would like to defer the requirement "make the output of fsck machine readable" to a separate JIRA issue.
        Hide
        Hadoop QA added a comment -

        +1, because http://issues.apache.org/jira/secure/attachment/12351197/fsckPrintCorrupt.patch applied and successfully tested against trunk revision r507276.

        Show
        Hadoop QA added a comment - +1, because http://issues.apache.org/jira/secure/attachment/12351197/fsckPrintCorrupt.patch applied and successfully tested against trunk revision r507276.
        Hide
        Doug Cutting added a comment -

        I just committed this. Thanks, Dhruba.

        Show
        Doug Cutting added a comment - I just committed this. Thanks, Dhruba.

          People

          • Assignee:
            dhruba borthakur
            Reporter:
            Koji Noguchi
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development