Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-763

DataBlockScanner reporting of bad blocks is slightly misleading

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.21.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The Datanode generates a report of the period block scanning that verifies crcs. It reports something like the following:

      Scans since restart : 192266
      Scan errors since restart : 33
      Transient scan errors : 0

      The statement saying that there were 33 errors is slightly midleading because these are not crc mismatches, rather the block was being deleted when the crc verification was about to happen.

      I propose that DataBlockScanner.totalScanErrors is not updated if the dataset.getFile(block) is null, i.e. the block is now deleted from the datanode.

      1. scanErrors.txt
        2 kB
        dhruba borthakur
      2. scanErrors.txt
        0.8 kB
        dhruba borthakur
      3. scanErrors.txt
        0.8 kB
        dhruba borthakur

        Activity

        dhruba borthakur created issue -
        dhruba borthakur made changes -
        Field Original Value New Value
        Attachment scanErrors.txt [ 12424760 ]
        dhruba borthakur made changes -
        Attachment scanErrors.txt [ 12424762 ]
        dhruba borthakur made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        dhruba borthakur made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        dhruba borthakur made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        dhruba borthakur made changes -
        Attachment scanErrors.txt [ 12424830 ]
        dhruba borthakur made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        dhruba borthakur made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        dhruba borthakur made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Fix Version/s 0.22.0 [ 12314241 ]
        Resolution Fixed [ 1 ]
        Tom White made changes -
        Fix Version/s 0.21.0 [ 12314046 ]
        Fix Version/s 0.22.0 [ 12314241 ]
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            dhruba borthakur
            Reporter:
            dhruba borthakur
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development