Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-763

DataBlockScanner reporting of bad blocks is slightly misleading

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.21.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The Datanode generates a report of the period block scanning that verifies crcs. It reports something like the following:

      Scans since restart : 192266
      Scan errors since restart : 33
      Transient scan errors : 0

      The statement saying that there were 33 errors is slightly midleading because these are not crc mismatches, rather the block was being deleted when the crc verification was about to happen.

      I propose that DataBlockScanner.totalScanErrors is not updated if the dataset.getFile(block) is null, i.e. the block is now deleted from the datanode.

      1. scanErrors.txt
        2 kB
        dhruba borthakur
      2. scanErrors.txt
        0.8 kB
        dhruba borthakur
      3. scanErrors.txt
        0.8 kB
        dhruba borthakur

        Activity

          People

          • Assignee:
            dhruba borthakur
            Reporter:
            dhruba borthakur
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development