Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2798

Append may race with datanode block scanner, causing replica to be incorrectly marked corrupt

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Duplicate
    • Affects Version/s: 0.22.0, 0.23.0
    • Fix Version/s: None
    • Component/s: datanode
    • Labels:
      None

      Description

      When a pipeline is setup for append, the block's metadata file is renamed before the block is removed from the datanode block scanner queues. This can cause a race condition where the block scanner incorrectly marks the block as corrupt, since it tries to scan the file corresponding to the old genstamp.

      This causes TestFileAppend2 to time out in extremely rare circumstances - the corrupt replica prevents the writer thread from completing the file.

        Attachments

          Activity

            People

            • Assignee:
              brandonli Brandon Li
              Reporter:
              tlipcon Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: