Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5723

Append failed FINALIZED replica should not be accepted as valid when that block is underconstruction

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.6.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Scenario:
      1. 3 node cluster with dfs.client.block.write.replace-datanode-on-failure.enable set to false.
      2. One file is written with 3 replicas, blk_id_gs1
      3. One of the datanode DN1 is down.
      4. File was opened with append and some more data is added to the file and synced. (to only 2 live nodes DN2 and DN3)-- blk_id_gs2
      5. Now DN1 restarted
      6. In this block report, DN1 reported FINALIZED block blk_id_gs1, this should be marked corrupted.
      but since NN having appended block state as UnderConstruction, at this time its not detecting this block as corrupt and adding to valid block locations.

      As long as the namenode is alive, this datanode also will be considered as valid replica and read/append will fail in that datanode.

        Attachments

        1. HDFS-5723.patch
          6 kB
          Vinayakumar B
        2. HDFS-5723.patch
          6 kB
          Vinayakumar B
        3. HDFS-5723.patch
          4 kB
          Vinayakumar B

          Activity

            People

            • Assignee:
              vinayakumarb Vinayakumar B
              Reporter:
              vinayakumarb Vinayakumar B
            • Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: