Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-338

When a block is severely under replicated at creation time, a request for block replication should be scheduled immediately

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      During writing a block to data nodes, if the dfs client detects a bad data node in the write pipeline, it will re-construct a new data pipeline,
      excluding the detected bad data node. This implies that when the client finishes writing the block, the number of the replicas for the block
      may be lower than the intended replication factor. If the ratio of the number of replicas to the intended replication factor is lower than
      certain threshold (say 0.68), then the client should send a request to the name node to replicate that block immediately.

        Activity

        Hide
        dhruba dhruba borthakur added a comment -

        When a client gets a new block from the namenode, it can tell the namenode the number of replicas of the previous block that it successfully wrote to. If this number is smaller than the target replication factor for the file, the namenode can immediately schedule replication for it,

        Show
        dhruba dhruba borthakur added a comment - When a client gets a new block from the namenode, it can tell the namenode the number of replicas of the previous block that it successfully wrote to. If this number is smaller than the target replication factor for the file, the namenode can immediately schedule replication for it,
        Hide
        aw Allen Wittenauer added a comment -

        Closing this as stale.

        Show
        aw Allen Wittenauer added a comment - Closing this as stale.

          People

          • Assignee:
            Unassigned
            Reporter:
            runping Runping Qi
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development