Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4975

Branch-1-win TestReplicationPolicy failed caused by stale data node handling

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1-win
    • Fix Version/s: 1-win
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      TestReplicationPolicy failed on

      • testChooseTargetWithMoreThanAvailableNodes()
      • testChooseTargetWithStaleNodes()
      • testChooseTargetWithHalfStaleNodes()

      The root of cause of testChooseTargetWithMoreThanAvailableNodes failing is the following:
      In BlockPlacementPolicyDefault#chooseTarget()

        chooseRandom(numOfReplicas, NodeBase.ROOT, excludedNodes, 
              blocksize, maxNodesPerRack, results);
          } catch (NotEnoughReplicasException e) {
            FSNamesystem.LOG.warn("Not able to place enough replicas, still in need of " + numOfReplicas);
      

      However, numOfReplicas is passed into chooseRandom() as int (primitive type in java) by value. The updating operation for numOfReplicas in chooseRandom() will not change the value in chooseTarget().

      The root cause for testChooseTargetWithStaleNodes() and testChooseTargetWithHalfStaleNodes() is the current BlockPlacementPolicyDefault#chooseTarget() doesn't check if a node is stale.

        Attachments

        1. HDFS-4975.2.patch
          22 kB
          Douma Fang
        2. HADOOP-9714.1.patch
          15 kB
          Douma Fang

          Issue Links

            Activity

              People

              • Assignee:
                xifang Douma Fang
                Reporter:
                xifang Douma Fang
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: