Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10285 Storage Policy Satisfier in HDFS
  3. HDFS-11293

[SPS]: Local DN should be given preference as source node, when target available in same node

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: HDFS-10285
    • Fix Version/s: HDFS-10285, 3.2.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In FsDatasetImpl#createTemporary, we use volumeMap to get replica info by block pool id. But in this situation:

      datanode A => {DISK, SSD}, datanode B => {DISK, ARCHIVE}.
      1. the same block replica exists in A[DISK] and B[DISK].
      2. the block pool id of datanode A and datanode B are the same.
      

      Then we start to change the file's storage policy and move the block replica in the cluster. Very likely we have to move block from B[DISK] to A[SSD], at this time, datanode A throws ReplicaAlreadyExistsException.

        Attachments

        1. HDFS-11293-HDFS-10285-00.patch
          5 kB
          Uma Maheswara Rao G
        2. HDFS-11293-HDFS-10285-01.patch
          9 kB
          Uma Maheswara Rao G
        3. HDFS-11293-HDFS-10285-02.patch
          9 kB
          Uma Maheswara Rao G

          Issue Links

            Activity

              People

              • Assignee:
                umamaheswararao Uma Maheswara Rao G
                Reporter:
                yuanbo Yuanbo Liu
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: