Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11628

Clarify the behavior of HDFS Mover in documentation

    Details

    • Hadoop Flags:
      Reviewed

      Description

      It's helpful to state that Mover always tries to move block replicas within the same node whenever possible. If that is not possible (e.g. when a node doesn’t have the target storage type) then it will copy the block replica to another node over the network.

      1. HDFS-11628.001.patch
        2 kB
        Xiaobing Zhou
      2. HDFS-11628.000.patch
        2 kB
        Xiaobing Zhou

        Activity

        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Posted a patch. Will have a dry run to visualize the changes for sanity check.

        Show
        xiaobingo Xiaobing Zhou added a comment - Posted a patch. Will have a dry run to visualize the changes for sanity check.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 28s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvninstall 16m 4s trunk passed
        +1 mvnsite 0m 54s trunk passed
        +1 mvnsite 0m 52s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        18m 51s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11628
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862131/HDFS-11628.000.patch
        Optional Tests asflicense mvnsite
        uname Linux 09085e0fb74b 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11 16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e8071aa
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18987/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 28s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 16m 4s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvnsite 0m 52s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 18m 51s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11628 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862131/HDFS-11628.000.patch Optional Tests asflicense mvnsite uname Linux 09085e0fb74b 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11 16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e8071aa modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18987/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        liuml07 Mingliang Liu added a comment - - edited

        +1. Akira Ajisaka do you have 2nd opinion?

        Show
        liuml07 Mingliang Liu added a comment - - edited +1. Akira Ajisaka do you have 2nd opinion?
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        +1 for the patch. A nitpick, can you please remove the extra linebreak here?

        .. Note that
        +it always tries to move block...
        
        Show
        arpitagarwal Arpit Agarwal added a comment - +1 for the patch. A nitpick, can you please remove the extra linebreak here? .. Note that +it always tries to move block...
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        v1 fixed that. Thanks Mingliang Liu and Arpit Agarwal.

        Show
        xiaobingo Xiaobing Zhou added a comment - v1 fixed that. Thanks Mingliang Liu and Arpit Agarwal .
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        +1 pending Jenkins. Thanks Xiaobing Zhou.

        Show
        arpitagarwal Arpit Agarwal added a comment - +1 pending Jenkins. Thanks Xiaobing Zhou .
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvninstall 13m 50s trunk passed
        +1 mvnsite 1m 2s trunk passed
        +1 mvnsite 1m 1s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        16m 51s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11628
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862178/HDFS-11628.001.patch
        Optional Tests asflicense mvnsite
        uname Linux dcac54cf2919 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11 16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 7d963c4
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18990/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 13m 50s trunk passed +1 mvnsite 1m 2s trunk passed +1 mvnsite 1m 1s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 16m 51s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11628 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862178/HDFS-11628.001.patch Optional Tests asflicense mvnsite uname Linux dcac54cf2919 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11 16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7d963c4 modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18990/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Thanks for the contribution Xiaobing Zhou. I've committed this.

        Thanks for the review Mingliang Liu.

        Show
        arpitagarwal Arpit Agarwal added a comment - Thanks for the contribution Xiaobing Zhou . I've committed this. Thanks for the review Mingliang Liu .
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11534 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11534/)
        HDFS-11628. Clarify the behavior of HDFS Mover in documentation. (arp: rev 3db8d68d63f832d1747efcdf079e02ecda0b0127)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ArchivalStorage.md
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11534 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11534/ ) HDFS-11628 . Clarify the behavior of HDFS Mover in documentation. (arp: rev 3db8d68d63f832d1747efcdf079e02ecda0b0127) (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ArchivalStorage.md
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11591 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11591/)
        HDFS-11628. Clarify the behavior of HDFS Mover in documentation. (arp: rev 3db8d68d63f832d1747efcdf079e02ecda0b0127)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ArchivalStorage.md
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11591 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11591/ ) HDFS-11628 . Clarify the behavior of HDFS Mover in documentation. (arp: rev 3db8d68d63f832d1747efcdf079e02ecda0b0127) (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ArchivalStorage.md
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - 2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

          People

          • Assignee:
            xiaobingo Xiaobing Zhou
            Reporter:
            xiaobingo Xiaobing Zhou
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development