Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10895

Update HDFS Erasure Coding doc to add how to use ISA-L based coder

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Recently, native ISA-L library support is added into HDFS erasure code RS codec. With ISA-L, the performance of erasure code RS encode/decode will be significantly improved. This task is to add the deploy process into the HDFS Erasure Code Document, make it easy for HDFS user to use ISA-L enabled erasure code policy.

      1. HDFS-10895-v1.patch
        2 kB
        SammiChen
      2. HDFS-10895-v2.patch
        3 kB
        SammiChen
      3. HDFS-10895-v3.patch
        3 kB
        SammiChen
      4. HDFS-10895-v4.patch
        3 kB
        SammiChen
      5. HDFS-10895-v5.patch
        4 kB
        SammiChen

        Activity

        Hide
        Sammi SammiChen added a comment -

        Initial patch

        Show
        Sammi SammiChen added a comment - Initial patch
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvninstall 7m 30s trunk passed
        +1 mvnsite 0m 50s trunk passed
        +1 mvnsite 0m 49s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        9m 57s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10895
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830660/HDFS-10895-v1.patch
        Optional Tests asflicense mvnsite
        uname Linux 2a4f2a91d98c 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 03f519a
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16904/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 7m 30s trunk passed +1 mvnsite 0m 50s trunk passed +1 mvnsite 0m 49s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 9m 57s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10895 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830660/HDFS-10895-v1.patch Optional Tests asflicense mvnsite uname Linux 2a4f2a91d98c 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 03f519a modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16904/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        Sammi SammiChen added a comment -

        Rebase patch against latest trunk doc

        Show
        Sammi SammiChen added a comment - Rebase patch against latest trunk doc
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvninstall 8m 15s trunk passed
        +1 mvnsite 0m 59s trunk passed
        +1 mvnsite 0m 58s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 asflicense 0m 27s The patch does not generate ASF License warnings.
        11m 14s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10895
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832309/HDFS-10895-v2.patch
        Optional Tests asflicense mvnsite
        uname Linux ffd513e4ed72 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4d10621
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17071/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 8m 15s trunk passed +1 mvnsite 0m 59s trunk passed +1 mvnsite 0m 58s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 11m 14s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10895 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832309/HDFS-10895-v2.patch Optional Tests asflicense mvnsite uname Linux ffd513e4ed72 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4d10621 modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17071/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        Sammi SammiChen added a comment -

        Improve the document by leverage "Build instructions for Hadoop"(BUILDING.txt)

        Show
        Sammi SammiChen added a comment - Improve the document by leverage "Build instructions for Hadoop"(BUILDING.txt)
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvninstall 6m 36s trunk passed
        +1 mvnsite 0m 50s trunk passed
        +1 mvnsite 0m 47s the patch passed
        -1 whitespace 0m 0s The patch has 5 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        9m 1s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10895
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832314/HDFS-10895-v3.patch
        Optional Tests asflicense mvnsite
        uname Linux 66f101a55cce 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4d10621
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17073/artifact/patchprocess/whitespace-eol.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17073/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 6m 36s trunk passed +1 mvnsite 0m 50s trunk passed +1 mvnsite 0m 47s the patch passed -1 whitespace 0m 0s The patch has 5 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 asflicense 0m 15s The patch does not generate ASF License warnings. 9m 1s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10895 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832314/HDFS-10895-v3.patch Optional Tests asflicense mvnsite uname Linux 66f101a55cce 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4d10621 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17073/artifact/patchprocess/whitespace-eol.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17073/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        Sammi SammiChen added a comment -

        remove whitespaces

        Show
        Sammi SammiChen added a comment - remove whitespaces
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvninstall 6m 31s trunk passed
        +1 mvnsite 0m 51s trunk passed
        +1 mvnsite 0m 49s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        8m 56s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10895
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832316/HDFS-10895-v4.patch
        Optional Tests asflicense mvnsite
        uname Linux 2294b5ecd2d5 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4d10621
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17074/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 6m 31s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvnsite 0m 49s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 8m 56s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10895 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832316/HDFS-10895-v4.patch Optional Tests asflicense mvnsite uname Linux 2294b5ecd2d5 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4d10621 modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17074/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        Sammi SammiChen added a comment -

        Improve the patch after offline discussion with Kai.

        Show
        Sammi SammiChen added a comment - Improve the patch after offline discussion with Kai.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvninstall 6m 48s trunk passed
        +1 mvnsite 0m 51s trunk passed
        +1 mvnsite 0m 47s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        9m 14s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10895
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832402/HDFS-10895-v5.patch
        Optional Tests asflicense mvnsite
        uname Linux 0cf8bcbf68c7 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / bea004e
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17077/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 6m 48s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvnsite 0m 47s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 9m 14s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10895 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832402/HDFS-10895-v5.patch Optional Tests asflicense mvnsite uname Linux 0cf8bcbf68c7 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bea004e modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17077/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        drankye Kai Zheng added a comment -

        The latest patch looks good to me. +1

        Show
        drankye Kai Zheng added a comment - The latest patch looks good to me. +1
        Hide
        drankye Kai Zheng added a comment -

        Committed to trunk. Thanks SammiChen for the contribution!

        Show
        drankye Kai Zheng added a comment - Committed to trunk. Thanks SammiChen for the contribution!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10576 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10576/)
        HDFS-10895. Update HDFS Erasure Coding doc to add how to use ISA-L based (kai.zheng: rev af50da3298f92a52cc20d5f6aab6f6ad8134efbd)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10576 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10576/ ) HDFS-10895 . Update HDFS Erasure Coding doc to add how to use ISA-L based (kai.zheng: rev af50da3298f92a52cc20d5f6aab6f6ad8134efbd) (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md

          People

          • Assignee:
            Sammi SammiChen
            Reporter:
            Sammi SammiChen
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development