Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12438

Rename dfs.datanode.ec.reconstruction.stripedblock.threads.size to dfs.datanode.ec.reconstruction.threads

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha3
    • Fix Version/s: 3.0.0-beta1
    • Component/s: None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Hide
      <!-- markdown -->

      Config key `dfs.datanode.ec.reconstruction.stripedblock.threads.size` has been renamed to `dfs.datanode.ec.reconstruction.threads`.
      Show
      <!-- markdown --> Config key `dfs.datanode.ec.reconstruction.stripedblock.threads.size` has been renamed to `dfs.datanode.ec.reconstruction.threads`.

      Description

      We should rename this config key to match other config keys used to size thread pools.

      1. HDFS-12438.001.patch
        3 kB
        Andrew Wang
      2. HDFS-12438.002.patch
        5 kB
        Andrew Wang

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12870 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12870/)
        HDFS-12438. Rename (kai.zheng: rev e12f3e85bde0e7e83142b383a45c4ea945dfd64e)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/erasurecode/ErasureCodingWorker.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReconstructStripedFile.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12870 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12870/ ) HDFS-12438 . Rename (kai.zheng: rev e12f3e85bde0e7e83142b383a45c4ea945dfd64e) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/erasurecode/ErasureCodingWorker.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReconstructStripedFile.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md
        Hide
        drankye Kai Zheng added a comment -

        Committed to trunk and branch-3.0. Thanks Andrew Wang for the contribution and Lei (Eddy) Xu for the review!

        Show
        drankye Kai Zheng added a comment - Committed to trunk and branch-3.0. Thanks Andrew Wang for the contribution and Lei (Eddy) Xu for the review!
        Hide
        drankye Kai Zheng added a comment -

        Thanks Andrew for the update. +1 on LGTM. Will commit it shortly.

        Show
        drankye Kai Zheng added a comment - Thanks Andrew for the update. +1 on LGTM. Will commit it shortly.
        Hide
        andrew.wang Andrew Wang added a comment -

        Checkstyle is not applicable since it's in DFSConfigKeys. Someone mind doing another quick review?

        Show
        andrew.wang Andrew Wang added a comment - Checkstyle is not applicable since it's in DFSConfigKeys. Someone mind doing another quick review?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
              trunk Compile Tests
        +1 mvninstall 15m 43s trunk passed
        +1 compile 0m 52s trunk passed
        +1 checkstyle 0m 46s trunk passed
        +1 mvnsite 1m 8s trunk passed
        +1 findbugs 1m 42s trunk passed
        +1 javadoc 0m 40s trunk passed
              Patch Compile Tests
        +1 mvninstall 0m 49s the patch passed
        +1 compile 0m 45s the patch passed
        +1 javac 0m 45s the patch passed
        -0 checkstyle 0m 39s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 421 unchanged - 2 fixed = 422 total (was 423)
        +1 mvnsite 0m 51s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 1m 46s the patch passed
        +1 javadoc 0m 40s the patch passed
              Other Tests
        -1 unit 81m 57s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        110m 5s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup
          hadoop.hdfs.TestAppendDifferentChecksum
          hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
          hadoop.hdfs.TestFileAppend2
          hadoop.hdfs.TestLeaseRecoveryStriped
          hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
        Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:71bbb86
        JIRA Issue HDFS-12438
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886920/HDFS-12438.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux f98243b2ee29 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 5324388
        Default Java 1.8.0_144
        findbugs v3.1.0-RC1
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21123/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/21123/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21123/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21123/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests +1 mvninstall 15m 43s trunk passed +1 compile 0m 52s trunk passed +1 checkstyle 0m 46s trunk passed +1 mvnsite 1m 8s trunk passed +1 findbugs 1m 42s trunk passed +1 javadoc 0m 40s trunk passed       Patch Compile Tests +1 mvninstall 0m 49s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed -0 checkstyle 0m 39s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 421 unchanged - 2 fixed = 422 total (was 423) +1 mvnsite 0m 51s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 40s the patch passed       Other Tests -1 unit 81m 57s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 110m 5s Reason Tests Failed junit tests hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup   hadoop.hdfs.TestAppendDifferentChecksum   hadoop.hdfs.server.blockmanagement.TestReplicationPolicy   hadoop.hdfs.TestFileAppend2   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HDFS-12438 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886920/HDFS-12438.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux f98243b2ee29 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5324388 Default Java 1.8.0_144 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21123/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/21123/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21123/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21123/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Thanks for taking a look Kai! Sure, I like that even better. Renamed again.

        Show
        andrew.wang Andrew Wang added a comment - Thanks for taking a look Kai! Sure, I like that even better. Renamed again.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
              trunk Compile Tests
        +1 mvninstall 18m 31s trunk passed
        +1 compile 0m 58s trunk passed
        +1 checkstyle 0m 45s trunk passed
        +1 mvnsite 1m 6s trunk passed
        +1 findbugs 1m 59s trunk passed
        +1 javadoc 0m 47s trunk passed
              Patch Compile Tests
        +1 mvninstall 1m 0s the patch passed
        +1 compile 0m 56s the patch passed
        +1 javac 0m 56s the patch passed
        -0 checkstyle 0m 42s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 416 unchanged - 1 fixed = 417 total (was 417)
        +1 mvnsite 1m 1s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 2m 9s the patch passed
        +1 javadoc 0m 41s the patch passed
              Other Tests
        -1 unit 92m 54s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        125m 23s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestClientProtocolForPipelineRecovery
          hadoop.hdfs.TestDFSShellGenericOptions
          hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
          hadoop.hdfs.TestEncryptionZones
          hadoop.hdfs.TestDFSStripedOutputStream
          hadoop.hdfs.TestLeaseRecoveryStriped
          hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
          hadoop.hdfs.TestReconstructStripedFile
        Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:71bbb86
        JIRA Issue HDFS-12438
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886763/HDFS-12438.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 74d3419e5784 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / f4b6267
        Default Java 1.8.0_144
        findbugs v3.1.0-RC1
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21109/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/21109/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21109/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21109/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       trunk Compile Tests +1 mvninstall 18m 31s trunk passed +1 compile 0m 58s trunk passed +1 checkstyle 0m 45s trunk passed +1 mvnsite 1m 6s trunk passed +1 findbugs 1m 59s trunk passed +1 javadoc 0m 47s trunk passed       Patch Compile Tests +1 mvninstall 1m 0s the patch passed +1 compile 0m 56s the patch passed +1 javac 0m 56s the patch passed -0 checkstyle 0m 42s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 416 unchanged - 1 fixed = 417 total (was 417) +1 mvnsite 1m 1s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 9s the patch passed +1 javadoc 0m 41s the patch passed       Other Tests -1 unit 92m 54s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 125m 23s Reason Tests Failed junit tests hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.TestDFSShellGenericOptions   hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.TestDFSStripedOutputStream   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks   hadoop.hdfs.TestReconstructStripedFile Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HDFS-12438 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886763/HDFS-12438.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 74d3419e5784 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f4b6267 Default Java 1.8.0_144 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21109/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/21109/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21109/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21109/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        drankye Kai Zheng added a comment -

        Sorry for the late. Maybe we could change it further, ending with dfs.datanode.ec.reconstruction.threads?

        Show
        drankye Kai Zheng added a comment - Sorry for the late. Maybe we could change it further, ending with dfs.datanode.ec.reconstruction.threads ?
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        +1 pending jenkins.

        Thanks for taking care of this, Andrew Wang

        Show
        eddyxu Lei (Eddy) Xu added a comment - +1 pending jenkins. Thanks for taking care of this, Andrew Wang
        Hide
        andrew.wang Andrew Wang added a comment -

        Hi Lei (Eddy) Xu do you mind doing a quick review?

        Show
        andrew.wang Andrew Wang added a comment - Hi Lei (Eddy) Xu do you mind doing a quick review?

          People

          • Assignee:
            andrew.wang Andrew Wang
            Reporter:
            andrew.wang Andrew Wang
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development