Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12438

Rename dfs.datanode.ec.reconstruction.stripedblock.threads.size to dfs.datanode.ec.reconstruction.threads

        Details

        • Type: Improvement
        • Status: Resolved
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: 3.0.0-alpha3
        • Fix Version/s: 3.0.0-beta1
        • Component/s: None
        • Target Version/s:
        • Hadoop Flags:
          Incompatible change, Reviewed
        • Release Note:
          Hide
          <!-- markdown -->

          Config key `dfs.datanode.ec.reconstruction.stripedblock.threads.size` has been renamed to `dfs.datanode.ec.reconstruction.threads`.
          Show
          <!-- markdown --> Config key `dfs.datanode.ec.reconstruction.stripedblock.threads.size` has been renamed to `dfs.datanode.ec.reconstruction.threads`.

          Description

          We should rename this config key to match other config keys used to size thread pools.

          1. Text File
            HDFS-12438.001.patch
            3 kB
            Andrew Wang
          2. Text File
            HDFS-12438.002.patch
            5 kB
            Andrew Wang

            Activity

            Descending order - Click to sort in ascending order
            Hide
            Permalink
            hudson Hudson added a comment -

            SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12870 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12870/)
            HDFS-12438. Rename (kai.zheng: rev e12f3e85bde0e7e83142b383a45c4ea945dfd64e)

            • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
            • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/erasurecode/ErasureCodingWorker.java
            • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
            • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReconstructStripedFile.java
            • (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md
            Show
            hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12870 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12870/ ) HDFS-12438 . Rename (kai.zheng: rev e12f3e85bde0e7e83142b383a45c4ea945dfd64e) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/erasurecode/ErasureCodingWorker.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReconstructStripedFile.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md
            Hide
            Permalink
            drankye Kai Zheng added a comment -

            Committed to trunk and branch-3.0. Thanks Andrew Wang for the contribution and Lei (Eddy) Xu for the review!

            Show
            drankye Kai Zheng added a comment - Committed to trunk and branch-3.0. Thanks Andrew Wang for the contribution and Lei (Eddy) Xu for the review!
            Hide
            Permalink
            drankye Kai Zheng added a comment -

            Thanks Andrew for the update. +1 on LGTM. Will commit it shortly.

            Show
            drankye Kai Zheng added a comment - Thanks Andrew for the update. +1 on LGTM. Will commit it shortly.
            Hide
            Permalink
            andrew.wang Andrew Wang added a comment -

            Checkstyle is not applicable since it's in DFSConfigKeys. Someone mind doing another quick review?

            Show
            andrew.wang Andrew Wang added a comment - Checkstyle is not applicable since it's in DFSConfigKeys. Someone mind doing another quick review?
            Hide
            Permalink
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 12s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                  trunk Compile Tests
            +1 mvninstall 15m 43s trunk passed
            +1 compile 0m 52s trunk passed
            +1 checkstyle 0m 46s trunk passed
            +1 mvnsite 1m 8s trunk passed
            +1 findbugs 1m 42s trunk passed
            +1 javadoc 0m 40s trunk passed
                  Patch Compile Tests
            +1 mvninstall 0m 49s the patch passed
            +1 compile 0m 45s the patch passed
            +1 javac 0m 45s the patch passed
            -0 checkstyle 0m 39s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 421 unchanged - 2 fixed = 422 total (was 423)
            +1 mvnsite 0m 51s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 xml 0m 1s The patch has no ill-formed XML file.
            +1 findbugs 1m 46s the patch passed
            +1 javadoc 0m 40s the patch passed
                  Other Tests
            -1 unit 81m 57s hadoop-hdfs in the patch failed.
            +1 asflicense 0m 16s The patch does not generate ASF License warnings.
            110m 5s



            Reason Tests
            Failed junit tests hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup
              hadoop.hdfs.TestAppendDifferentChecksum
              hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
              hadoop.hdfs.TestFileAppend2
              hadoop.hdfs.TestLeaseRecoveryStriped
              hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
            Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



            Subsystem Report/Notes
            Docker Image:yetus/hadoop:71bbb86
            JIRA Issue HDFS-12438
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886920/HDFS-12438.002.patch
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
            uname Linux f98243b2ee29 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
            git revision trunk / 5324388
            Default Java 1.8.0_144
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21123/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
            unit https://builds.apache.org/job/PreCommit-HDFS-Build/21123/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
            Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21123/testReport/
            modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
            Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21123/console
            Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            Show
            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests +1 mvninstall 15m 43s trunk passed +1 compile 0m 52s trunk passed +1 checkstyle 0m 46s trunk passed +1 mvnsite 1m 8s trunk passed +1 findbugs 1m 42s trunk passed +1 javadoc 0m 40s trunk passed       Patch Compile Tests +1 mvninstall 0m 49s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed -0 checkstyle 0m 39s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 421 unchanged - 2 fixed = 422 total (was 423) +1 mvnsite 0m 51s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 40s the patch passed       Other Tests -1 unit 81m 57s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 110m 5s Reason Tests Failed junit tests hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup   hadoop.hdfs.TestAppendDifferentChecksum   hadoop.hdfs.server.blockmanagement.TestReplicationPolicy   hadoop.hdfs.TestFileAppend2   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HDFS-12438 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886920/HDFS-12438.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux f98243b2ee29 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5324388 Default Java 1.8.0_144 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21123/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/21123/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21123/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21123/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            Hide
            Permalink
            andrew.wang Andrew Wang added a comment -

            Thanks for taking a look Kai! Sure, I like that even better. Renamed again.

            Show
            andrew.wang Andrew Wang added a comment - Thanks for taking a look Kai! Sure, I like that even better. Renamed again.
            Hide
            Permalink
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 15s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                  trunk Compile Tests
            +1 mvninstall 18m 31s trunk passed
            +1 compile 0m 58s trunk passed
            +1 checkstyle 0m 45s trunk passed
            +1 mvnsite 1m 6s trunk passed
            +1 findbugs 1m 59s trunk passed
            +1 javadoc 0m 47s trunk passed
                  Patch Compile Tests
            +1 mvninstall 1m 0s the patch passed
            +1 compile 0m 56s the patch passed
            +1 javac 0m 56s the patch passed
            -0 checkstyle 0m 42s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 416 unchanged - 1 fixed = 417 total (was 417)
            +1 mvnsite 1m 1s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 xml 0m 1s The patch has no ill-formed XML file.
            +1 findbugs 2m 9s the patch passed
            +1 javadoc 0m 41s the patch passed
                  Other Tests
            -1 unit 92m 54s hadoop-hdfs in the patch failed.
            +1 asflicense 0m 16s The patch does not generate ASF License warnings.
            125m 23s



            Reason Tests
            Failed junit tests hadoop.hdfs.TestClientProtocolForPipelineRecovery
              hadoop.hdfs.TestDFSShellGenericOptions
              hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
              hadoop.hdfs.TestEncryptionZones
              hadoop.hdfs.TestDFSStripedOutputStream
              hadoop.hdfs.TestLeaseRecoveryStriped
              hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
              hadoop.hdfs.TestReconstructStripedFile
            Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



            Subsystem Report/Notes
            Docker Image:yetus/hadoop:71bbb86
            JIRA Issue HDFS-12438
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886763/HDFS-12438.001.patch
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
            uname Linux 74d3419e5784 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
            git revision trunk / f4b6267
            Default Java 1.8.0_144
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21109/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
            unit https://builds.apache.org/job/PreCommit-HDFS-Build/21109/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
            Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21109/testReport/
            modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
            Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21109/console
            Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            Show
            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       trunk Compile Tests +1 mvninstall 18m 31s trunk passed +1 compile 0m 58s trunk passed +1 checkstyle 0m 45s trunk passed +1 mvnsite 1m 6s trunk passed +1 findbugs 1m 59s trunk passed +1 javadoc 0m 47s trunk passed       Patch Compile Tests +1 mvninstall 1m 0s the patch passed +1 compile 0m 56s the patch passed +1 javac 0m 56s the patch passed -0 checkstyle 0m 42s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 416 unchanged - 1 fixed = 417 total (was 417) +1 mvnsite 1m 1s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 9s the patch passed +1 javadoc 0m 41s the patch passed       Other Tests -1 unit 92m 54s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 125m 23s Reason Tests Failed junit tests hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.TestDFSShellGenericOptions   hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.TestDFSStripedOutputStream   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks   hadoop.hdfs.TestReconstructStripedFile Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HDFS-12438 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886763/HDFS-12438.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 74d3419e5784 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f4b6267 Default Java 1.8.0_144 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21109/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/21109/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21109/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21109/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            Hide
            Permalink
            drankye Kai Zheng added a comment -

            Sorry for the late. Maybe we could change it further, ending with dfs.datanode.ec.reconstruction.threads?

            Show
            drankye Kai Zheng added a comment - Sorry for the late. Maybe we could change it further, ending with dfs.datanode.ec.reconstruction.threads ?
            Hide
            Permalink
            eddyxu Lei (Eddy) Xu added a comment -

            +1 pending jenkins.

            Thanks for taking care of this, Andrew Wang

            Show
            eddyxu Lei (Eddy) Xu added a comment - +1 pending jenkins. Thanks for taking care of this, Andrew Wang
            Hide
            Permalink
            andrew.wang Andrew Wang added a comment -

            Hi Lei (Eddy) Xu do you mind doing a quick review?

            Show
            andrew.wang Andrew Wang added a comment - Hi Lei (Eddy) Xu do you mind doing a quick review?

              People

              • Assignee:
                andrew.wang Andrew Wang
                Reporter:
                andrew.wang Andrew Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development