Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10266

Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: None
    • Labels:
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The properties:

      dfs.client.file-block-storage-locations.num-threads
      dfs.client.file-block-storage-locations.timeout.millis

      exist in DfsConfigKeys and HdfsClientConfigKeys but nowhere else. It should be safe to remove them.

        Issue Links

          Activity

          Hide
          andrew.wang Andrew Wang added a comment -

          Hi Ray, these keys also used in DfsClientConf, and the config keys themselves are still valid. Should we resolve as INVALID?

          Show
          andrew.wang Andrew Wang added a comment - Hi Ray, these keys also used in DfsClientConf, and the config keys themselves are still valid. Should we resolve as INVALID?
          Hide
          rchiang Ray Chiang added a comment -

          Are you sure? Using grep and IntelliJ, I can find the definitions, but I'm not actually seeing them used anywhere, including DfsClientConf.java. Admittedly, I'm not that familiar with the code, so I could be missing something.

          I only found references in these four files:

          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/HdfsConfiguration.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json
          Show
          rchiang Ray Chiang added a comment - Are you sure? Using grep and IntelliJ, I can find the definitions, but I'm not actually seeing them used anywhere, including DfsClientConf.java. Admittedly, I'm not that familiar with the code, so I could be missing something. I only found references in these four files: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/HdfsConfiguration.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json
          Hide
          andrew.wang Andrew Wang added a comment -

          Ah, you're talking about in trunk, not branch-2. I'll change the affects version.

          I removed this code in trunk, guess I missed those two config keys

          Show
          andrew.wang Andrew Wang added a comment - Ah, you're talking about in trunk, not branch-2. I'll change the affects version. I removed this code in trunk, guess I missed those two config keys
          Hide
          rchiang Ray Chiang added a comment -

          Remove all references to the two properties.

          Show
          rchiang Ray Chiang added a comment - Remove all references to the two properties.
          Hide
          andrew.wang Andrew Wang added a comment -

          LGTM +1 pending, thanks Ray

          Show
          andrew.wang Andrew Wang added a comment - LGTM +1 pending, thanks Ray
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 16m 48s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 20s Maven dependency ordering for branch
          +1 mvninstall 8m 7s trunk passed
          +1 compile 9m 14s trunk passed with JDK v1.8.0_77
          +1 compile 10m 37s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 36s trunk passed
          +1 mvnsite 2m 26s trunk passed
          +1 mvneclipse 0m 52s trunk passed
          +1 findbugs 6m 17s trunk passed
          +1 javadoc 2m 37s trunk passed with JDK v1.8.0_77
          +1 javadoc 2m 53s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 1m 47s the patch passed
          +1 compile 9m 0s the patch passed with JDK v1.8.0_77
          +1 javac 9m 0s the patch passed
          +1 compile 8m 12s the patch passed with JDK v1.7.0_95
          +1 javac 8m 12s the patch passed
          +1 checkstyle 1m 18s the patch passed
          +1 mvnsite 2m 5s the patch passed
          +1 mvneclipse 0m 46s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 5m 35s the patch passed
          +1 javadoc 2m 15s the patch passed with JDK v1.8.0_77
          +1 javadoc 4m 18s the patch passed with JDK v1.7.0_95
          +1 unit 1m 44s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77.
          -1 unit 93m 4s hadoop-hdfs in the patch failed with JDK v1.8.0_77.
          +1 unit 1m 12s hadoop-sls in the patch passed with JDK v1.8.0_77.
          +1 unit 1m 6s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          -1 unit 82m 54s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 unit 0m 59s hadoop-sls in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 27s Patch does not generate ASF License warnings.
          282m 10s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.TestRollingUpgrade
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.server.datanode.TestDataNodeMXBean
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
            hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797608/HDFS-10266.001.patch
          JIRA Issue HDFS-10266
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux a5826e54c6c8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a62637a
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15107/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15107/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15107/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15107/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15107/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-sls U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15107/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 16m 48s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 20s Maven dependency ordering for branch +1 mvninstall 8m 7s trunk passed +1 compile 9m 14s trunk passed with JDK v1.8.0_77 +1 compile 10m 37s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 36s trunk passed +1 mvnsite 2m 26s trunk passed +1 mvneclipse 0m 52s trunk passed +1 findbugs 6m 17s trunk passed +1 javadoc 2m 37s trunk passed with JDK v1.8.0_77 +1 javadoc 2m 53s trunk passed with JDK v1.7.0_95 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 47s the patch passed +1 compile 9m 0s the patch passed with JDK v1.8.0_77 +1 javac 9m 0s the patch passed +1 compile 8m 12s the patch passed with JDK v1.7.0_95 +1 javac 8m 12s the patch passed +1 checkstyle 1m 18s the patch passed +1 mvnsite 2m 5s the patch passed +1 mvneclipse 0m 46s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 5m 35s the patch passed +1 javadoc 2m 15s the patch passed with JDK v1.8.0_77 +1 javadoc 4m 18s the patch passed with JDK v1.7.0_95 +1 unit 1m 44s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77. -1 unit 93m 4s hadoop-hdfs in the patch failed with JDK v1.8.0_77. +1 unit 1m 12s hadoop-sls in the patch passed with JDK v1.8.0_77. +1 unit 1m 6s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. -1 unit 82m 54s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 unit 0m 59s hadoop-sls in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 27s Patch does not generate ASF License warnings. 282m 10s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.TestFileAppend   hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.server.datanode.TestDataNodeMXBean   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.hdfs.server.namenode.TestEditLog   hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797608/HDFS-10266.001.patch JIRA Issue HDFS-10266 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a5826e54c6c8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a62637a Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/15107/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15107/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15107/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15107/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15107/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-sls U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15107/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          +1, the test failures are unrelated. I ran all the failed tests locally and they passed.

          Show
          ajisakaa Akira Ajisaka added a comment - +1, the test failures are unrelated. I ran all the failed tests locally and they passed.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed this to trunk. Thanks Ray Chiang for the contribution and thanks Andrew Wang for the review.

          Show
          ajisakaa Akira Ajisaka added a comment - Committed this to trunk. Thanks Ray Chiang for the contribution and thanks Andrew Wang for the review.
          Hide
          andrew.wang Andrew Wang added a comment -

          Beat me to it, I had the commit staged locally Thanks Akira for commit, and of course Ray for the find and fix.

          Show
          andrew.wang Andrew Wang added a comment - Beat me to it, I had the commit staged locally Thanks Akira for commit, and of course Ray for the find and fix.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9577 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9577/)
          HDFS-10266. Remove unused properties (aajisaka: rev 9c32f8785e8d7957e3f8a3946cfd15a4d5c82fec)

          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/HdfsConfiguration.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9577 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9577/ ) HDFS-10266 . Remove unused properties (aajisaka: rev 9c32f8785e8d7957e3f8a3946cfd15a4d5c82fec) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/HdfsConfiguration.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java
          Hide
          rchiang Ray Chiang added a comment -

          Thanks for the feedback guys and thanks for the commit Akira Ajisaka.

          Show
          rchiang Ray Chiang added a comment - Thanks for the feedback guys and thanks for the commit Akira Ajisaka .

            People

            • Assignee:
              rchiang Ray Chiang
              Reporter:
              rchiang Ray Chiang
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development