Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12085

Reconfigure namenode heartbeat interval fails if the interval was set with time unit

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha4
    • Fix Version/s: 3.0.0-beta1
    • Component/s: hdfs, tools
    • Labels:
      None

      Description

      It fails when I set duration with time unit, e.g 5s, error

      Reconfiguring status for node [localhost:8111]: started at Tue Jul 04 08:14:18 PDT 2017 and finished at Tue Jul 04 08:14:18 PDT 2017.
      FAILED: Change property dfs.heartbeat.interval
      	From: "3s"
      	To: "5s"
      	Error: For input string: "5s".
      

      time unit support was added via HDFS-9847.

      1. HDFS-12085.001.patch
        3 kB
        Weiwei Yang
      2. HDFS-12085.002.patch
        4 kB
        Weiwei Yang

        Issue Links

          Activity

          Hide
          cheersyang Weiwei Yang added a comment -

          Thanks for the help Yiqun Lin.

          Show
          cheersyang Weiwei Yang added a comment - Thanks for the help Yiqun Lin .
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11987 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11987/)
          HDFS-12085. Reconfigure namenode heartbeat interval fails if the (yqlin: rev 3a7f02b81520ad4d3eebf92e9dbca662beec0302)

          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeReconfigure.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11987 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11987/ ) HDFS-12085 . Reconfigure namenode heartbeat interval fails if the (yqlin: rev 3a7f02b81520ad4d3eebf92e9dbca662beec0302) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeReconfigure.java
          Hide
          linyiqun Yiqun Lin added a comment -

          Test failures are not related. Committed to trunk. Thanks Weiwei Yang for the contribution!

          Show
          linyiqun Yiqun Lin added a comment - Test failures are not related. Committed to trunk. Thanks Weiwei Yang for the contribution!
          Hide
          cheersyang Weiwei Yang added a comment -

          Sure, thanks Yiqun Lin. I thought this was a bug impacting basic user experience so triage it to a critical bug. Thanks for correcting it, I agree the change is small.

          Show
          cheersyang Weiwei Yang added a comment - Sure, thanks Yiqun Lin . I thought this was a bug impacting basic user experience so triage it to a critical bug. Thanks for correcting it, I agree the change is small.
          Hide
          linyiqun Yiqun Lin added a comment -

          LGTM, +1. Will commit tomorrow in case others have further comments on this.
          I change the priority of this JIRA since this seems a minor change not a critical bug. Thanks Weiwei Yang.

          Show
          linyiqun Yiqun Lin added a comment - LGTM, +1. Will commit tomorrow in case others have further comments on this. I change the priority of this JIRA since this seems a minor change not a critical bug. Thanks Weiwei Yang .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                trunk Compile Tests
          0 mvndep 1m 22s Maven dependency ordering for branch
          +1 mvninstall 13m 10s trunk passed
          +1 compile 13m 31s trunk passed
          +1 checkstyle 1m 52s trunk passed
          +1 mvnsite 2m 26s trunk passed
          -1 findbugs 1m 47s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings.
          +1 javadoc 1m 37s trunk passed
                Patch Compile Tests
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 27s the patch passed
          +1 compile 10m 13s the patch passed
          +1 javac 10m 13s the patch passed
          +1 checkstyle 1m 56s the patch passed
          +1 mvnsite 2m 24s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 25s the patch passed
          +1 javadoc 1m 36s the patch passed
                Other Tests
          +1 unit 7m 46s hadoop-common in the patch passed.
          -1 unit 64m 39s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 35s The patch does not generate ASF License warnings.
          132m 47s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.ha.TestHASafeMode
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-12085
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12876359/HDFS-12085.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux d09ac09a9745 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3de47ab
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20207/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/20207/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20207/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20207/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests 0 mvndep 1m 22s Maven dependency ordering for branch +1 mvninstall 13m 10s trunk passed +1 compile 13m 31s trunk passed +1 checkstyle 1m 52s trunk passed +1 mvnsite 2m 26s trunk passed -1 findbugs 1m 47s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. +1 javadoc 1m 37s trunk passed       Patch Compile Tests 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 27s the patch passed +1 compile 10m 13s the patch passed +1 javac 10m 13s the patch passed +1 checkstyle 1m 56s the patch passed +1 mvnsite 2m 24s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 25s the patch passed +1 javadoc 1m 36s the patch passed       Other Tests +1 unit 7m 46s hadoop-common in the patch passed. -1 unit 64m 39s hadoop-hdfs in the patch failed. +1 asflicense 0m 35s The patch does not generate ASF License warnings. 132m 47s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.ha.TestHASafeMode   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140 Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-12085 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12876359/HDFS-12085.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d09ac09a9745 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3de47ab Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20207/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/20207/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20207/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20207/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cheersyang Weiwei Yang added a comment -

          Thank you Yiqun Lin for the review and comments, I just uploaded v2 patch to address them. Thanks a lot.

          Show
          cheersyang Weiwei Yang added a comment - Thank you Yiqun Lin for the review and comments, I just uploaded v2 patch to address them. Thanks a lot.
          Hide
          linyiqun Yiqun Lin added a comment -

          Weiwei Yang, the patch almost looks good to me. One minor comment: I think we can just change private long getTimeDurationHelper to public long getTimeDurationHelper. I didn't see any difference between getTimeDurationHelper and new added method getTimeDurationFromValue. Also please add the comment for this method that will be better understand for callers, like the following:

            /**
             * Return time duration in the given time unit. Valid units are encoded in
             * properties as suffixes: nanoseconds (ns), microseconds (us), milliseconds
             * (ms), seconds (s), minutes (m), hours (h), and days (d).
             * @param name Property name
             * @param vStr The string value with time unit suffix to be converted.
             * @param unit Unit to convert the stored property, if it exists.
             */
             public long getTimeDurationHelper(String name, String vStr, TimeUnit unit) {
          
          Show
          linyiqun Yiqun Lin added a comment - Weiwei Yang , the patch almost looks good to me. One minor comment: I think we can just change private long getTimeDurationHelper to public long getTimeDurationHelper . I didn't see any difference between getTimeDurationHelper and new added method getTimeDurationFromValue . Also please add the comment for this method that will be better understand for callers, like the following: /** * Return time duration in the given time unit. Valid units are encoded in * properties as suffixes: nanoseconds (ns), microseconds (us), milliseconds * (ms), seconds (s), minutes (m), hours (h), and days (d). * @param name Property name * @param vStr The string value with time unit suffix to be converted. * @param unit Unit to convert the stored property, if it exists. */ public long getTimeDurationHelper( String name, String vStr, TimeUnit unit) {
          Hide
          cheersyang Weiwei Yang added a comment -

          Looks like Xiaobing Zhou is not around .
          Ping Yiqun Lin, since you contributed HDFS-9847, can you please help to review this one?
          Appreciate, thank you.

          Show
          cheersyang Weiwei Yang added a comment - Looks like Xiaobing Zhou is not around . Ping Yiqun Lin , since you contributed HDFS-9847 , can you please help to review this one? Appreciate, thank you.
          Hide
          cheersyang Weiwei Yang added a comment - - edited

          Hi Xiaobing Zhou Can you please take a look at this issue as it is related to HDFS-1477? Thanks a lot.

          Show
          cheersyang Weiwei Yang added a comment - - edited Hi Xiaobing Zhou Can you please take a look at this issue as it is related to HDFS-1477 ? Thanks a lot.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 1m 37s Maven dependency ordering for branch
          +1 mvninstall 14m 15s trunk passed
          +1 compile 14m 2s trunk passed
          +1 checkstyle 2m 0s trunk passed
          +1 mvnsite 2m 13s trunk passed
          -1 findbugs 1m 54s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings.
          +1 javadoc 1m 38s trunk passed
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 33s the patch passed
          +1 compile 10m 40s the patch passed
          +1 javac 10m 40s the patch passed
          +1 checkstyle 1m 58s the patch passed
          +1 mvnsite 2m 5s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 33s the patch passed
          +1 javadoc 1m 35s the patch passed
          +1 unit 7m 40s hadoop-common in the patch passed.
          -1 unit 69m 27s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 35s The patch does not generate ASF License warnings.
          140m 1s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestMaintenanceState
            hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080
          Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-12085
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12875659/HDFS-12085.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 247e005c0c86 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / b17e655
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20154/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/20154/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20154/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20154/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 1m 37s Maven dependency ordering for branch +1 mvninstall 14m 15s trunk passed +1 compile 14m 2s trunk passed +1 checkstyle 2m 0s trunk passed +1 mvnsite 2m 13s trunk passed -1 findbugs 1m 54s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. +1 javadoc 1m 38s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 33s the patch passed +1 compile 10m 40s the patch passed +1 javac 10m 40s the patch passed +1 checkstyle 1m 58s the patch passed +1 mvnsite 2m 5s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 33s the patch passed +1 javadoc 1m 35s the patch passed +1 unit 7m 40s hadoop-common in the patch passed. -1 unit 69m 27s hadoop-hdfs in the patch failed. +1 asflicense 0m 35s The patch does not generate ASF License warnings. 140m 1s Reason Tests Failed junit tests hadoop.hdfs.TestMaintenanceState   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-12085 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12875659/HDFS-12085.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 247e005c0c86 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b17e655 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20154/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/20154/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20154/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20154/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

            People

            • Assignee:
              cheersyang Weiwei Yang
              Reporter:
              cheersyang Weiwei Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development