Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11088

Quash unnecessary safemode WARN message during NameNode startup

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:

      Description

      I tried starting a NameNode on a freshly formatted cluster, and it produced this WARN log:

      16/11/01 16:42:44 WARN blockmanagement.BlockManagerSafeMode: forceExit used when normal exist would suffice. Treating force exit as normal safe mode exit.
      

      I didn't do any special commands related to safemode, so this log should be quashed.

      I didn't try the branch-2's, but it's likely this issue exists there as well.

      1. HDFS-11088.001.patch
        2 kB
        Yiqun Lin
      2. HDFS-11088.002.patch
        1 kB
        Yiqun Lin
      3. HDFS-11088.003.patch
        2 kB
        Yiqun Lin

        Activity

        Hide
        linyiqun Yiqun Lin added a comment -

        Hi Andrew Wang, I have looked into this. I think this WARN log isn't the unnesessary log. The reason why you see this log is that there is one special case that namenode can leave safemode in a force way without input the force option from users. The related codes(BlockManagerSafeMode#activate):

          void activate(long total) {
            assert namesystem.hasWriteLock();
            assert status == BMSafeModeStatus.OFF;
        
            startTime = monotonicNow();
            setBlockTotal(total);
            if (areThresholdsMet()) {
              leaveSafeMode(true);
            }
            ...
        

        I think we can pass the param from true to false here. In addition, I found other twp minors in class BlockManagerSafeMode. I think we can also fix these in this JIRA or file another JIRA to have a fix that anything you like.

        • One typo in WARN log.
          LOG.warn("The threshold value should't be greater than 1," +
          

          Here should't should be shouldn't.

        • The default extension value DFS_NAMENODE_SAFEMODE_EXTENSION_DEFAULT not used here, it just use 0.
          this.extension = conf.getInt(DFS_NAMENODE_SAFEMODE_EXTENSION_KEY, 0);
          

        Attach a patch with these all fixed. Looking forward to seeing your response, Andrew Wang. Thanks!

        Show
        linyiqun Yiqun Lin added a comment - Hi Andrew Wang , I have looked into this. I think this WARN log isn't the unnesessary log. The reason why you see this log is that there is one special case that namenode can leave safemode in a force way without input the force option from users. The related codes( BlockManagerSafeMode#activate ): void activate( long total) { assert namesystem.hasWriteLock(); assert status == BMSafeModeStatus.OFF; startTime = monotonicNow(); setBlockTotal(total); if (areThresholdsMet()) { leaveSafeMode( true ); } ... I think we can pass the param from true to false here. In addition, I found other twp minors in class BlockManagerSafeMode . I think we can also fix these in this JIRA or file another JIRA to have a fix that anything you like. One typo in WARN log. LOG.warn( "The threshold value should't be greater than 1," + Here should't should be shouldn't . The default extension value DFS_NAMENODE_SAFEMODE_EXTENSION_DEFAULT not used here, it just use 0 . this .extension = conf.getInt(DFS_NAMENODE_SAFEMODE_EXTENSION_KEY, 0); Attach a patch with these all fixed. Looking forward to seeing your response, Andrew Wang . Thanks!
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 24s trunk passed
        +1 compile 0m 47s trunk passed
        +1 checkstyle 0m 26s trunk passed
        +1 mvnsite 0m 52s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 43s trunk passed
        +1 javadoc 0m 40s trunk passed
        +1 mvninstall 0m 49s the patch passed
        +1 compile 0m 49s the patch passed
        +1 javac 0m 49s the patch passed
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 0m 50s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 52s the patch passed
        +1 javadoc 0m 38s the patch passed
        +1 unit 56m 5s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        75m 32s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-11088
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836548/HDFS-11088.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux bf0d7b056558 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / cb5cc0d
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17384/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17384/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 24s trunk passed +1 compile 0m 47s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 43s trunk passed +1 javadoc 0m 40s trunk passed +1 mvninstall 0m 49s the patch passed +1 compile 0m 49s the patch passed +1 javac 0m 49s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 52s the patch passed +1 javadoc 0m 38s the patch passed +1 unit 56m 5s hadoop-hdfs in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 75m 32s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-11088 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836548/HDFS-11088.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux bf0d7b056558 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / cb5cc0d Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17384/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17384/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        This looks okay, since checkSafeMode behaves very similarly. It's a bit concerning though that we don't check the return value; I think numberOfBytesInFutureBlocks will always be zero here since no block reports have been received so force doesn't apply, but maybe we should add a Precondition check on the return value for safety.

        Mingliang Liu / Jing Zhao care to comment as the local safemode experts?

        Show
        andrew.wang Andrew Wang added a comment - This looks okay, since checkSafeMode behaves very similarly. It's a bit concerning though that we don't check the return value; I think numberOfBytesInFutureBlocks will always be zero here since no block reports have been received so force doesn't apply, but maybe we should add a Precondition check on the return value for safety. Mingliang Liu / Jing Zhao care to comment as the local safemode experts?
        Hide
        liuml07 Mingliang Liu added a comment -

        The default extension value DFS_NAMENODE_SAFEMODE_EXTENSION_DEFAULT not used here, it just use 0.

        Sorry I didn't notice your comment in this JIRA before I fixed this today. See HDFS-11095.

        Show
        liuml07 Mingliang Liu added a comment - The default extension value DFS_NAMENODE_SAFEMODE_EXTENSION_DEFAULT not used here, it just use 0. Sorry I didn't notice your comment in this JIRA before I fixed this today. See HDFS-11095 .
        Hide
        linyiqun Yiqun Lin added a comment - - edited

        Thanks Andrew Wang for the comments.

        but maybe we should add a Precondition check on the return value for safety.

        +1 for this idea. I think we can file another JIRA to make a change if we think it's necessary to do here. This JIRA will be better to focus on the change as the this JIRA's title mentioned.

        Since HDFS-11095 has fixed one minor that I mentioned in my last comment, rebased the code and attach a new patch.

        Show
        linyiqun Yiqun Lin added a comment - - edited Thanks Andrew Wang for the comments. but maybe we should add a Precondition check on the return value for safety. +1 for this idea. I think we can file another JIRA to make a change if we think it's necessary to do here. This JIRA will be better to focus on the change as the this JIRA's title mentioned. Since HDFS-11095 has fixed one minor that I mentioned in my last comment, rebased the code and attach a new patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 33s trunk passed
        +1 compile 0m 47s trunk passed
        +1 checkstyle 0m 27s trunk passed
        +1 mvnsite 0m 54s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 50s trunk passed
        +1 javadoc 0m 44s trunk passed
        +1 mvninstall 0m 56s the patch passed
        +1 compile 0m 50s the patch passed
        +1 javac 0m 50s the patch passed
        +1 checkstyle 0m 26s the patch passed
        +1 mvnsite 0m 52s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 59s the patch passed
        +1 javadoc 0m 38s the patch passed
        +1 unit 59m 34s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        79m 38s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-11088
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836712/HDFS-11088.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux db4cf336b9f1 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 7e521c5
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17397/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17397/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 33s trunk passed +1 compile 0m 47s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 50s trunk passed +1 javadoc 0m 44s trunk passed +1 mvninstall 0m 56s the patch passed +1 compile 0m 50s the patch passed +1 javac 0m 50s the patch passed +1 checkstyle 0m 26s the patch passed +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 59s the patch passed +1 javadoc 0m 38s the patch passed +1 unit 59m 34s hadoop-hdfs in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 79m 38s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-11088 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836712/HDFS-11088.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux db4cf336b9f1 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7e521c5 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17397/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17397/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        liuml07 Mingliang Liu added a comment -

        I think the proposal and the solution make sense. Thanks for pinging me Andrew Wang.

        I think we can file another JIRA to make a change if we think it's necessary to do here. This JIRA will be better to focus on the change as the this JIRA's title mentioned.

        I think we should add the check in this patch as it is needed largely because we're changing the "force" parameter in activate(). If "force" is true as it was, we can ignore the return value blindly in that it is supposed to return true after leaving safe mode ignoring the orphan blocks (if any); here it becomes false, we'd better add Precondition check on the return value for safety.

        Show
        liuml07 Mingliang Liu added a comment - I think the proposal and the solution make sense. Thanks for pinging me Andrew Wang . I think we can file another JIRA to make a change if we think it's necessary to do here. This JIRA will be better to focus on the change as the this JIRA's title mentioned. I think we should add the check in this patch as it is needed largely because we're changing the "force" parameter in activate() . If "force" is true as it was, we can ignore the return value blindly in that it is supposed to return true after leaving safe mode ignoring the orphan blocks (if any); here it becomes false, we'd better add Precondition check on the return value for safety.
        Hide
        linyiqun Yiqun Lin added a comment -

        Thanks Mingliang Liu for the comments. Attach a new patch to address your comment.

        Show
        linyiqun Yiqun Lin added a comment - Thanks Mingliang Liu for the comments. Attach a new patch to address your comment.
        Hide
        liuml07 Mingliang Liu added a comment -

        If only we have tests... +1 anyway.

        Show
        liuml07 Mingliang Liu added a comment - If only we have tests... +1 anyway.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 8m 0s trunk passed
        +1 compile 0m 57s trunk passed
        +1 checkstyle 0m 27s trunk passed
        +1 mvnsite 0m 56s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 47s trunk passed
        +1 javadoc 0m 40s trunk passed
        +1 mvninstall 0m 46s the patch passed
        +1 compile 0m 47s the patch passed
        +1 javac 0m 47s the patch passed
        +1 checkstyle 0m 25s the patch passed
        +1 mvnsite 0m 57s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 56s the patch passed
        +1 javadoc 0m 38s the patch passed
        -1 unit 57m 26s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        77m 51s



        Reason Tests
        Failed junit tests hadoop.hdfs.web.TestWebHDFS



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-11088
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836738/HDFS-11088.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 408978a3e6ca 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 7e521c5
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17399/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17399/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17399/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 8m 0s trunk passed +1 compile 0m 57s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 47s trunk passed +1 javadoc 0m 40s trunk passed +1 mvninstall 0m 46s the patch passed +1 compile 0m 47s the patch passed +1 javac 0m 47s the patch passed +1 checkstyle 0m 25s the patch passed +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 56s the patch passed +1 javadoc 0m 38s the patch passed -1 unit 57m 26s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 77m 51s Reason Tests Failed junit tests hadoop.hdfs.web.TestWebHDFS Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-11088 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836738/HDFS-11088.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 408978a3e6ca 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7e521c5 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17399/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17399/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17399/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Thanks for the patch Yiqun and reviewing Mingliang! I've committed this to trunk and branch-2.

        Show
        andrew.wang Andrew Wang added a comment - Thanks for the patch Yiqun and reviewing Mingliang! I've committed this to trunk and branch-2.
        Hide
        linyiqun Yiqun Lin added a comment -
        Show
        linyiqun Yiqun Lin added a comment - Thanks Andrew Wang and Mingliang Liu !

          People

          • Assignee:
            linyiqun Yiqun Lin
            Reporter:
            andrew.wang Andrew Wang
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development