Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7875

Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      By mistake i configured dfs.datanode.failed.volumes.tolerated equal to the number of volume configured. Got stuck for some time in debugging since the log message didn't give much details.

      The log message can be more detail. Added a patch with change in message.
      Please have a look

      1. 0001-HDFS-7875.patch
        1 kB
        nijel
      2. 0002-HDFS-7875.patch
        1 kB
        nijel
      3. 0003-HDFS-7875.patch
        1 kB
        nijel
      4. 0004-HDFS-7875.patch
        1 kB
        nijel

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2093 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2093/)
        HDFS-7875. Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. (harsh: rev eda02540ce53732585b3f31411b2e65db569eb25)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2093 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2093/ ) HDFS-7875 . Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. (harsh: rev eda02540ce53732585b3f31411b2e65db569eb25) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #143 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/143/)
        HDFS-7875. Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. (harsh: rev eda02540ce53732585b3f31411b2e65db569eb25)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #143 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/143/ ) HDFS-7875 . Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. (harsh: rev eda02540ce53732585b3f31411b2e65db569eb25) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #134 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/134/)
        HDFS-7875. Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. (harsh: rev eda02540ce53732585b3f31411b2e65db569eb25)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #134 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/134/ ) HDFS-7875 . Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. (harsh: rev eda02540ce53732585b3f31411b2e65db569eb25) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2075 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2075/)
        HDFS-7875. Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. (harsh: rev eda02540ce53732585b3f31411b2e65db569eb25)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2075 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2075/ ) HDFS-7875 . Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. (harsh: rev eda02540ce53732585b3f31411b2e65db569eb25) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #877 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/877/)
        HDFS-7875. Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. (harsh: rev eda02540ce53732585b3f31411b2e65db569eb25)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #877 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/877/ ) HDFS-7875 . Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. (harsh: rev eda02540ce53732585b3f31411b2e65db569eb25) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #143 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/143/)
        HDFS-7875. Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. (harsh: rev eda02540ce53732585b3f31411b2e65db569eb25)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #143 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/143/ ) HDFS-7875 . Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. (harsh: rev eda02540ce53732585b3f31411b2e65db569eb25) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        nijel nijel added a comment -

        Thanks Harsh

        Show
        nijel nijel added a comment - Thanks Harsh
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7420 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7420/)
        HDFS-7875. Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. (harsh: rev eda02540ce53732585b3f31411b2e65db569eb25)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7420 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7420/ ) HDFS-7875 . Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. (harsh: rev eda02540ce53732585b3f31411b2e65db569eb25) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        qwertymaniac Harsh J added a comment -

        Committed to branch-2 and trunk. Many thanks again for the improvement!

        Show
        qwertymaniac Harsh J added a comment - Committed to branch-2 and trunk. Many thanks again for the improvement!
        Hide
        qwertymaniac Harsh J added a comment -

        Test seems flaky, as it passed without and with the patch locally. Going ahead with the commit.

        Show
        qwertymaniac Harsh J added a comment - Test seems flaky, as it passed without and with the patch locally. Going ahead with the commit.
        Hide
        qwertymaniac Harsh J added a comment -

        Thanks! Change looks good to me, +1. At first glance, the TestDatanodeManager failure appears unrelated, but re-running it locally to be certain anyhow. Will commit after that.

        The TestTracing failure appears to be something broken in trunk instead and can be ignored here.

        Show
        qwertymaniac Harsh J added a comment - Thanks! Change looks good to me, +1. At first glance, the TestDatanodeManager failure appears unrelated, but re-running it locally to be certain anyhow. Will commit after that. The TestTracing failure appears to be something broken in trunk instead and can be ignored here.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12706911/0004-HDFS-7875.patch
        against trunk revision 3ca5bd1.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager
        org.apache.hadoop.tracing.TestTracing

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10051//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10051//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12706911/0004-HDFS-7875.patch against trunk revision 3ca5bd1. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager org.apache.hadoop.tracing.TestTracing Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10051//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10051//console This message is automatically generated.
        Hide
        nijel nijel added a comment -

        thanks Harsh for the comments
        Updated patch with the changes

        Show
        nijel nijel added a comment - thanks Harsh for the comments Updated patch with the changes
        Hide
        qwertymaniac Harsh J added a comment -

        The new error looks better, but do you feel its possible to also print the # of currently configured volumes, to add further context into the error?

        Also two nits:

        1. "configurd" -> "configured"
        2. "Value configured is either 0" -> "Value configured is either less than 0" (0 is a valid value)

        Show
        qwertymaniac Harsh J added a comment - The new error looks better, but do you feel its possible to also print the # of currently configured volumes, to add further context into the error? Also two nits: 1. "configurd" -> "configured" 2. "Value configured is either 0" -> "Value configured is either less than 0" (0 is a valid value)
        Hide
        nijel nijel added a comment -

        Build failure looks like not related to patch

        cp: cannot stat '/home/jenkins/buildSupport/lib/': No such file or directory*
        Checking patch hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java...

        Show
        nijel nijel added a comment - Build failure looks like not related to patch cp: cannot stat '/home/jenkins/buildSupport/lib/ ': No such file or directory* Checking patch hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java...
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12703408/0003-HDFS-7875.patch
        against trunk revision 5578e22.

        -1 patch. Trunk compilation may be broken.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9794//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12703408/0003-HDFS-7875.patch against trunk revision 5578e22. -1 patch . Trunk compilation may be broken. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9794//console This message is automatically generated.
        Hide
        nijel nijel added a comment -

        Updated for the comments.
        sorry for the simple mistake

        Show
        nijel nijel added a comment - Updated for the comments. sorry for the simple mistake
        Hide
        aw Allen Wittenauer added a comment -

        Let's put a space in between the . and Value. There is also extraneous space at the end of that line.

        Show
        aw Allen Wittenauer added a comment - Let's put a space in between the . and Value. There is also extraneous space at the end of that line.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12702348/0002-HDFS-7875.patch
        against trunk revision f2d7a67.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9719//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9719//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12702348/0002-HDFS-7875.patch against trunk revision f2d7a67. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9719//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9719//console This message is automatically generated.
        Hide
        nijel nijel added a comment -

        Thanks for having a look in this change
        Updated the patch with format change. Hope it will be ok

        Show
        nijel nijel added a comment - Thanks for having a look in this change Updated the patch with format change. Hope it will be ok
        Hide
        aw Allen Wittenauer added a comment -

        Weird. I can't get to those builds to force a re-run. So cancel/submit here we go...

        That said, spacing is weird here:

        +              + ".Value configured is either 0 or >= to the configured volumes. ");
        
        Show
        aw Allen Wittenauer added a comment - Weird. I can't get to those builds to force a re-run. So cancel/submit here we go... That said, spacing is weird here: + + ".Value configured is either 0 or >= to the configured volumes. " );
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12702164/0001-HDFS-7875.patch
        against trunk revision 4228de9.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.TestFileTruncate
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9711//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9711//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12702164/0001-HDFS-7875.patch against trunk revision 4228de9. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.TestFileTruncate org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9711//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9711//console This message is automatically generated.
        Hide
        nijel nijel added a comment -

        patch for log message change

        Show
        nijel nijel added a comment - patch for log message change

          People

          • Assignee:
            nijel nijel
            Reporter:
            nijel nijel
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development