Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3611

NameNode prints unnecessary WARNs about edit log normally skipping a few bytes

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: namenode
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      The NameNode currently warns these form of lines at every startup, even if there's no trouble really. For instance, the below is from a NN startup that was only just freshly formatted.

      12/07/08 20:00:22 WARN namenode.EditLogInputStream: skipping 1048563 bytes at the end of edit log  '/Users/harshchouraria/Work/installs/temp-space/tmp-default/dfs-cdh4/data/current/edits_0000000000000000003-0000000000000000003': reached txid 3 out of 3
      

      If this skipping is not really a cause for warning, we should not log it at a WARN level but at an INFO or even DEBUG one. Avoids users getting unnecessarily concerned.

      1. HDFS-3611.001.patch
        1.0 kB
        Colin Patrick McCabe
      2. HDFS-3611.001.patch
        2 kB
        Harsh J

        Activity

        Hide
        Colin Patrick McCabe added a comment -

        I guess changing it to an INFO might be appropriate. It's definitely not worth a WARN.

        Show
        Colin Patrick McCabe added a comment - I guess changing it to an INFO might be appropriate. It's definitely not worth a WARN.
        Hide
        Colin Patrick McCabe added a comment -
        • log 'skipping bytes' message at DEBUG rather than WARN. It's not an error or unusual condition of any kind.
        Show
        Colin Patrick McCabe added a comment - log 'skipping bytes' message at DEBUG rather than WARN. It's not an error or unusual condition of any kind.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12535963/HDFS-3611.001.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2784//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/2784//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2784//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12535963/HDFS-3611.001.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2784//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/2784//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2784//console This message is automatically generated.
        Hide
        Harsh J added a comment -

        +1, test isn't required and findbugs are unrelated. One nit, that LOG.debug statements must always be wrapped inside LOG.isDebugEnabled() condition blocks, I've fixed myself since it was trivial to. Please just ensure this in the future debug log printing/changes.

        Committing.

        Show
        Harsh J added a comment - +1, test isn't required and findbugs are unrelated. One nit, that LOG.debug statements must always be wrapped inside LOG.isDebugEnabled() condition blocks, I've fixed myself since it was trivial to. Please just ensure this in the future debug log printing/changes. Committing.
        Hide
        Harsh J added a comment -

        Committed to branch-2 and trunk. Thanks for improving this and contributing Colin!

        Show
        Harsh J added a comment - Committed to branch-2 and trunk. Thanks for improving this and contributing Colin!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2445 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2445/)
        HDFS-3611. NameNode prints unnecessary WARNs about edit log normally skipping a few bytes. Contributed by Colin Patrick McCabe. (harsh) (Revision 1360001)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360001
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2445 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2445/ ) HDFS-3611 . NameNode prints unnecessary WARNs about edit log normally skipping a few bytes. Contributed by Colin Patrick McCabe. (harsh) (Revision 1360001) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360001 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2512 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2512/)
        HDFS-3611. NameNode prints unnecessary WARNs about edit log normally skipping a few bytes. Contributed by Colin Patrick McCabe. (harsh) (Revision 1360001)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360001
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2512 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2512/ ) HDFS-3611 . NameNode prints unnecessary WARNs about edit log normally skipping a few bytes. Contributed by Colin Patrick McCabe. (harsh) (Revision 1360001) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360001 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2464 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2464/)
        HDFS-3611. NameNode prints unnecessary WARNs about edit log normally skipping a few bytes. Contributed by Colin Patrick McCabe. (harsh) (Revision 1360001)

        Result = FAILURE
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360001
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2464 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2464/ ) HDFS-3611 . NameNode prints unnecessary WARNs about edit log normally skipping a few bytes. Contributed by Colin Patrick McCabe. (harsh) (Revision 1360001) Result = FAILURE harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360001 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1100 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1100/)
        HDFS-3611. NameNode prints unnecessary WARNs about edit log normally skipping a few bytes. Contributed by Colin Patrick McCabe. (harsh) (Revision 1360001)

        Result = FAILURE
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360001
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1100 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1100/ ) HDFS-3611 . NameNode prints unnecessary WARNs about edit log normally skipping a few bytes. Contributed by Colin Patrick McCabe. (harsh) (Revision 1360001) Result = FAILURE harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360001 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1133 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1133/)
        HDFS-3611. NameNode prints unnecessary WARNs about edit log normally skipping a few bytes. Contributed by Colin Patrick McCabe. (harsh) (Revision 1360001)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360001
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1133 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1133/ ) HDFS-3611 . NameNode prints unnecessary WARNs about edit log normally skipping a few bytes. Contributed by Colin Patrick McCabe. (harsh) (Revision 1360001) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360001 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java
        Hide
        Colin Patrick McCabe added a comment -

        Thanks, Harsh.

        Show
        Colin Patrick McCabe added a comment - Thanks, Harsh.

          People

          • Assignee:
            Colin Patrick McCabe
            Reporter:
            Harsh J
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development