Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9941

Do not log StandbyException on NN, other minor logging fixes

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The NameNode can skip logging StandbyException messages. These are seen regularly in normal operation and convey no useful information.

      We no longer log the locations of newly allocated blocks in 2.8.0. The DN IDs can be useful for debugging so let's add that back.

      1. HDFS-9941.01.patch
        9 kB
        Arpit Agarwal
      2. HDFS-9941.02.patch
        9 kB
        Arpit Agarwal
      3. HDFS-9941.03.patch
        9 kB
        Arpit Agarwal
      4. HDFS-9941-branch-2.03.patch
        8 kB
        Arpit Agarwal

        Activity

        Hide
        liuml07 Mingliang Liu added a comment -

        This proposal makes sense to me. Thanks for reporting this, Arpit Agarwal.

        Show
        liuml07 Mingliang Liu added a comment - This proposal makes sense to me. Thanks for reporting this, Arpit Agarwal .
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user arp7 opened a pull request:

        https://github.com/apache/hadoop/pull/83

        HDFS-9941. Do not log StandbyException on NN, other minor logging fixes.

        v1 patch includes the following fixes:

        1. Suppress StandbyException log messages for NameNode.
        2. saveAllocatedBlock logs the block locations (DN xfer addresses).
        3. logBlockReplicationInfo logs to the blockStateChangeLog instead of DecomissionManager#LOG. Also added a log level guard.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/arp7/hadoop HDFS-9941

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/hadoop/pull/83.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #83


        commit fbf1bf7fd6d530687b2e9610b8c160e9c7a3fdcb
        Author: Arpit Agarwal <arp@apache.org>
        Date: 2016-03-10T19:17:17Z

        HDFS-9941. Do not log StandbyException on NN, other minor logging fixes.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user arp7 opened a pull request: https://github.com/apache/hadoop/pull/83 HDFS-9941 . Do not log StandbyException on NN, other minor logging fixes. v1 patch includes the following fixes: Suppress StandbyException log messages for NameNode. saveAllocatedBlock logs the block locations (DN xfer addresses). logBlockReplicationInfo logs to the blockStateChangeLog instead of DecomissionManager#LOG . Also added a log level guard. You can merge this pull request into a Git repository by running: $ git pull https://github.com/arp7/hadoop HDFS-9941 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/83.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #83 commit fbf1bf7fd6d530687b2e9610b8c160e9c7a3fdcb Author: Arpit Agarwal <arp@apache.org> Date: 2016-03-10T19:17:17Z HDFS-9941 . Do not log StandbyException on NN, other minor logging fixes.
        Hide
        cnauroth Chris Nauroth added a comment -

        Arpit Agarwal, thank you for the patch.

          public void appendUCPartsConcise(StringBuilder sb) {
            sb.append("replicas=");
            int i = 0;
            for (ReplicaUnderConstruction r : replicas) {
              sb.append(r.getExpectedStorageLocation().getDatanodeDescriptor());
              if (++i < replicas.length) {
                sb.append(", ");
              }
            }
            sb.append("");
          }
        

        The last line looks like a no-op. Can it be removed?

            b.<Block>appendStringTo(sb);
        

        Was the type qualifier here necessary? It seems like I can compile fine without it.

        Show
        cnauroth Chris Nauroth added a comment - Arpit Agarwal , thank you for the patch. public void appendUCPartsConcise(StringBuilder sb) { sb.append( "replicas=" ); int i = 0; for (ReplicaUnderConstruction r : replicas) { sb.append(r.getExpectedStorageLocation().getDatanodeDescriptor()); if (++i < replicas.length) { sb.append( ", " ); } } sb.append(""); } The last line looks like a no-op. Can it be removed? b.<Block>appendStringTo(sb); Was the type qualifier here necessary? It seems like I can compile fine without it.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Thanks for the quick review Chris Nauroth.

        The last line looks like a no-op. Can it be removed?

        Fixed. Updated the pull request and attached patch.

        Was the type qualifier here necessary? It seems like I can compile fine without it.

        I want to explicitly invoke Block#appendStringTo to just print blk_<blkid>_<genstamp>. There may be overloads that print a lot more information.

        Show
        arpitagarwal Arpit Agarwal added a comment - Thanks for the quick review Chris Nauroth . The last line looks like a no-op. Can it be removed? Fixed. Updated the pull request and attached patch. Was the type qualifier here necessary? It seems like I can compile fine without it. I want to explicitly invoke Block#appendStringTo to just print blk_<blkid>_<genstamp> . There may be overloads that print a lot more information.
        Hide
        arpitagarwal Arpit Agarwal added a comment - - edited

        Actually turns out I was confused. The dispatch doesn't work the way I thought. I've updated the v3 patch. Thanks for catching this.

        Show
        arpitagarwal Arpit Agarwal added a comment - - edited Actually turns out I was confused. The dispatch doesn't work the way I thought. I've updated the v3 patch. Thanks for catching this.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 6m 43s trunk passed
        +1 compile 0m 41s trunk passed with JDK v1.8.0_74
        +1 compile 0m 41s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 21s trunk passed
        +1 mvnsite 0m 51s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 53s trunk passed
        +1 javadoc 1m 7s trunk passed with JDK v1.8.0_74
        +1 javadoc 1m 49s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 47s the patch passed
        +1 compile 0m 39s the patch passed with JDK v1.8.0_74
        +1 javac 0m 39s the patch passed
        +1 compile 0m 38s the patch passed with JDK v1.7.0_95
        +1 javac 0m 38s the patch passed
        -1 checkstyle 0m 21s hadoop-hdfs-project/hadoop-hdfs: patch generated 1 new + 83 unchanged - 1 fixed = 84 total (was 84)
        +1 mvnsite 0m 48s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 2m 8s the patch passed
        +1 javadoc 1m 6s the patch passed with JDK v1.8.0_74
        +1 javadoc 1m 46s the patch passed with JDK v1.7.0_95
        -1 unit 59m 51s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
        -1 unit 53m 46s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 20s Patch does not generate ASF License warnings.
        138m 58s



        Reason Tests
        JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.balancer.TestBalancer
          hadoop.hdfs.server.datanode.TestDataNodeLifeline
        JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestHFlush
          hadoop.hdfs.TestModTime



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Issue HDFS-9941
        GITHUB PR https://github.com/apache/hadoop/pull/83
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 9dc1715e5c0c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / adf1cdf
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14785/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14785/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14785/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14785/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14785/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14785/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14785/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14785/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 43s trunk passed +1 compile 0m 41s trunk passed with JDK v1.8.0_74 +1 compile 0m 41s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 53s trunk passed +1 javadoc 1m 7s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 49s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 47s the patch passed +1 compile 0m 39s the patch passed with JDK v1.8.0_74 +1 javac 0m 39s the patch passed +1 compile 0m 38s the patch passed with JDK v1.7.0_95 +1 javac 0m 38s the patch passed -1 checkstyle 0m 21s hadoop-hdfs-project/hadoop-hdfs: patch generated 1 new + 83 unchanged - 1 fixed = 84 total (was 84) +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 11s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 2m 8s the patch passed +1 javadoc 1m 6s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 46s the patch passed with JDK v1.7.0_95 -1 unit 59m 51s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 53m 46s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 20s Patch does not generate ASF License warnings. 138m 58s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.datanode.TestDataNodeLifeline JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestHFlush   hadoop.hdfs.TestModTime Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Issue HDFS-9941 GITHUB PR https://github.com/apache/hadoop/pull/83 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9dc1715e5c0c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / adf1cdf Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14785/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14785/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14785/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14785/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14785/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14785/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14785/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14785/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 12m 10s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 6m 38s trunk passed
        +1 compile 0m 40s trunk passed with JDK v1.8.0_74
        +1 compile 0m 42s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 0m 50s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 55s trunk passed
        +1 javadoc 1m 7s trunk passed with JDK v1.8.0_74
        +1 javadoc 1m 49s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 46s the patch passed
        +1 compile 0m 35s the patch passed with JDK v1.8.0_74
        +1 javac 0m 35s the patch passed
        +1 compile 0m 37s the patch passed with JDK v1.7.0_95
        +1 javac 0m 37s the patch passed
        -1 checkstyle 0m 18s hadoop-hdfs-project/hadoop-hdfs: patch generated 1 new + 83 unchanged - 1 fixed = 84 total (was 84)
        +1 mvnsite 0m 48s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 2m 9s the patch passed
        +1 javadoc 1m 3s the patch passed with JDK v1.8.0_74
        +1 javadoc 1m 43s the patch passed with JDK v1.7.0_95
        -1 unit 59m 58s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
        -1 unit 57m 6s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 21s Patch does not generate ASF License warnings.
        154m 14s



        Reason Tests
        JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeMetrics
        JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockManager
          hadoop.hdfs.TestHFlush



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Issue HDFS-9941
        GITHUB PR https://github.com/apache/hadoop/pull/83
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 64700d30ee32 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e01c6ea
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14789/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14789/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14789/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14789/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14789/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14789/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14789/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14789/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 12m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 38s trunk passed +1 compile 0m 40s trunk passed with JDK v1.8.0_74 +1 compile 0m 42s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 50s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 55s trunk passed +1 javadoc 1m 7s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 49s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 46s the patch passed +1 compile 0m 35s the patch passed with JDK v1.8.0_74 +1 javac 0m 35s the patch passed +1 compile 0m 37s the patch passed with JDK v1.7.0_95 +1 javac 0m 37s the patch passed -1 checkstyle 0m 18s hadoop-hdfs-project/hadoop-hdfs: patch generated 1 new + 83 unchanged - 1 fixed = 84 total (was 84) +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 11s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 2m 9s the patch passed +1 javadoc 1m 3s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 43s the patch passed with JDK v1.7.0_95 -1 unit 59m 58s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 57m 6s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 154m 14s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeMetrics JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.hdfs.TestHFlush Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Issue HDFS-9941 GITHUB PR https://github.com/apache/hadoop/pull/83 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 64700d30ee32 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e01c6ea Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14789/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14789/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14789/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14789/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14789/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14789/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14789/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14789/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        The Checkstyle warning is not practical to address in scope of this patch. The test failures are unrelated.

        While reviewing this, I squinted at the static Block#toString method for an hour thinking that I was misunderstanding something about the way Java dynamic dispatch works. It turns out no, my understanding is fine, and HDFS-9350 introduced a bug in that method. This isn't directly related to the HDFS-9941 patch though, so I left a comment on HDFS-9350 for follow-up.

        I am +1 for patch v003 going into trunk, but it doesn't merge cleanly to branch-2. Arpit Agarwal, would you please post a patch for branch-2/branch-2.8? Thank you.

        Show
        cnauroth Chris Nauroth added a comment - The Checkstyle warning is not practical to address in scope of this patch. The test failures are unrelated. While reviewing this, I squinted at the static Block#toString method for an hour thinking that I was misunderstanding something about the way Java dynamic dispatch works. It turns out no, my understanding is fine, and HDFS-9350 introduced a bug in that method. This isn't directly related to the HDFS-9941 patch though, so I left a comment on HDFS-9350 for follow-up. I am +1 for patch v003 going into trunk, but it doesn't merge cleanly to branch-2. Arpit Agarwal , would you please post a patch for branch-2/branch-2.8? Thank you.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Thank you for the review Chris Nauroth. branch-2 patch attached.? It also applies cleanly to branch-2.8.

        Show
        arpitagarwal Arpit Agarwal added a comment - Thank you for the review Chris Nauroth . branch-2 patch attached.? It also applies cleanly to branch-2.8.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 30s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 9m 30s trunk passed
        +1 compile 1m 17s trunk passed with JDK v1.8.0_74
        +1 compile 1m 3s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 33s trunk passed
        +1 mvnsite 1m 20s trunk passed
        +1 mvneclipse 0m 21s trunk passed
        +1 findbugs 2m 38s trunk passed
        +1 javadoc 1m 23s trunk passed with JDK v1.8.0_74
        +1 javadoc 1m 59s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 52s the patch passed
        +1 compile 0m 51s the patch passed with JDK v1.8.0_74
        +1 javac 0m 51s the patch passed
        +1 compile 0m 44s the patch passed with JDK v1.7.0_95
        +1 javac 0m 44s the patch passed
        -1 checkstyle 0m 20s hadoop-hdfs-project/hadoop-hdfs: patch generated 1 new + 83 unchanged - 1 fixed = 84 total (was 84)
        +1 mvnsite 0m 57s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 2m 18s the patch passed
        +1 javadoc 1m 17s the patch passed with JDK v1.8.0_74
        +1 javadoc 2m 2s the patch passed with JDK v1.7.0_95
        -1 unit 81m 45s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
        -1 unit 71m 52s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 20s Patch does not generate ASF License warnings.
        186m 36s



        Reason Tests
        JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockManager
          hadoop.hdfs.server.datanode.TestDataNodeMetrics
          hadoop.hdfs.server.balancer.TestBalancer
          hadoop.hdfs.server.namenode.TestEditLog
          hadoop.hdfs.TestDFSClientRetries
        JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.TestEditLog
          hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Issue HDFS-9941
        GITHUB PR https://github.com/apache/hadoop/pull/83
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 5f2680e3542d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 658ee95
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14799/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14799/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14799/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14799/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14799/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14799/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14799/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14799/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 30s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 30s trunk passed +1 compile 1m 17s trunk passed with JDK v1.8.0_74 +1 compile 1m 3s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 33s trunk passed +1 mvnsite 1m 20s trunk passed +1 mvneclipse 0m 21s trunk passed +1 findbugs 2m 38s trunk passed +1 javadoc 1m 23s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 59s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 52s the patch passed +1 compile 0m 51s the patch passed with JDK v1.8.0_74 +1 javac 0m 51s the patch passed +1 compile 0m 44s the patch passed with JDK v1.7.0_95 +1 javac 0m 44s the patch passed -1 checkstyle 0m 20s hadoop-hdfs-project/hadoop-hdfs: patch generated 1 new + 83 unchanged - 1 fixed = 84 total (was 84) +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 12s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 2m 18s the patch passed +1 javadoc 1m 17s the patch passed with JDK v1.8.0_74 +1 javadoc 2m 2s the patch passed with JDK v1.7.0_95 -1 unit 81m 45s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 71m 52s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 20s Patch does not generate ASF License warnings. 186m 36s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.hdfs.server.datanode.TestDataNodeMetrics   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.TestDFSClientRetries JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Issue HDFS-9941 GITHUB PR https://github.com/apache/hadoop/pull/83 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5f2680e3542d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 658ee95 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14799/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14799/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14799/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14799/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14799/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14799/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14799/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14799/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        +1 for both the trunk patch and the branch-2 patch. I confirmed that the test failures in the last pre-commit run are unrelated and don't repro. I have committed this to trunk, branch-2 and branch-2.8. Arpit Agarwal, thank you for contributing this patch.

        Show
        cnauroth Chris Nauroth added a comment - +1 for both the trunk patch and the branch-2 patch. I confirmed that the test failures in the last pre-commit run are unrelated and don't repro. I have committed this to trunk, branch-2 and branch-2.8. Arpit Agarwal , thank you for contributing this patch.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9459 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9459/)
        HDFS-9941. Do not log StandbyException on NN, other minor logging fixes. (cnauroth: rev 5644137adad30c84e40d2c4719627b3aabc73628)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockUnderConstructionFeature.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9459 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9459/ ) HDFS-9941 . Do not log StandbyException on NN, other minor logging fixes. (cnauroth: rev 5644137adad30c84e40d2c4719627b3aabc73628) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockUnderConstructionFeature.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Thank you Chris Nauroth.

        Show
        arpitagarwal Arpit Agarwal added a comment - Thank you Chris Nauroth .

          People

          • Assignee:
            arpitagarwal Arpit Agarwal
            Reporter:
            arpitagarwal Arpit Agarwal
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development