Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12224

Add tests to TestJournalNodeSync for sync after JN downtime

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-beta1
    • Component/s: hdfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Adding unit tests for testing JN sync when the JN has a downtime and is formatted.

      1. HDFS-12224.005.patch
        31 kB
        Hanisha Koneru
      2. HDFS-12224.004.patch
        32 kB
        Hanisha Koneru
      3. HDFS-12224.003.patch
        29 kB
        Hanisha Koneru
      4. HDFS-12224.002.patch
        26 kB
        Hanisha Koneru
      5. HDFS-12224.001.patch
        26 kB
        Hanisha Koneru

        Activity

        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Hi Hanisha Koneru, thanks for adding more test coverage for JN sync.

        There is a recursive call here that causes a stack overflow:

          /**
           * Does an edit and rolls the Edit Log.
           *
           * @return the startTxId of next segment after rolling edits.
           */
          private long generateEditLog() throws IOException {
            return generateEditLog();
          }
        
        Show
        arpitagarwal Arpit Agarwal added a comment - Hi Hanisha Koneru , thanks for adding more test coverage for JN sync. There is a recursive call here that causes a stack overflow: /** * Does an edit and rolls the Edit Log. * * @ return the startTxId of next segment after rolling edits. */ private long generateEditLog() throws IOException { return generateEditLog(); }
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Thanks Arpit Agarwal for reviewing the patch. I have fixed the erroneous recursive call in patch v02.

        Show
        hanishakoneru Hanisha Koneru added a comment - Thanks Arpit Agarwal for reviewing the patch. I have fixed the erroneous recursive call in patch v02.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
              trunk Compile Tests
        +1 mvninstall 14m 39s trunk passed
        +1 compile 0m 53s trunk passed
        +1 checkstyle 0m 36s trunk passed
        +1 mvnsite 1m 3s trunk passed
        -1 findbugs 1m 52s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings.
        +1 javadoc 0m 43s trunk passed
              Patch Compile Tests
        +1 mvninstall 0m 49s the patch passed
        +1 compile 0m 48s the patch passed
        -1 javac 0m 48s hadoop-hdfs-project_hadoop-hdfs generated 1 new + 408 unchanged - 1 fixed = 409 total (was 409)
        -0 checkstyle 0m 32s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
        +1 mvnsite 0m 50s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 47s the patch passed
        +1 javadoc 0m 38s the patch passed
              Other Tests
        -1 unit 82m 53s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        109m 54s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner
          hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.qjournal.server.TestJournalNodeSync
          hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
          hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
          hadoop.hdfs.server.blockmanagement.TestReconstructStripedBlocksWithRackAwareness
        Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-12224
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12879462/HDFS-12224.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b835598b0257 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 91f120f
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20519/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        javac https://builds.apache.org/job/PreCommit-HDFS-Build/20519/artifact/patchprocess/diff-compile-javac-hadoop-hdfs-project_hadoop-hdfs.txt
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20519/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/20519/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20519/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20519/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.       trunk Compile Tests +1 mvninstall 14m 39s trunk passed +1 compile 0m 53s trunk passed +1 checkstyle 0m 36s trunk passed +1 mvnsite 1m 3s trunk passed -1 findbugs 1m 52s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. +1 javadoc 0m 43s trunk passed       Patch Compile Tests +1 mvninstall 0m 49s the patch passed +1 compile 0m 48s the patch passed -1 javac 0m 48s hadoop-hdfs-project_hadoop-hdfs generated 1 new + 408 unchanged - 1 fixed = 409 total (was 409) -0 checkstyle 0m 32s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 mvnsite 0m 50s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 38s the patch passed       Other Tests -1 unit 82m 53s hadoop-hdfs in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 109m 54s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.qjournal.server.TestJournalNodeSync   hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation   hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped   hadoop.hdfs.server.blockmanagement.TestReconstructStripedBlocksWithRackAwareness Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-12224 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12879462/HDFS-12224.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b835598b0257 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 91f120f Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20519/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html javac https://builds.apache.org/job/PreCommit-HDFS-Build/20519/artifact/patchprocess/diff-compile-javac-hadoop-hdfs-project_hadoop-hdfs.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20519/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/20519/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20519/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20519/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
              trunk Compile Tests
        +1 mvninstall 15m 37s trunk passed
        +1 compile 0m 52s trunk passed
        +1 checkstyle 0m 35s trunk passed
        +1 mvnsite 0m 54s trunk passed
        -1 findbugs 1m 41s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings.
        +1 javadoc 0m 40s trunk passed
              Patch Compile Tests
        +1 mvninstall 0m 48s the patch passed
        +1 compile 0m 45s the patch passed
        -1 javac 0m 45s hadoop-hdfs-project_hadoop-hdfs generated 1 new + 408 unchanged - 1 fixed = 409 total (was 409)
        -0 checkstyle 0m 32s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
        +1 mvnsite 0m 50s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 46s the patch passed
        +1 javadoc 0m 39s the patch passed
              Other Tests
        -1 unit 65m 22s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        92m 46s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140
          hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-12224
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12879875/HDFS-12224.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux ef68189a9496 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 91f120f
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20520/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        javac https://builds.apache.org/job/PreCommit-HDFS-Build/20520/artifact/patchprocess/diff-compile-javac-hadoop-hdfs-project_hadoop-hdfs.txt
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20520/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/20520/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20520/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20520/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.       trunk Compile Tests +1 mvninstall 15m 37s trunk passed +1 compile 0m 52s trunk passed +1 checkstyle 0m 35s trunk passed +1 mvnsite 0m 54s trunk passed -1 findbugs 1m 41s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. +1 javadoc 0m 40s trunk passed       Patch Compile Tests +1 mvninstall 0m 48s the patch passed +1 compile 0m 45s the patch passed -1 javac 0m 45s hadoop-hdfs-project_hadoop-hdfs generated 1 new + 408 unchanged - 1 fixed = 409 total (was 409) -0 checkstyle 0m 32s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 mvnsite 0m 50s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 39s the patch passed       Other Tests -1 unit 65m 22s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 92m 46s Reason Tests Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-12224 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12879875/HDFS-12224.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ef68189a9496 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 91f120f Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20520/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html javac https://builds.apache.org/job/PreCommit-HDFS-Build/20520/artifact/patchprocess/diff-compile-javac-hadoop-hdfs-project_hadoop-hdfs.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20520/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/20520/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20520/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20520/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        +1 for the v2 patch, with Jenkins issues addressed. Thanks Hanisha Koneru.

        Show
        arpitagarwal Arpit Agarwal added a comment - +1 for the v2 patch, with Jenkins issues addressed. Thanks Hanisha Koneru .
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Thanks Arpit Agarwal.
        I have made a fix for one of the tests and included a metric for number of edit log downloads through JNSync in patch v03. Also fixed checkstyle issue.
        The findbugs and unit test failures are unrelated.

        Show
        hanishakoneru Hanisha Koneru added a comment - Thanks Arpit Agarwal . I have made a fix for one of the tests and included a metric for number of edit log downloads through JNSync in patch v03. Also fixed checkstyle issue. The findbugs and unit test failures are unrelated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 22s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
              trunk Compile Tests
        +1 mvninstall 15m 33s trunk passed
        +1 compile 0m 50s trunk passed
        +1 checkstyle 0m 37s trunk passed
        +1 mvnsite 1m 2s trunk passed
        -1 findbugs 1m 51s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings.
        +1 javadoc 0m 45s trunk passed
              Patch Compile Tests
        +1 mvninstall 0m 56s the patch passed
        +1 compile 0m 54s the patch passed
        -1 javac 0m 54s hadoop-hdfs-project_hadoop-hdfs generated 1 new + 441 unchanged - 1 fixed = 442 total (was 442)
        -0 checkstyle 0m 32s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10)
        +1 mvnsite 0m 50s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 44s the patch passed
        +1 javadoc 0m 38s the patch passed
              Other Tests
        -1 unit 87m 41s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        115m 48s



        Reason Tests
        Failed junit tests hadoop.hdfs.qjournal.server.TestJournalNodeSync
          hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
          hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-12224
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880130/HDFS-12224.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux de60f38e54d7 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 79df1e7
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20538/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        javac https://builds.apache.org/job/PreCommit-HDFS-Build/20538/artifact/patchprocess/diff-compile-javac-hadoop-hdfs-project_hadoop-hdfs.txt
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20538/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/20538/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20538/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20538/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.       trunk Compile Tests +1 mvninstall 15m 33s trunk passed +1 compile 0m 50s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 1m 2s trunk passed -1 findbugs 1m 51s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings. +1 javadoc 0m 45s trunk passed       Patch Compile Tests +1 mvninstall 0m 56s the patch passed +1 compile 0m 54s the patch passed -1 javac 0m 54s hadoop-hdfs-project_hadoop-hdfs generated 1 new + 441 unchanged - 1 fixed = 442 total (was 442) -0 checkstyle 0m 32s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10) +1 mvnsite 0m 50s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 44s the patch passed +1 javadoc 0m 38s the patch passed       Other Tests -1 unit 87m 41s hadoop-hdfs in the patch failed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 115m 48s Reason Tests Failed junit tests hadoop.hdfs.qjournal.server.TestJournalNodeSync   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-12224 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880130/HDFS-12224.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux de60f38e54d7 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 79df1e7 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20538/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html javac https://builds.apache.org/job/PreCommit-HDFS-Build/20538/artifact/patchprocess/diff-compile-javac-hadoop-hdfs-project_hadoop-hdfs.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20538/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/20538/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20538/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20538/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Thanks for the updated patch Hanisha Koneru.

        1. Can you please add the @VisibleForTesting annotation to getNumEditLogsSynced?
        2. We already instantiate a MetricsRegistry via JournalMetrics. Can we expose the new metric from there?

        The patch looks good otherwise.

        Show
        arpitagarwal Arpit Agarwal added a comment - Thanks for the updated patch Hanisha Koneru . Can you please add the @VisibleForTesting annotation to getNumEditLogsSynced? We already instantiate a MetricsRegistry via JournalMetrics. Can we expose the new metric from there? The patch looks good otherwise.
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Thanks for the review Arpit Agarwal.
        Patch v04 addresses your comments and fixes the failed unit test.

        Show
        hanishakoneru Hanisha Koneru added a comment - Thanks for the review Arpit Agarwal . Patch v04 addresses your comments and fixes the failed unit test.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        The @Metrics annotation on JournalNodeSyncer looks unnecessary.

        +1 with that addressed.

        Show
        arpitagarwal Arpit Agarwal added a comment - The @Metrics annotation on JournalNodeSyncer looks unnecessary. +1 with that addressed.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
              trunk Compile Tests
        +1 mvninstall 15m 31s trunk passed
        +1 compile 0m 52s trunk passed
        +1 checkstyle 0m 37s trunk passed
        +1 mvnsite 1m 5s trunk passed
        -1 findbugs 1m 50s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings.
        +1 javadoc 0m 47s trunk passed
              Patch Compile Tests
        +1 mvninstall 0m 54s the patch passed
        +1 compile 0m 55s the patch passed
        -1 javac 0m 55s hadoop-hdfs-project_hadoop-hdfs generated 1 new + 441 unchanged - 1 fixed = 442 total (was 442)
        -0 checkstyle 0m 33s hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 47 unchanged - 0 fixed = 52 total (was 47)
        +1 mvnsite 0m 51s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 45s the patch passed
        +1 javadoc 0m 38s the patch passed
              Other Tests
        -1 unit 65m 37s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        93m 40s



        Reason Tests
        Failed junit tests hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-12224
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880302/HDFS-12224.004.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 0ba970442d96 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / f4c6b00
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20549/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        javac https://builds.apache.org/job/PreCommit-HDFS-Build/20549/artifact/patchprocess/diff-compile-javac-hadoop-hdfs-project_hadoop-hdfs.txt
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20549/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/20549/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20549/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20549/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests +1 mvninstall 15m 31s trunk passed +1 compile 0m 52s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 1m 5s trunk passed -1 findbugs 1m 50s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings. +1 javadoc 0m 47s trunk passed       Patch Compile Tests +1 mvninstall 0m 54s the patch passed +1 compile 0m 55s the patch passed -1 javac 0m 55s hadoop-hdfs-project_hadoop-hdfs generated 1 new + 441 unchanged - 1 fixed = 442 total (was 442) -0 checkstyle 0m 33s hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 47 unchanged - 0 fixed = 52 total (was 47) +1 mvnsite 0m 51s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 38s the patch passed       Other Tests -1 unit 65m 37s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 93m 40s Reason Tests Failed junit tests hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-12224 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880302/HDFS-12224.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0ba970442d96 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f4c6b00 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20549/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html javac https://builds.apache.org/job/PreCommit-HDFS-Build/20549/artifact/patchprocess/diff-compile-javac-hadoop-hdfs-project_hadoop-hdfs.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20549/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/20549/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20549/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20549/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Thanks Arpit Agarwal. I have addressed your comments and checkstyle issues in patch v05.

        Show
        hanishakoneru Hanisha Koneru added a comment - Thanks Arpit Agarwal . I have addressed your comments and checkstyle issues in patch v05.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Thanks Hanisha Koneru.

        +1 for the v5 patch pending Jenkins.

        Show
        arpitagarwal Arpit Agarwal added a comment - Thanks Hanisha Koneru . +1 for the v5 patch pending Jenkins.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
              trunk Compile Tests
        +1 mvninstall 14m 15s trunk passed
        +1 compile 0m 53s trunk passed
        +1 checkstyle 0m 37s trunk passed
        +1 mvnsite 1m 1s trunk passed
        -1 findbugs 1m 45s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings.
        +1 javadoc 0m 43s trunk passed
              Patch Compile Tests
        +1 mvninstall 0m 57s the patch passed
        +1 compile 0m 51s the patch passed
        -1 javac 0m 51s hadoop-hdfs-project_hadoop-hdfs generated 1 new + 441 unchanged - 1 fixed = 442 total (was 442)
        +1 checkstyle 0m 34s the patch passed
        +1 mvnsite 0m 54s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 51s the patch passed
        +1 javadoc 0m 39s the patch passed
              Other Tests
        -1 unit 73m 58s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        100m 51s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.server.datanode.TestDirectoryScanner



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-12224
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880434/HDFS-12224.005.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 18611a1ca5a9 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 02bf328
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20556/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        javac https://builds.apache.org/job/PreCommit-HDFS-Build/20556/artifact/patchprocess/diff-compile-javac-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/20556/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20556/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20556/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests +1 mvninstall 14m 15s trunk passed +1 compile 0m 53s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 1m 1s trunk passed -1 findbugs 1m 45s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings. +1 javadoc 0m 43s trunk passed       Patch Compile Tests +1 mvninstall 0m 57s the patch passed +1 compile 0m 51s the patch passed -1 javac 0m 51s hadoop-hdfs-project_hadoop-hdfs generated 1 new + 441 unchanged - 1 fixed = 442 total (was 442) +1 checkstyle 0m 34s the patch passed +1 mvnsite 0m 54s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 51s the patch passed +1 javadoc 0m 39s the patch passed       Other Tests -1 unit 73m 58s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 100m 51s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.datanode.TestDirectoryScanner Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-12224 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880434/HDFS-12224.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 18611a1ca5a9 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 02bf328 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20556/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html javac https://builds.apache.org/job/PreCommit-HDFS-Build/20556/artifact/patchprocess/diff-compile-javac-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/20556/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20556/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20556/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        The javac warning was not introduced by this patch and the unit test failures are unrelated.

        I've committed this to trunk. Thanks for the contribution Hanisha Koneru.

        Show
        arpitagarwal Arpit Agarwal added a comment - The javac warning was not introduced by this patch and the unit test failures are unrelated. I've committed this to trunk. Thanks for the contribution Hanisha Koneru .
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12121 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12121/)
        HDFS-12224. Add tests to TestJournalNodeSync for sync after JN downtime. (arp: rev bbc6d254c8a953abba69415d80edeede3ee6269d)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JournalMetrics.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JournalNodeSyncer.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/Journal.java
        • (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/server/TestJournalNodeSync.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/server/TestJournalNode.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/TestJournalNodeSync.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12121 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12121/ ) HDFS-12224 . Add tests to TestJournalNodeSync for sync after JN downtime. (arp: rev bbc6d254c8a953abba69415d80edeede3ee6269d) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JournalMetrics.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JournalNodeSyncer.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/Journal.java (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/server/TestJournalNodeSync.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/server/TestJournalNode.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/TestJournalNodeSync.java

          People

          • Assignee:
            hanishakoneru Hanisha Koneru
            Reporter:
            hanishakoneru Hanisha Koneru
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development