Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6804

Add timeout when starting JobHistoryServer in MiniMRYarnCluster

    Details

    • Type: Test
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This JIRA is to follow up a TODO in MiniMRYarnCluster.
      //TODO Add a timeout. State.STOPPED check ?
      I think State.STOPPED check is not needed. I do not see the value to check STOPPED state here.

      1. YARN-5160.01.patch
        1 kB
        Andras Bokor
      2. YARN-5160-branch-2.01.patch
        2 kB
        Andras Bokor

        Activity

        Hide
        boky01 Andras Bokor added a comment -

        Uploading first patch.
        I could not write JUnit test. I should have mocked the JobHistoryServer object but it is created inside serviceStart so I could not mock it.
        I did some manual test and it worked.

        Show
        boky01 Andras Bokor added a comment - Uploading first patch. I could not write JUnit test. I should have mocked the JobHistoryServer object but it is created inside serviceStart so I could not mock it. I did some manual test and it worked.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 58s trunk passed
        +1 compile 0m 24s trunk passed
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 30s trunk passed
        +1 mvneclipse 0m 17s trunk passed
        +1 findbugs 0m 26s trunk passed
        +1 javadoc 0m 13s trunk passed
        +1 mvninstall 0m 24s the patch passed
        +1 compile 0m 22s the patch passed
        +1 javac 0m 22s the patch passed
        +1 checkstyle 0m 14s the patch passed
        +1 mvnsite 0m 29s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 0m 29s the patch passed
        +1 javadoc 0m 10s the patch passed
        -1 unit 115m 42s hadoop-mapreduce-client-jobclient in the patch failed.
        +1 asflicense 0m 27s The patch does not generate ASF License warnings.
        128m 35s



        Reason Tests
        Failed junit tests hadoop.mapred.TestMRCJCFileOutputCommitter



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821558/YARN-5160.01.patch
        JIRA Issue YARN-5160
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e507e0c41f2d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 7fc70c6
        Default Java 1.8.0_101
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12610/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12610/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12610/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12610/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12610/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 58s trunk passed +1 compile 0m 24s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 30s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 0m 26s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 24s the patch passed +1 compile 0m 22s the patch passed +1 javac 0m 22s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 29s the patch passed +1 mvneclipse 0m 14s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 0m 29s the patch passed +1 javadoc 0m 10s the patch passed -1 unit 115m 42s hadoop-mapreduce-client-jobclient in the patch failed. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 128m 35s Reason Tests Failed junit tests hadoop.mapred.TestMRCJCFileOutputCommitter Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821558/YARN-5160.01.patch JIRA Issue YARN-5160 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e507e0c41f2d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7fc70c6 Default Java 1.8.0_101 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12610/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12610/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12610/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12610/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-YARN-Build/12610/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 53s trunk passed
        +1 compile 0m 28s trunk passed
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 33s trunk passed
        +1 mvneclipse 0m 18s trunk passed
        +1 findbugs 0m 24s trunk passed
        +1 javadoc 0m 13s trunk passed
        +1 mvninstall 0m 22s the patch passed
        +1 compile 0m 21s the patch passed
        +1 javac 0m 21s the patch passed
        +1 checkstyle 0m 16s the patch passed
        +1 mvnsite 0m 32s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 0m 32s the patch passed
        +1 javadoc 0m 12s the patch passed
        +1 unit 106m 43s hadoop-mapreduce-client-jobclient in the patch passed.
        +1 asflicense 0m 26s The patch does not generate ASF License warnings.
        120m 38s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821558/YARN-5160.01.patch
        JIRA Issue YARN-5160
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 93583a146059 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c62ae71
        Default Java 1.8.0_101
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13431/artifact/patchprocess/whitespace-eol.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13431/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13431/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 53s trunk passed +1 compile 0m 28s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 33s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 0m 24s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 22s the patch passed +1 compile 0m 21s the patch passed +1 javac 0m 21s the patch passed +1 checkstyle 0m 16s the patch passed +1 mvnsite 0m 32s the patch passed +1 mvneclipse 0m 14s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 0m 32s the patch passed +1 javadoc 0m 12s the patch passed +1 unit 106m 43s hadoop-mapreduce-client-jobclient in the patch passed. +1 asflicense 0m 26s The patch does not generate ASF License warnings. 120m 38s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821558/YARN-5160.01.patch JIRA Issue YARN-5160 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 93583a146059 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c62ae71 Default Java 1.8.0_101 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13431/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13431/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-YARN-Build/13431/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jlowe Jason Lowe added a comment -

        Thanks for the patch!

        Patch seems OK to me, but it cannot be applied to anything but trunk since it uses JDK8 features. Would you mind providing a patch that can be used against branch-2 as well?

        Show
        jlowe Jason Lowe added a comment - Thanks for the patch! Patch seems OK to me, but it cannot be applied to anything but trunk since it uses JDK8 features. Would you mind providing a patch that can be used against branch-2 as well?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 18m 3s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 48s branch-2 passed
        +1 compile 0m 20s branch-2 passed with JDK v1.8.0_111
        +1 compile 0m 23s branch-2 passed with JDK v1.7.0_111
        +1 checkstyle 0m 18s branch-2 passed
        +1 mvnsite 0m 28s branch-2 passed
        +1 mvneclipse 0m 15s branch-2 passed
        +1 findbugs 0m 29s branch-2 passed
        +1 javadoc 0m 12s branch-2 passed with JDK v1.8.0_111
        +1 javadoc 0m 14s branch-2 passed with JDK v1.7.0_111
        +1 mvninstall 0m 22s the patch passed
        +1 compile 0m 17s the patch passed with JDK v1.8.0_111
        +1 javac 0m 17s the patch passed
        +1 compile 0m 21s the patch passed with JDK v1.7.0_111
        +1 javac 0m 21s the patch passed
        +1 checkstyle 0m 15s the patch passed
        +1 mvnsite 0m 27s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 38s the patch passed
        +1 javadoc 0m 11s the patch passed with JDK v1.8.0_111
        +1 javadoc 0m 13s the patch passed with JDK v1.7.0_111
        -1 unit 108m 47s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_111.
        -1 unit 117m 9s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_111.
        +1 asflicense 0m 34s The patch does not generate ASF License warnings.
        258m 0s



        Reason Tests
        JDK v1.8.0_111 Failed junit tests hadoop.mapred.pipes.TestPipeApplication
          hadoop.mapred.TestMRCJCFileOutputCommitter
        JDK v1.7.0_111 Failed junit tests hadoop.mapred.pipes.TestPipeApplication
          hadoop.mapred.TestMRCJCFileOutputCommitter



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:b59b8b7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12839007/YARN-5160-branch-2.01.patch
        JIRA Issue MAPREDUCE-6804
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 17e9cf2db899 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2 / 260f3a9
        Default Java 1.7.0_111
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6811/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_111.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6811/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_111.txt
        unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6811/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_111.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6811/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_111.txt
        JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6811/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6811/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 18m 3s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 48s branch-2 passed +1 compile 0m 20s branch-2 passed with JDK v1.8.0_111 +1 compile 0m 23s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 0m 18s branch-2 passed +1 mvnsite 0m 28s branch-2 passed +1 mvneclipse 0m 15s branch-2 passed +1 findbugs 0m 29s branch-2 passed +1 javadoc 0m 12s branch-2 passed with JDK v1.8.0_111 +1 javadoc 0m 14s branch-2 passed with JDK v1.7.0_111 +1 mvninstall 0m 22s the patch passed +1 compile 0m 17s the patch passed with JDK v1.8.0_111 +1 javac 0m 17s the patch passed +1 compile 0m 21s the patch passed with JDK v1.7.0_111 +1 javac 0m 21s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 0m 27s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 38s the patch passed +1 javadoc 0m 11s the patch passed with JDK v1.8.0_111 +1 javadoc 0m 13s the patch passed with JDK v1.7.0_111 -1 unit 108m 47s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_111. -1 unit 117m 9s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_111. +1 asflicense 0m 34s The patch does not generate ASF License warnings. 258m 0s Reason Tests JDK v1.8.0_111 Failed junit tests hadoop.mapred.pipes.TestPipeApplication   hadoop.mapred.TestMRCJCFileOutputCommitter JDK v1.7.0_111 Failed junit tests hadoop.mapred.pipes.TestPipeApplication   hadoop.mapred.TestMRCJCFileOutputCommitter Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12839007/YARN-5160-branch-2.01.patch JIRA Issue MAPREDUCE-6804 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 17e9cf2db899 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 260f3a9 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6811/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_111.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6811/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_111.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6811/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_111.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6811/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6811/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6811/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        boky01 Andras Bokor added a comment -

        Jason Lowe,

        Thanks for taking a look on this.
        I uploaded a patch for branch-2. The JUnit failures seem unrelated:

        • TestPipeApplication fails either with patch or without patch
        • TestMRCJCFileOutputCommitter does no fail locally

        The patch for trunk has white space error. Do you want me to upload a new one?

        Show
        boky01 Andras Bokor added a comment - Jason Lowe , Thanks for taking a look on this. I uploaded a patch for branch-2. The JUnit failures seem unrelated: TestPipeApplication fails either with patch or without patch TestMRCJCFileOutputCommitter does no fail locally The patch for trunk has white space error. Do you want me to upload a new one?
        Hide
        jlowe Jason Lowe added a comment -

        +1 lgtm. I'll fix the whitespace issue during the commit.

        Show
        jlowe Jason Lowe added a comment - +1 lgtm. I'll fix the whitespace issue during the commit.
        Hide
        jlowe Jason Lowe added a comment -

        Thanks, Andras Bokor! I committed this to trunk, branch-2, and branch-2.8.

        Show
        jlowe Jason Lowe added a comment - Thanks, Andras Bokor ! I committed this to trunk, branch-2, and branch-2.8.
        Hide
        boky01 Andras Bokor added a comment -

        Thanks a lot, Jason Lowe.

        Show
        boky01 Andras Bokor added a comment - Thanks a lot, Jason Lowe .

          People

          • Assignee:
            boky01 Andras Bokor
            Reporter:
            boky01 Andras Bokor
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development