Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5752

TestLocalResourcesTrackerImpl#testLocalResourceCache times out

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      java.lang.Exception: test timed out after 1000 milliseconds
      	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:133)
      	at java.io.OutputStreamWriter.write(OutputStreamWriter.java:220)
      	at java.io.Writer.write(Writer.java:157)
      	at org.apache.log4j.helpers.QuietWriter.write(QuietWriter.java:48)
      	at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:310)
      	at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
      	at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
      	at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
      	at org.apache.log4j.Category.callAppenders(Category.java:206)
      	at org.apache.log4j.Category.forcedLog(Category.java:391)
      	at org.apache.log4j.Category.log(Category.java:856)
      	at org.apache.commons.logging.impl.Log4JLogger.info(Log4JLogger.java:176)
      	at org.apache.hadoop.yarn.event.AsyncDispatcher.register(AsyncDispatcher.java:209)
      	at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.TestLocalResourcesTrackerImpl.testLocalResourceCache(TestLocalResourcesTrackerImpl.java:258)
      
      1. YARN-5752.002.patch
        1 kB
        Eric Badger
      2. YARN-5752.001.patch
        1 kB
        Eric Badger

        Activity

        Hide
        ebadger Eric Badger added a comment -

        1 second timeout is much too low. It is easy to reproduce timeout failures by increasing the load on the machine where the test is running. Attaching a patch to increase the timeout to 10 seconds.

        Show
        ebadger Eric Badger added a comment - 1 second timeout is much too low. It is easy to reproduce timeout failures by increasing the load on the machine where the test is running. Attaching a patch to increase the timeout to 10 seconds.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 51s trunk passed
        +1 compile 0m 26s trunk passed
        +1 checkstyle 0m 16s trunk passed
        +1 mvnsite 0m 26s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 42s trunk passed
        +1 javadoc 0m 17s trunk passed
        +1 mvninstall 0m 21s the patch passed
        +1 compile 0m 25s the patch passed
        +1 javac 0m 25s the patch passed
        +1 checkstyle 0m 14s the patch passed
        +1 mvnsite 0m 25s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 49s the patch passed
        +1 javadoc 0m 13s the patch passed
        +1 unit 14m 50s hadoop-yarn-server-nodemanager in the patch passed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        27m 39s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834177/YARN-5752.001.patch
        JIRA Issue YARN-5752
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux faea22e33020 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c5573e6
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13439/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13439/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 51s trunk passed +1 compile 0m 26s trunk passed +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 26s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 42s trunk passed +1 javadoc 0m 17s trunk passed +1 mvninstall 0m 21s the patch passed +1 compile 0m 25s the patch passed +1 javac 0m 25s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 49s the patch passed +1 javadoc 0m 13s the patch passed +1 unit 14m 50s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 27m 39s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834177/YARN-5752.001.patch JIRA Issue YARN-5752 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux faea22e33020 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c5573e6 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13439/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13439/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Thank you, Eric Badger for the fix! You might want to consider changing the timeout value on testHierarchicalLocalCacheDirectories to 10000, too. I checked and the runtime is comparable to testLocalResourceCache.

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Thank you, Eric Badger for the fix! You might want to consider changing the timeout value on testHierarchicalLocalCacheDirectories to 10000, too. I checked and the runtime is comparable to testLocalResourceCache.
        Hide
        ebadger Eric Badger added a comment -

        Miklos Szegedi, yea the 100 second timeout on testHierarchicalLocalCacheDirectories does seem excessive given its consistent sub-1 second runtime. Attaching patch to set timeout to 10 seconds for both.

        Show
        ebadger Eric Badger added a comment - Miklos Szegedi , yea the 100 second timeout on testHierarchicalLocalCacheDirectories does seem excessive given its consistent sub-1 second runtime. Attaching patch to set timeout to 10 seconds for both.
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        +1 non-binding. The patch looks good to me. Thank you, Eric Badger!

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - +1 non-binding. The patch looks good to me. Thank you, Eric Badger !
        Hide
        varun_saxena Varun Saxena added a comment -

        +1.
        Committing this pending Jenkins.

        Show
        varun_saxena Varun Saxena added a comment - +1. Committing this pending Jenkins.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 23s trunk passed
        +1 compile 0m 31s trunk passed
        +1 checkstyle 0m 18s trunk passed
        +1 mvnsite 0m 31s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 0m 51s trunk passed
        +1 javadoc 0m 20s trunk passed
        +1 mvninstall 0m 27s the patch passed
        +1 compile 0m 28s the patch passed
        +1 javac 0m 28s the patch passed
        +1 checkstyle 0m 15s the patch passed
        +1 mvnsite 0m 25s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 46s the patch passed
        +1 javadoc 0m 16s the patch passed
        +1 unit 15m 19s hadoop-yarn-server-nodemanager in the patch passed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        31m 8s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue YARN-5752
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12835332/YARN-5752.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 7c451eb5e933 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a895c66
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13540/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13540/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 23s trunk passed +1 compile 0m 31s trunk passed +1 checkstyle 0m 18s trunk passed +1 mvnsite 0m 31s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 0m 51s trunk passed +1 javadoc 0m 20s trunk passed +1 mvninstall 0m 27s the patch passed +1 compile 0m 28s the patch passed +1 javac 0m 28s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 46s the patch passed +1 javadoc 0m 16s the patch passed +1 unit 15m 19s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 31m 8s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue YARN-5752 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12835332/YARN-5752.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7c451eb5e933 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a895c66 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13540/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13540/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        varun_saxena Varun Saxena added a comment -

        Committed to trunk, branch-2.
        Thanks Eric Badger for your contribution.
        Thanks Miklos Szegedi for review.

        Show
        varun_saxena Varun Saxena added a comment - Committed to trunk, branch-2. Thanks Eric Badger for your contribution. Thanks Miklos Szegedi for review.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10699 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10699/)
        YARN-5752. TestLocalResourcesTrackerImpl#testLocalResourceCache times (varunsaxena: rev 5c39a419a6429a6ec9d510a57c19a04808d3b9cc)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResourcesTrackerImpl.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10699 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10699/ ) YARN-5752 . TestLocalResourcesTrackerImpl#testLocalResourceCache times (varunsaxena: rev 5c39a419a6429a6ec9d510a57c19a04808d3b9cc) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResourcesTrackerImpl.java

          People

          • Assignee:
            ebadger Eric Badger
            Reporter:
            ebadger Eric Badger
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development