Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6217

TestLocalCacheDirectoryManager test timeout is too aggressive

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4, 2.8.2
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      TestLocalCacheDirectoryManager#testDirectoryStateChangeFromFullToNonFull has only a one second timeout. If the test machine hits an I/O hiccup it can fail. The test timeout is too aggressive, and I question whether this test even needs an explicit timeout specified.

      1. YARN-6217.001.patch
        2 kB
        Miklos Szegedi
      2. YARN-6217.000.patch
        1 kB
        Miklos Szegedi

        Activity

        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - 2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Thank you for the commit, Jason Lowe, and for the review Yufei Gu!

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Thank you for the commit, Jason Lowe , and for the review Yufei Gu !
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11422 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11422/)
        YARN-6217. TestLocalCacheDirectoryManager test timeout is too (jlowe: rev 4a8e3045027036afebbcb80f23b7a2886e56c255)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalCacheDirectoryManager.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11422 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11422/ ) YARN-6217 . TestLocalCacheDirectoryManager test timeout is too (jlowe: rev 4a8e3045027036afebbcb80f23b7a2886e56c255) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalCacheDirectoryManager.java
        Hide
        jlowe Jason Lowe added a comment -

        Thanks to Miklos Szegedi for the contribution and to Yufei Gu for additional review! I committed this to trunk, branch-2, and branch-2.8.

        Show
        jlowe Jason Lowe added a comment - Thanks to Miklos Szegedi for the contribution and to Yufei Gu for additional review! I committed this to trunk, branch-2, and branch-2.8.
        Hide
        jlowe Jason Lowe added a comment -

        +1 lgtm. Committing this.

        Show
        jlowe Jason Lowe added a comment - +1 lgtm. Committing this.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 13m 56s trunk passed
        +1 compile 0m 27s trunk passed
        +1 checkstyle 0m 18s trunk passed
        +1 mvnsite 0m 25s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 39s trunk passed
        +1 javadoc 0m 17s trunk passed
        +1 mvninstall 0m 23s the patch passed
        +1 compile 0m 23s the patch passed
        +1 javac 0m 23s the patch passed
        +1 checkstyle 0m 15s the patch passed
        +1 mvnsite 0m 23s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 1s The patch has no whitespace issues.
        +1 findbugs 0m 45s the patch passed
        +1 javadoc 0m 15s the patch passed
        +1 unit 13m 4s hadoop-yarn-server-nodemanager in the patch passed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        33m 50s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-6217
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859332/YARN-6217.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 4a24adfb4549 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 86035c1
        Default Java 1.8.0_121
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15319/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/15319/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 13m 56s trunk passed +1 compile 0m 27s trunk passed +1 checkstyle 0m 18s trunk passed +1 mvnsite 0m 25s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 39s trunk passed +1 javadoc 0m 17s trunk passed +1 mvninstall 0m 23s the patch passed +1 compile 0m 23s the patch passed +1 javac 0m 23s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 0m 23s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 1s The patch has no whitespace issues. +1 findbugs 0m 45s the patch passed +1 javadoc 0m 15s the patch passed +1 unit 13m 4s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 33m 50s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6217 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859332/YARN-6217.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4a24adfb4549 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 86035c1 Default Java 1.8.0_121 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15319/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/15319/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        yufeigu Yufei Gu added a comment -

        +1 (non-binding)

        Show
        yufeigu Yufei Gu added a comment - +1 (non-binding)
        Hide
        jlowe Jason Lowe added a comment -

        I tend to agree. Originally there was an edict to put a timeout on each test because the build wasn't doing a good job of handling tests that timed out. However that has since been fixed. An explicit test timeout still makes a lot of sense when the test has a good chance of deadlocking when broken (e.g.: needing to carefully synchronize a number of threads, wait for barriers, doing a polling loop, etc.), but I don't think that's the case with the tests here.

        Show
        jlowe Jason Lowe added a comment - I tend to agree. Originally there was an edict to put a timeout on each test because the build wasn't doing a good job of handling tests that timed out. However that has since been fixed. An explicit test timeout still makes a lot of sense when the test has a good chance of deadlocking when broken (e.g.: needing to carefully synchronize a number of threads, wait for barriers, doing a polling loop, etc.), but I don't think that's the case with the tests here.
        Hide
        yufeigu Yufei Gu added a comment -

        If we remove the timeout in this test, may be consistent to remove timeout for other two tests, testHierarchicalSubDirectoryCreation and testMinimumPerDirectoryFileLimit.

        Show
        yufeigu Yufei Gu added a comment - If we remove the timeout in this test, may be consistent to remove timeout for other two tests, testHierarchicalSubDirectoryCreation and testMinimumPerDirectoryFileLimit .
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 24s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 15m 34s trunk passed
        +1 compile 0m 37s trunk passed
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 0m 36s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 1m 1s trunk passed
        +1 javadoc 0m 24s trunk passed
        +1 mvninstall 0m 33s the patch passed
        +1 compile 0m 34s the patch passed
        +1 javac 0m 34s the patch passed
        +1 checkstyle 0m 21s the patch passed
        +1 mvnsite 0m 34s the patch passed
        +1 mvneclipse 0m 15s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 56s the patch passed
        +1 javadoc 0m 18s the patch passed
        +1 unit 13m 20s hadoop-yarn-server-nodemanager in the patch passed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        37m 48s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-6217
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859222/YARN-6217.000.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 1ccd7576e29b 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 7536815
        Default Java 1.8.0_121
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15307/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/15307/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 24s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 15m 34s trunk passed +1 compile 0m 37s trunk passed +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 36s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 1m 1s trunk passed +1 javadoc 0m 24s trunk passed +1 mvninstall 0m 33s the patch passed +1 compile 0m 34s the patch passed +1 javac 0m 34s the patch passed +1 checkstyle 0m 21s the patch passed +1 mvnsite 0m 34s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 56s the patch passed +1 javadoc 0m 18s the patch passed +1 unit 13m 20s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 37m 48s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6217 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859222/YARN-6217.000.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1ccd7576e29b 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7536815 Default Java 1.8.0_121 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15307/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/15307/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Attaching suggested patch.

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Attaching suggested patch.

          People

          • Assignee:
            miklos.szegedi@cloudera.com Miklos Szegedi
            Reporter:
            jlowe Jason Lowe
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development