Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12989

Some tests in org.apache.hadoop.fs.shell.find occasionally time out

    Details

    • Hadoop Flags:
      Reviewed

      Description

      An example:

      java.lang.Exception: test timed out after 1000 milliseconds
      	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
      	at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
      	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
      	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1872)
      	at java.lang.Runtime.loadLibrary0(Runtime.java:849)
      	at java.lang.System.loadLibrary(System.java:1088)
      	at sun.security.action.LoadLibraryAction.run(LoadLibraryAction.java:67)
      	at sun.security.action.LoadLibraryAction.run(LoadLibraryAction.java:47)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at java.net.NetworkInterface.<clinit>(NetworkInterface.java:56)
      	at org.apache.htrace.core.TracerId.getBestIpString(TracerId.java:179)
      	at org.apache.htrace.core.TracerId.processShellVar(TracerId.java:145)
      	at org.apache.htrace.core.TracerId.<init>(TracerId.java:116)
      	at org.apache.htrace.core.Tracer$Builder.build(Tracer.java:159)
      	at org.apache.hadoop.fs.FsTracer.get(FsTracer.java:42)
      	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2794)
      	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99)
      	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2837)
      	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2819)
      	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:381)
      	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:180)
      	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:365)
      	at org.apache.hadoop.fs.shell.PathData.<init>(PathData.java:81)
      	at org.apache.hadoop.fs.shell.find.TestName.applyGlob(TestName.java:74)
      
      1. HADOOP-12989.1.patch
        3 kB
        Takashi Ohnishi
      2. HADOOP-12989.2.patch
        20 kB
        Takashi Ohnishi

        Issue Links

          Activity

          Hide
          ajisakaa Akira Ajisaka added a comment -

          Adding the following line in the classes and remove (timeout=1000) from tests seems fine.

          TestRule globalTimeout = new Timeout(10000);
          
          Show
          ajisakaa Akira Ajisaka added a comment - Adding the following line in the classes and remove (timeout=1000) from tests seems fine. TestRule globalTimeout = new Timeout(10000);
          Hide
          xiaochen Xiao Chen added a comment -

          Thanks for creating this Akira. I think 1 second timeout is indeed a little aggressive. +1 for the idea of consolidating a timeout rule and make it 10s. (Noticed this to be a newbie JIRA, so will let anyone interested to take on).

          Show
          xiaochen Xiao Chen added a comment - Thanks for creating this Akira. I think 1 second timeout is indeed a little aggressive. +1 for the idea of consolidating a timeout rule and make it 10s. (Noticed this to be a newbie JIRA, so will let anyone interested to take on).
          Hide
          bwtakacy Takashi Ohnishi added a comment -

          Hi!

          I want to try this.
          I will submit a patch as suggested.

          Show
          bwtakacy Takashi Ohnishi added a comment - Hi! I want to try this. I will submit a patch as suggested.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 37s trunk passed
          +1 compile 5m 55s trunk passed with JDK v1.8.0_77
          +1 compile 6m 42s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 20s trunk passed
          +1 mvnsite 0m 55s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 35s trunk passed
          +1 javadoc 0m 51s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 2s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 40s the patch passed
          +1 compile 5m 45s the patch passed with JDK v1.8.0_77
          +1 javac 5m 45s the patch passed
          +1 compile 6m 39s the patch passed with JDK v1.7.0_95
          +1 javac 6m 39s the patch passed
          +1 checkstyle 0m 20s the patch passed
          +1 mvnsite 0m 54s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 47s the patch passed
          +1 javadoc 0m 52s the patch passed with JDK v1.8.0_77
          +1 javadoc 1m 2s the patch passed with JDK v1.7.0_95
          +1 unit 6m 51s hadoop-common in the patch passed with JDK v1.8.0_77.
          +1 unit 7m 12s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 21s Patch does not generate ASF License warnings.
          58m 8s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798640/HADOOP-12989.1.patch
          JIRA Issue HADOOP-12989
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 19b128bb8e26 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 27b131e
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9087/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9087/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 37s trunk passed +1 compile 5m 55s trunk passed with JDK v1.8.0_77 +1 compile 6m 42s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 20s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 35s trunk passed +1 javadoc 0m 51s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 2s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 40s the patch passed +1 compile 5m 45s the patch passed with JDK v1.8.0_77 +1 javac 5m 45s the patch passed +1 compile 6m 39s the patch passed with JDK v1.7.0_95 +1 javac 6m 39s the patch passed +1 checkstyle 0m 20s the patch passed +1 mvnsite 0m 54s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 52s the patch passed with JDK v1.8.0_77 +1 javadoc 1m 2s the patch passed with JDK v1.7.0_95 +1 unit 6m 51s hadoop-common in the patch passed with JDK v1.8.0_77. +1 unit 7m 12s hadoop-common in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 58m 8s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798640/HADOOP-12989.1.patch JIRA Issue HADOOP-12989 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 19b128bb8e26 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 27b131e Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9087/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9087/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thank you for taking this issue Takashi Ohnishi. Would you add global timeout to other tests in the same directory as well?

          Show
          ajisakaa Akira Ajisaka added a comment - Thank you for taking this issue Takashi Ohnishi . Would you add global timeout to other tests in the same directory as well?
          Hide
          bwtakacy Takashi Ohnishi added a comment -

          Thank you Akira Ajisaka for commenting!

          Would you add global timeout to other tests in the same directory as well?

          Oops. I wrongly recognized that this is only for TestName from the stack trace.
          I will update the patch.

          Show
          bwtakacy Takashi Ohnishi added a comment - Thank you Akira Ajisaka for commenting! Would you add global timeout to other tests in the same directory as well? Oops. I wrongly recognized that this is only for TestName from the stack trace. I will update the patch.
          Hide
          bwtakacy Takashi Ohnishi added a comment -

          Attached the updated patch.

          Show
          bwtakacy Takashi Ohnishi added a comment - Attached the updated patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 8s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          +1 mvninstall 6m 46s trunk passed
          +1 compile 7m 34s trunk passed with JDK v1.8.0_77
          +1 compile 7m 4s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 20s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 35s trunk passed
          +1 javadoc 0m 56s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 0s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 41s the patch passed
          +1 compile 5m 48s the patch passed with JDK v1.8.0_77
          +1 javac 5m 48s the patch passed
          +1 compile 6m 39s the patch passed with JDK v1.7.0_95
          +1 javac 6m 39s the patch passed
          +1 checkstyle 0m 21s the patch passed
          +1 mvnsite 0m 58s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 52s the patch passed
          +1 javadoc 0m 54s the patch passed with JDK v1.8.0_77
          +1 javadoc 1m 4s the patch passed with JDK v1.7.0_95
          +1 unit 7m 59s hadoop-common in the patch passed with JDK v1.8.0_77.
          +1 unit 8m 14s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 21s Patch does not generate ASF License warnings.
          62m 45s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798724/HADOOP-12989.2.patch
          JIRA Issue HADOOP-12989
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c37099212e26 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / df18b6e9
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9093/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9093/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. +1 mvninstall 6m 46s trunk passed +1 compile 7m 34s trunk passed with JDK v1.8.0_77 +1 compile 7m 4s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 20s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 35s trunk passed +1 javadoc 0m 56s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 0s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 41s the patch passed +1 compile 5m 48s the patch passed with JDK v1.8.0_77 +1 javac 5m 48s the patch passed +1 compile 6m 39s the patch passed with JDK v1.7.0_95 +1 javac 6m 39s the patch passed +1 checkstyle 0m 21s the patch passed +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 52s the patch passed +1 javadoc 0m 54s the patch passed with JDK v1.8.0_77 +1 javadoc 1m 4s the patch passed with JDK v1.7.0_95 +1 unit 7m 59s hadoop-common in the patch passed with JDK v1.8.0_77. +1 unit 8m 14s hadoop-common in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 62m 45s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798724/HADOOP-12989.2.patch JIRA Issue HADOOP-12989 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c37099212e26 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / df18b6e9 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9093/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9093/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thank you for updating the patch Takashi Ohnishi! +1, committing this.

          Show
          ajisakaa Akira Ajisaka added a comment - Thank you for updating the patch Takashi Ohnishi ! +1, committing this.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed this to branch-2.7 and above. Thanks Takashi Ohnishi for the contribution!

          Show
          ajisakaa Akira Ajisaka added a comment - Committed this to branch-2.7 and above. Thanks Takashi Ohnishi for the contribution!
          Hide
          ajisakaa Akira Ajisaka added a comment -

          and thanks Xiao for the comment.

          Show
          ajisakaa Akira Ajisaka added a comment - and thanks Xiao for the comment.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9617 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9617/)
          HADOOP-12989. Some tests in org.apache.hadoop.fs.shell.find occasionally (aajisaka: rev 6e6b6dd5aaf93cfb373833cd175ee722d2cb708f)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestResult.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestPrint.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestPrint0.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestIname.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestFilterExpression.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestName.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestFind.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestAnd.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9617 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9617/ ) HADOOP-12989 . Some tests in org.apache.hadoop.fs.shell.find occasionally (aajisaka: rev 6e6b6dd5aaf93cfb373833cd175ee722d2cb708f) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestResult.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestPrint.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestPrint0.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestIname.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestFilterExpression.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestName.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestFind.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/find/TestAnd.java
          Hide
          bwtakacy Takashi Ohnishi added a comment -

          Thank you Akira Ajisaka for committing!

          Show
          bwtakacy Takashi Ohnishi added a comment - Thank you Akira Ajisaka for committing!
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Closing the JIRA as part of 2.7.3 release.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.

            People

            • Assignee:
              bwtakacy Takashi Ohnishi
              Reporter:
              ajisakaa Akira Ajisaka
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development