Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: test
    • Labels:
      None

      Description

      https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/166/testReport/junit/org.apache.hadoop.yarn.server.nodemanager/TestDefaultContainerExecutor/testStartLocalizer/

      java.lang.AssertionError: StartLocalizer failed to copy token file: java.io.FileNotFoundException: File file:/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestDefaultContainerExecutor/tokenDir/test.tokens does not exist
      	at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:645)
      	at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:871)
      	at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:635)
      	at org.apache.hadoop.fs.DelegateToFileSystem.getFileStatus(DelegateToFileSystem.java:126)
      	at org.apache.hadoop.fs.FilterFs.getFileStatus(FilterFs.java:121)
      	at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1167)
      	at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1163)
      	at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
      	at org.apache.hadoop.fs.FileContext.getFileStatus(FileContext.java:1169)
      	at org.apache.hadoop.fs.FileContext$Util.copy(FileContext.java:2076)
      	at org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:452)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
      
      	at org.junit.Assert.fail(Assert.java:88)
      	at org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:462)
      
      1. YARN-5657.001.patch
        2 kB
        Arun Suresh
      2. YARN-5657.002.patch
        2 kB
        Arun Suresh

        Issue Links

          Activity

          Hide
          ajisakaa Akira Ajisaka added a comment -

          Ran git bisect and found YARN-5620 broke this test.

          Show
          ajisakaa Akira Ajisaka added a comment - Ran git bisect and found YARN-5620 broke this test.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          YARN-5620 made this change

          DefaultContainerExecutor.java
             protected void copyFile(Path src, Path dst, String owner) throws IOException {
          -    lfs.util().copy(src, dst);
          +    lfs.util().copy(src, dst, false, true);
             }
          

          but the following code was not updated accordingly.

          TestDefaultContainerExecutor.java
              }).when(mockUtil).copy(any(Path.class), any(Path.class));
          
          Show
          ajisakaa Akira Ajisaka added a comment - YARN-5620 made this change DefaultContainerExecutor.java protected void copyFile(Path src, Path dst, String owner) throws IOException { - lfs.util().copy(src, dst); + lfs.util().copy(src, dst, false , true ); } but the following code was not updated accordingly. TestDefaultContainerExecutor.java }).when(mockUtil).copy(any(Path.class), any(Path.class));
          Hide
          asuresh Arun Suresh added a comment -

          Apologize.. uploading patch

          Show
          asuresh Arun Suresh added a comment - Apologize.. uploading patch
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thank you Arun Suresh. LGTM, +1 pending Jenkins.

          Show
          ajisakaa Akira Ajisaka added a comment - Thank you Arun Suresh . LGTM, +1 pending Jenkins.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 36s trunk passed
          +1 compile 0m 26s trunk passed
          +1 checkstyle 0m 16s trunk passed
          +1 mvnsite 0m 26s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 0m 39s trunk passed
          +1 javadoc 0m 17s trunk passed
          +1 mvninstall 0m 21s the patch passed
          +1 compile 0m 24s the patch passed
          +1 javac 0m 24s the patch passed
          -1 checkstyle 0m 13s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10)
          +1 mvnsite 0m 27s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 58s the patch passed
          +1 javadoc 0m 19s the patch passed
          +1 unit 14m 8s hadoop-yarn-server-nodemanager in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          27m 7s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828882/YARN-5657.001.patch
          JIRA Issue YARN-5657
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 9b531e34fb23 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / b09a03c
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13126/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13126/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13126/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 36s trunk passed +1 compile 0m 26s trunk passed +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 26s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 39s trunk passed +1 javadoc 0m 17s trunk passed +1 mvninstall 0m 21s the patch passed +1 compile 0m 24s the patch passed +1 javac 0m 24s the patch passed -1 checkstyle 0m 13s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10) +1 mvnsite 0m 27s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 58s the patch passed +1 javadoc 0m 19s the patch passed +1 unit 14m 8s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 27m 7s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828882/YARN-5657.001.patch JIRA Issue YARN-5657 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9b531e34fb23 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b09a03c Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13126/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13126/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13126/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          asuresh Arun Suresh added a comment - - edited

          hmmm... not sure how to fix this checkstyle...
          Akira Ajisaka, think this can be ignored as it conforms to similar code in the file ?

          Show
          asuresh Arun Suresh added a comment - - edited hmmm... not sure how to fix this checkstyle... Akira Ajisaka , think this can be ignored as it conforms to similar code in the file ?
          Hide
          ajisakaa Akira Ajisaka added a comment -

          This seems to be false-positive. Anyway, the following code fixes the warning.

                }}).when(mockUtil).copy(any(Path.class), any(Path.class),
                  anyBoolean(), anyBoolean());
          
          Show
          ajisakaa Akira Ajisaka added a comment - This seems to be false-positive. Anyway, the following code fixes the warning. }}).when(mockUtil).copy(any(Path.class), any(Path.class), anyBoolean(), anyBoolean());
          Hide
          asuresh Arun Suresh added a comment -

          fixing checkstyle...

          Show
          asuresh Arun Suresh added a comment - fixing checkstyle...
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 22s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 47s trunk passed
          +1 compile 0m 26s trunk passed
          +1 checkstyle 0m 16s trunk passed
          +1 mvnsite 0m 26s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 0m 41s trunk passed
          +1 javadoc 0m 17s trunk passed
          +1 mvninstall 0m 21s the patch passed
          +1 compile 0m 23s the patch passed
          +1 javac 0m 23s the patch passed
          +1 checkstyle 0m 14s the patch passed
          +1 mvnsite 0m 24s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 48s the patch passed
          +1 javadoc 0m 15s the patch passed
          +1 unit 14m 13s hadoop-yarn-server-nodemanager in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          27m 8s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829059/YARN-5657.002.patch
          JIRA Issue YARN-5657
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux a2c5c84a8ecc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 4174b97
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13136/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13136/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 47s trunk passed +1 compile 0m 26s trunk passed +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 26s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 41s trunk passed +1 javadoc 0m 17s trunk passed +1 mvninstall 0m 21s the patch passed +1 compile 0m 23s the patch passed +1 javac 0m 23s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 24s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 48s the patch passed +1 javadoc 0m 15s the patch passed +1 unit 14m 13s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 27m 8s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829059/YARN-5657.002.patch JIRA Issue YARN-5657 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a2c5c84a8ecc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4174b97 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13136/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13136/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          +1, thanks Arun Suresh for updating the patch.

          Show
          ajisakaa Akira Ajisaka added a comment - +1, thanks Arun Suresh for updating the patch.
          Hide
          asuresh Arun Suresh added a comment -

          thanks for raising this Akira Ajisaka and the reviews..
          Committing this shortly

          Show
          asuresh Arun Suresh added a comment - thanks for raising this Akira Ajisaka and the reviews.. Committing this shortly
          Hide
          asuresh Arun Suresh added a comment -

          Committed to trunk and branch-2

          Show
          asuresh Arun Suresh added a comment - Committed to trunk and branch-2
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10455 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10455/)
          YARN-5657. Fix TestDefaultContainerExecutor. (asuresh) (arun suresh: rev f67237cbe7bc48a1b9088e990800b37529f1db2a)

          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDefaultContainerExecutor.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10455 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10455/ ) YARN-5657 . Fix TestDefaultContainerExecutor. (asuresh) (arun suresh: rev f67237cbe7bc48a1b9088e990800b37529f1db2a) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDefaultContainerExecutor.java

            People

            • Assignee:
              asuresh Arun Suresh
              Reporter:
              ajisakaa Akira Ajisaka
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development