Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha2
    • Component/s: timelineserver
    • Labels:
      None

      Description

      TestAHSWebServices.testContainerLogsForFinishedApps is failing.
      https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/176/testReport/

      java.lang.AssertionError: null
      	at org.junit.Assert.fail(Assert.java:86)
      	at org.junit.Assert.assertTrue(Assert.java:41)
      	at org.junit.Assert.assertTrue(Assert.java:52)
      	at org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.createContainerLogInLocalDir(TestAHSWebServices.java:675)
      	at org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.testContainerLogsForFinishedApps(TestAHSWebServices.java:581)
      
      java.lang.AssertionError: null
      	at org.junit.Assert.fail(Assert.java:86)
      	at org.junit.Assert.assertTrue(Assert.java:41)
      	at org.junit.Assert.assertTrue(Assert.java:52)
      	at org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.testContainerLogsForFinishedApps(TestAHSWebServices.java:519)
      
      1. YARN-5679.01.patch
        1 kB
        Akira Ajisaka
      2. YARN-5679.02.patch
        2 kB
        Akira Ajisaka

        Issue Links

          Activity

          Hide
          ajisakaa Akira Ajisaka added a comment -

          I ran git bisect and found HADOOP-13440 broke this test.

          Show
          ajisakaa Akira Ajisaka added a comment - I ran git bisect and found HADOOP-13440 broke this test.
          Hide
          ajisakaa Akira Ajisaka added a comment -
          TestAHSWebServices.java
              uploadContainerLogIntoRemoteDir(ugi, conf, rootLogDirList, nodeId2,
                  containerId100, path, fs);
          

          This creates LogWriter instance and calls FileContext.setUMask(137)

          AggregatedLogFormat.java
              public LogWriter(final Configuration conf, final Path remoteAppLogFile,
                  UserGroupInformation userUgi) throws IOException {
                try {
                  this.fsDataOStream =
                      userUgi.doAs(new PrivilegedExceptionAction<FSDataOutputStream>() {
                        @Override
                        public FSDataOutputStream run() throws Exception {
                          fc = FileContext.getFileContext(remoteAppLogFile.toUri(), conf);
                          fc.setUMask(APP_LOG_FILE_UMASK);
          

          After HADOOP-13140, RawFileContext reacts on changing umask via FileContext, so the umask of RawFileContext is set to 137 as well. Therefore the following code creates appLogsDir with umask 137, so the permission of appLogsDir becomes 642.

          TestAHSWebServices
              assertTrue(fs.mkdirs(appLogsDir));
          

          Finally the following code fails because appLogsDir is not executable.

          TestAHSWebServices
              createContainerLogInLocalDir(appLogsDir, containerId1ForApp100, fs,
                  fileName, ("Hello." + containerId1ForApp100));
          
          Show
          ajisakaa Akira Ajisaka added a comment - TestAHSWebServices.java uploadContainerLogIntoRemoteDir(ugi, conf, rootLogDirList, nodeId2, containerId100, path, fs); This creates LogWriter instance and calls FileContext.setUMask(137) AggregatedLogFormat.java public LogWriter( final Configuration conf, final Path remoteAppLogFile, UserGroupInformation userUgi) throws IOException { try { this .fsDataOStream = userUgi.doAs( new PrivilegedExceptionAction<FSDataOutputStream>() { @Override public FSDataOutputStream run() throws Exception { fc = FileContext.getFileContext(remoteAppLogFile.toUri(), conf); fc.setUMask(APP_LOG_FILE_UMASK); After HADOOP-13140 , RawFileContext reacts on changing umask via FileContext, so the umask of RawFileContext is set to 137 as well. Therefore the following code creates appLogsDir with umask 137, so the permission of appLogsDir becomes 642. TestAHSWebServices assertTrue(fs.mkdirs(appLogsDir)); Finally the following code fails because appLogsDir is not executable. TestAHSWebServices createContainerLogInLocalDir(appLogsDir, containerId1ForApp100, fs, fileName, ( "Hello." + containerId1ForApp100));
          Hide
          ajisakaa Akira Ajisaka added a comment -

          I uploaded a simple patch to reset the umask.

          Show
          ajisakaa Akira Ajisaka added a comment - I uploaded a simple patch to reset the umask.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 22s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 45s trunk passed
          +1 compile 0m 25s trunk passed
          +1 checkstyle 0m 18s trunk passed
          +1 mvnsite 0m 30s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 0m 54s trunk passed
          +1 javadoc 0m 27s trunk passed
          +1 mvninstall 0m 24s the patch passed
          +1 compile 0m 23s the patch passed
          +1 javac 0m 23s the patch passed
          +1 checkstyle 0m 14s the patch passed
          +1 mvnsite 0m 27s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 58s the patch passed
          +1 javadoc 0m 24s the patch passed
          +1 unit 2m 15s hadoop-yarn-common in the patch passed.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          16m 0s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832643/YARN-5679.01.patch
          JIRA Issue YARN-5679
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux e959dbbef207 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 96b1266
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13350/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13350/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 45s trunk passed +1 compile 0m 25s trunk passed +1 checkstyle 0m 18s trunk passed +1 mvnsite 0m 30s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 54s trunk passed +1 javadoc 0m 27s trunk passed +1 mvninstall 0m 24s the patch passed +1 compile 0m 23s the patch passed +1 javac 0m 23s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 27s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 58s the patch passed +1 javadoc 0m 24s the patch passed +1 unit 2m 15s hadoop-yarn-common in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 16m 0s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832643/YARN-5679.01.patch JIRA Issue YARN-5679 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e959dbbef207 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 96b1266 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13350/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/13350/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          I ran TestAHSWebServices successfully with the patch locally.

          Show
          ajisakaa Akira Ajisaka added a comment - I ran TestAHSWebServices successfully with the patch locally.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Hi Steve Loughran, would you please review this patch? This test is failing more than 2 months.

          Show
          ajisakaa Akira Ajisaka added a comment - Hi Steve Loughran , would you please review this patch? This test is failing more than 2 months.
          Hide
          miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

          I think there is an issue there. setUMask will change the umask in the Configuration object, which may be picked up by any other thread in the process using the same configuration object before it is reverted.
          One option is to create a private configuration object to change the umask, the other is to apply the both umasks to a permission object and create the file with this permission object set.

          Show
          miklos.szegedi@cloudera.com Miklos Szegedi added a comment - I think there is an issue there. setUMask will change the umask in the Configuration object, which may be picked up by any other thread in the process using the same configuration object before it is reverted. One option is to create a private configuration object to change the umask, the other is to apply the both umasks to a permission object and create the file with this permission object set.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thanks Miklos Szegedi for the comment!

          setUMask will change the umask in the Configuration object, which may be picked up by any other thread in the process using the same configuration object before it is reverted.

          Agreed.

          One option is to create a private configuration object to change the umask, the other is to apply the both umasks to a permission object and create the file with this permission object set.

          I chose the latter option because creating a private configuration object for each log costs too much.

          Show
          ajisakaa Akira Ajisaka added a comment - Thanks Miklos Szegedi for the comment! setUMask will change the umask in the Configuration object, which may be picked up by any other thread in the process using the same configuration object before it is reverted. Agreed. One option is to create a private configuration object to change the umask, the other is to apply the both umasks to a permission object and create the file with this permission object set. I chose the latter option because creating a private configuration object for each log costs too much.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 3s trunk passed
          +1 compile 0m 26s trunk passed
          +1 checkstyle 0m 17s trunk passed
          +1 mvnsite 0m 30s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 0m 56s trunk passed
          +1 javadoc 0m 26s trunk passed
          +1 mvninstall 0m 25s the patch passed
          +1 compile 0m 23s the patch passed
          +1 javac 0m 23s the patch passed
          +1 checkstyle 0m 15s the patch passed
          +1 mvnsite 0m 27s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 2s the patch passed
          +1 javadoc 0m 25s the patch passed
          +1 unit 2m 15s hadoop-yarn-common in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          16m 22s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833493/YARN-5679.02.patch
          JIRA Issue YARN-5679
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux f3508522d62a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 391ce53
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13403/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13403/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 3s trunk passed +1 compile 0m 26s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 30s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 56s trunk passed +1 javadoc 0m 26s trunk passed +1 mvninstall 0m 25s the patch passed +1 compile 0m 23s the patch passed +1 javac 0m 23s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 0m 27s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 2s the patch passed +1 javadoc 0m 25s the patch passed +1 unit 2m 15s hadoop-yarn-common in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 16m 22s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833493/YARN-5679.02.patch JIRA Issue YARN-5679 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f3508522d62a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 391ce53 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13403/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/13403/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

          +1 (Non binding) Thank you Akira Ajisaka!

          Show
          miklos.szegedi@cloudera.com Miklos Szegedi added a comment - +1 (Non binding) Thank you Akira Ajisaka !
          Hide
          rkanter Robert Kanter added a comment -

          +1

          Show
          rkanter Robert Kanter added a comment - +1
          Hide
          rkanter Robert Kanter added a comment -

          Thanks Akira Ajisaka and Miklos Szegedi. Committed to trunk!

          Show
          rkanter Robert Kanter added a comment - Thanks Akira Ajisaka and Miklos Szegedi . Committed to trunk!
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10657 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10657/)
          YARN-5679. TestAHSWebServices is failing (ajisakaa via rkanter) (rkanter: rev 23d7d53a41c6a59efa4093ae563c45af911005d4)

          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/AggregatedLogFormat.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10657 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10657/ ) YARN-5679 . TestAHSWebServices is failing (ajisakaa via rkanter) (rkanter: rev 23d7d53a41c6a59efa4093ae563c45af911005d4) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/AggregatedLogFormat.java
          Hide
          ajisakaa Akira Ajisaka added a comment -
          Show
          ajisakaa Akira Ajisaka added a comment - Thanks Miklos Szegedi and Robert Kanter !

            People

            • Assignee:
              ajisakaa Akira Ajisaka
              Reporter:
              ajisakaa Akira Ajisaka
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development