Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3971

Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: resourcemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Steps to reproduce

      1. Create label x,y
      2. Delete label x,y
      3. Create label x,y add capacity scheduler xml for labels x and y too
      4. Restart RM

      Both RM will become Standby.

      Since below exception is thrown on FileSystemNodeLabelsStore#recover

      2015-07-23 14:03:33,627 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager failed in state STARTED; cause: java.io.IOException: Cannot remove label=x, because queue=a1 is using this label. Please remove label on queue before remove the label
      java.io.IOException: Cannot remove label=x, because queue=a1 is using this label. Please remove label on queue before remove the label
              at org.apache.hadoop.yarn.server.resourcemanager.nodelabels.RMNodeLabelsManager.checkRemoveFromClusterNodeLabelsOfQueue(RMNodeLabelsManager.java:104)
              at org.apache.hadoop.yarn.server.resourcemanager.nodelabels.RMNodeLabelsManager.removeFromClusterNodeLabels(RMNodeLabelsManager.java:118)
              at org.apache.hadoop.yarn.nodelabels.FileSystemNodeLabelsStore.recover(FileSystemNodeLabelsStore.java:221)
              at org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.initNodeLabelStore(CommonNodeLabelsManager.java:232)
              at org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.serviceStart(CommonNodeLabelsManager.java:245)
              at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
              at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
              at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:587)
              at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
              at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:964)
              at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1005)
              at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1001)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:422)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1666)
              at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1001)
              at org.apache.hadoop.yarn.server.resourcemanager.AdminService.transitionToActive(AdminService.java:312)
              at org.apache.hadoop.yarn.server.resourcemanager.EmbeddedElectorService.becomeActive(EmbeddedElectorService.java:126)
              at org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:832)
              at org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:422)
              at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:599)
              at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
      
      
      1. 0001-YARN-3971.patch
        3 kB
        Bibin A Chundatt
      2. 0002-YARN-3971.patch
        5 kB
        Bibin A Chundatt
      3. 0003-YARN-3971.patch
        5 kB
        Bibin A Chundatt
      4. 0004-YARN-3971.patch
        5 kB
        Bibin A Chundatt
      5. 0005-YARN-3971.001.addendum.patch
        9 kB
        Bibin A Chundatt
      6. 0005-YARN-3971.addendum.patch
        9 kB
        Bibin A Chundatt
      7. 0005-YARN-3971.patch
        5 kB
        Wangda Tan

        Issue Links

          Activity

          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          1) Don't check checkRemoveFromClusterNodeLabelsOfQueue when replaying edit logs
          2) To do 1), you may need to create a local flag in CommonNodeLabelsManager. When doing initNodeLabelStore, the flag is true to indicate it's recovering.

          Wangda Tan Handled recovery as per you offline comments.

          Show
          bibinchundatt Bibin A Chundatt added a comment - 1) Don't check checkRemoveFromClusterNodeLabelsOfQueue when replaying edit logs 2) To do 1), you may need to create a local flag in CommonNodeLabelsManager. When doing initNodeLabelStore , the flag is true to indicate it's recovering. Wangda Tan Handled recovery as per you offline comments.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 12s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 56s There were no new javac warning messages.
          +1 javadoc 10m 2s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 20s The applied patch generated 1 new checkstyle issues (total was 32, now 33).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 25s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 3m 6s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 2m 1s Tests passed in hadoop-yarn-common.
          +1 yarn tests 52m 10s Tests passed in hadoop-yarn-server-resourcemanager.
              97m 24s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12747000/0001-YARN-3971.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 206d493
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8649/artifact/patchprocess/diffcheckstylehadoop-yarn-common.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8649/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8649/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8649/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8649/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 12s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 56s There were no new javac warning messages. +1 javadoc 10m 2s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 20s The applied patch generated 1 new checkstyle issues (total was 32, now 33). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 25s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 3m 6s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 2m 1s Tests passed in hadoop-yarn-common. +1 yarn tests 52m 10s Tests passed in hadoop-yarn-server-resourcemanager.     97m 24s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12747000/0001-YARN-3971.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 206d493 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8649/artifact/patchprocess/diffcheckstylehadoop-yarn-common.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8649/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8649/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8649/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8649/console This message was automatically generated.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Attaching patch with update and testcase.
          Wangda Tan Please review patch attached.

          Show
          bibinchundatt Bibin A Chundatt added a comment - Attaching patch with update and testcase. Wangda Tan Please review patch attached.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 35s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 46s There were no new javac warning messages.
          +1 javadoc 9m 44s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 19s The applied patch generated 1 new checkstyle issues (total was 31, now 32).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 22s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 3m 0s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common.
          -1 yarn tests 52m 58s Tests failed in hadoop-yarn-server-resourcemanager.
              96m 55s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12747193/0002-YARN-3971.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 156f24e
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8670/artifact/patchprocess/diffcheckstylehadoop-yarn-common.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8670/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8670/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8670/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8670/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 35s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 46s There were no new javac warning messages. +1 javadoc 9m 44s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 19s The applied patch generated 1 new checkstyle issues (total was 31, now 32). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 22s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 3m 0s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common. -1 yarn tests 52m 58s Tests failed in hadoop-yarn-server-resourcemanager.     96m 55s   Reason Tests Failed unit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12747193/0002-YARN-3971.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 156f24e checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8670/artifact/patchprocess/diffcheckstylehadoop-yarn-common.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8670/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8670/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8670/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8670/console This message was automatically generated.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Testcase failure is unrelated. Verified locally testcase is passing.
          Fixed checkstyle

          Show
          bibinchundatt Bibin A Chundatt added a comment - Testcase failure is unrelated. Verified locally testcase is passing. Fixed checkstyle
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 46s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 48s There were no new javac warning messages.
          +1 javadoc 9m 42s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 44s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 23s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 3m 1s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 1m 57s Tests passed in hadoop-yarn-common.
          +1 yarn tests 52m 20s Tests passed in hadoop-yarn-server-resourcemanager.
              96m 38s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12747203/0003-YARN-3971.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 156f24e
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8671/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8671/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8671/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8671/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 46s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 48s There were no new javac warning messages. +1 javadoc 9m 42s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 44s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 23s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 3m 1s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 1m 57s Tests passed in hadoop-yarn-common. +1 yarn tests 52m 20s Tests passed in hadoop-yarn-server-resourcemanager.     96m 38s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12747203/0003-YARN-3971.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 156f24e hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8671/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8671/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8671/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8671/console This message was automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Bibin A Chundatt.
          Thanks for working on this. I think one simpler solution for this is, we can leverage AbstractService#getServiceState. Doing remove label check only when state == STARTED.

          Thoughts?

          Show
          leftnoteasy Wangda Tan added a comment - Bibin A Chundatt . Thanks for working on this. I think one simpler solution for this is, we can leverage AbstractService#getServiceState . Doing remove label check only when state == STARTED. Thoughts?
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Wangda Tan Thank you for review. I agree with you. Updated patch as per you comments and also testcase updated.

          Show
          bibinchundatt Bibin A Chundatt added a comment - Wangda Tan Thank you for review. I agree with you. Updated patch as per you comments and also testcase updated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 46s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 8m 26s There were no new javac warning messages.
          +1 javadoc 10m 9s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 48s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 19s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 27s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 yarn tests 53m 31s Tests failed in hadoop-yarn-server-resourcemanager.
              94m 24s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12747403/0004-YARN-3971.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / f36835f
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8680/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8680/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8680/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 46s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 26s There were no new javac warning messages. +1 javadoc 10m 9s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 48s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 19s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 27s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 yarn tests 53m 31s Tests failed in hadoop-yarn-server-resourcemanager.     94m 24s   Reason Tests Failed unit tests hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12747403/0004-YARN-3971.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f36835f hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8680/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8680/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8680/console This message was automatically generated.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Test case failure is not related to this patch

          Show
          bibinchundatt Bibin A Chundatt added a comment - Test case failure is not related to this patch
          Hide
          leftnoteasy Wangda Tan added a comment -

          Looks good, committing... will add some comment to the change before commit.

          Show
          leftnoteasy Wangda Tan added a comment - Looks good, committing... will add some comment to the change before commit.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks Bibin A Chundatt, committed to trunk/branch-2!

          Show
          leftnoteasy Wangda Tan added a comment - Thanks Bibin A Chundatt , committed to trunk/branch-2!
          Hide
          leftnoteasy Wangda Tan added a comment -

          Attached latest patch committed to trunk.

          Show
          leftnoteasy Wangda Tan added a comment - Attached latest patch committed to trunk.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Thanks Wangda Tan for review and committing patch

          Show
          bibinchundatt Bibin A Chundatt added a comment - Thanks Wangda Tan for review and committing patch
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #8245 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8245/)
          YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) (wangda: rev 91b42e7d6e8169feb54dbe3715a3269eea6153a9)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #8245 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8245/ ) YARN-3971 . Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) (wangda: rev 91b42e7d6e8169feb54dbe3715a3269eea6153a9) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #273 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/273/)
          YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) (wangda: rev 91b42e7d6e8169feb54dbe3715a3269eea6153a9)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #273 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/273/ ) YARN-3971 . Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) (wangda: rev 91b42e7d6e8169feb54dbe3715a3269eea6153a9) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #1003 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1003/)
          YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) (wangda: rev 91b42e7d6e8169feb54dbe3715a3269eea6153a9)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1003 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1003/ ) YARN-3971 . Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) (wangda: rev 91b42e7d6e8169feb54dbe3715a3269eea6153a9) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #262 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/262/)
          YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) (wangda: rev 91b42e7d6e8169feb54dbe3715a3269eea6153a9)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #262 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/262/ ) YARN-3971 . Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) (wangda: rev 91b42e7d6e8169feb54dbe3715a3269eea6153a9) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2219 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2219/)
          YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) (wangda: rev 91b42e7d6e8169feb54dbe3715a3269eea6153a9)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2219 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2219/ ) YARN-3971 . Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) (wangda: rev 91b42e7d6e8169feb54dbe3715a3269eea6153a9) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2200 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2200/)
          YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) (wangda: rev 91b42e7d6e8169feb54dbe3715a3269eea6153a9)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2200 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2200/ ) YARN-3971 . Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) (wangda: rev 91b42e7d6e8169feb54dbe3715a3269eea6153a9) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #270 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/270/)
          YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) (wangda: rev 91b42e7d6e8169feb54dbe3715a3269eea6153a9)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #270 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/270/ ) YARN-3971 . Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) (wangda: rev 91b42e7d6e8169feb54dbe3715a3269eea6153a9) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Tan, Wangda/Naganarasimha G R
          The issue still exists can we reopen the jira so that will provide an updated patch.

          The check done last time was for checking the ServiceState=STARTED . But always the service state will be STARTED when FileSystemNodeLabelsStore#recover is called and not in INIT since the call is from AbstractService.start() and before the serviceStart() is done the state is set to STARTED.

           synchronized (stateChangeLock) {
                if (stateModel.enterState(STATE.STARTED) != STATE.STARTED) {
                  try {
                    startTime = System.currentTimeMillis();
                    serviceStart();
          
          }
          

          stateModel.enterState(STATE.STARTED) its directly set to SERVICE.STATE=STARTED

          Show
          bibinchundatt Bibin A Chundatt added a comment - Tan, Wangda / Naganarasimha G R The issue still exists can we reopen the jira so that will provide an updated patch. The check done last time was for checking the ServiceState=STARTED . But always the service state will be STARTED when FileSystemNodeLabelsStore#recover is called and not in INIT since the call is from AbstractService.start() and before the serviceStart() is done the state is set to STARTED. synchronized (stateChangeLock) { if (stateModel.enterState(STATE.STARTED) != STATE.STARTED) { try { startTime = System.currentTimeMillis(); serviceStart(); } stateModel.enterState(STATE.STARTED) its directly set to SERVICE.STATE=STARTED
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Bibin A Chundatt, Thanks for reopening this jira and yes agree with your analysis its wrong to handle in this way. So whats the approach you have in your mind ?
          I could think of having a flag in CommonNodeLabelsManager which set before calling initNodeLabelStore and reset after the call finishes. Thoughts?
          Also this time we need to better correct the test case too.

          Show
          Naganarasimha Naganarasimha G R added a comment - Bibin A Chundatt , Thanks for reopening this jira and yes agree with your analysis its wrong to handle in this way. So whats the approach you have in your mind ? I could think of having a flag in CommonNodeLabelsManager which set before calling initNodeLabelStore and reset after the call finishes. Thoughts? Also this time we need to better correct the test case too.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Naganarasimha G R
          Instead of state based approach have added a flag to identify the initStore state and setting the flag once initNodeLabelStore is completed.
          Please do help in review of the same.

          Show
          bibinchundatt Bibin A Chundatt added a comment - Naganarasimha G R Instead of state based approach have added a flag to identify the initStore state and setting the flag once initNodeLabelStore is completed. Please do help in review of the same.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 38s Maven dependency ordering for branch
          +1 mvninstall 6m 41s trunk passed
          +1 compile 1m 43s trunk passed with JDK v1.8.0_77
          +1 compile 2m 5s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 33s trunk passed
          +1 mvnsite 1m 4s trunk passed
          +1 mvneclipse 0m 28s trunk passed
          +1 findbugs 2m 11s trunk passed
          +1 javadoc 0m 47s trunk passed with JDK v1.8.0_77
          +1 javadoc 0m 58s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 10s Maven dependency ordering for patch
          +1 mvninstall 0m 55s the patch passed
          +1 compile 1m 40s the patch passed with JDK v1.8.0_77
          +1 javac 1m 40s the patch passed
          +1 compile 2m 2s the patch passed with JDK v1.7.0_95
          +1 javac 2m 2s the patch passed
          +1 checkstyle 0m 31s hadoop-yarn-project/hadoop-yarn: patch generated 0 new + 38 unchanged - 1 fixed = 38 total (was 39)
          +1 mvnsite 1m 2s the patch passed
          +1 mvneclipse 0m 25s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 33s the patch passed
          +1 javadoc 0m 46s the patch passed with JDK v1.8.0_77
          +1 javadoc 0m 54s the patch passed with JDK v1.7.0_95
          +1 unit 1m 54s hadoop-yarn-common in the patch passed with JDK v1.8.0_77.
          -1 unit 64m 13s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77.
          +1 unit 2m 5s hadoop-yarn-common in the patch passed with JDK v1.7.0_95.
          -1 unit 49m 22s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 19s Patch does not generate ASF License warnings.
          147m 21s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
            hadoop.yarn.webapp.TestRMWithCSRFFilter
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
            hadoop.yarn.webapp.TestRMWithCSRFFilter
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797489/0005-YARN-3971.addendum.patch
          JIRA Issue YARN-3971
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 2ca18e7dea0d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a62637a
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/10984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10984/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10984/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 38s Maven dependency ordering for branch +1 mvninstall 6m 41s trunk passed +1 compile 1m 43s trunk passed with JDK v1.8.0_77 +1 compile 2m 5s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 33s trunk passed +1 mvnsite 1m 4s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 2m 11s trunk passed +1 javadoc 0m 47s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 58s trunk passed with JDK v1.7.0_95 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 0m 55s the patch passed +1 compile 1m 40s the patch passed with JDK v1.8.0_77 +1 javac 1m 40s the patch passed +1 compile 2m 2s the patch passed with JDK v1.7.0_95 +1 javac 2m 2s the patch passed +1 checkstyle 0m 31s hadoop-yarn-project/hadoop-yarn: patch generated 0 new + 38 unchanged - 1 fixed = 38 total (was 39) +1 mvnsite 1m 2s the patch passed +1 mvneclipse 0m 25s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 33s the patch passed +1 javadoc 0m 46s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 54s the patch passed with JDK v1.7.0_95 +1 unit 1m 54s hadoop-yarn-common in the patch passed with JDK v1.8.0_77. -1 unit 64m 13s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77. +1 unit 2m 5s hadoop-yarn-common in the patch passed with JDK v1.7.0_95. -1 unit 49m 22s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 147m 21s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.webapp.TestRMWithCSRFFilter JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.webapp.TestRMWithCSRFFilter   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797489/0005-YARN-3971.addendum.patch JIRA Issue YARN-3971 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2ca18e7dea0d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a62637a Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/10984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/10984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10984/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/10984/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Test failure looks like not related to patch attached.Due to bind exception the same failed.

          com.sun.jersey.test.framework.spi.container.TestContainerException: java.net.BindException: Address already in use
          	at sun.nio.ch.Net.bind0(Native Method)
          	at sun.nio.ch.Net.bind(Net.java:463)
          	at sun.nio.ch.Net.bind(Net.java:455)
          	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
          	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
          	at org.glassfish.grizzly.nio.transport.TCPNIOTransport.bind(TCPNIOTransport.java:413)
          	at org.glassfish.grizzly.nio.transport.TCPNIOTransport.bind(TCPNIOTransport.java:384)
          	at org.glassfish.grizzly.nio.transport.TCPNIOTransport.bind(TCPNIOTransport.java:375)
          	at org.glassfish.grizzly.http.server.NetworkListener.start(NetworkListener.java:549)
          	at org.glassfish.grizzly.http.server.HttpServer.start(HttpServer.java:255)
          	at com.sun.jersey.api.container.grizzly2.GrizzlyServerFactory.createHttpServer(GrizzlyServerFactory.java:326)
          	at com.sun.jersey.api.container.grizzly2.GrizzlyServerFactory.createHttpServer(GrizzlyServerFactory.java:343)
          
          Show
          bibinchundatt Bibin A Chundatt added a comment - Test failure looks like not related to patch attached.Due to bind exception the same failed. com.sun.jersey.test.framework.spi.container.TestContainerException: java.net.BindException: Address already in use at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:463) at sun.nio.ch.Net.bind(Net.java:455) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.glassfish.grizzly.nio.transport.TCPNIOTransport.bind(TCPNIOTransport.java:413) at org.glassfish.grizzly.nio.transport.TCPNIOTransport.bind(TCPNIOTransport.java:384) at org.glassfish.grizzly.nio.transport.TCPNIOTransport.bind(TCPNIOTransport.java:375) at org.glassfish.grizzly.http.server.NetworkListener.start(NetworkListener.java:549) at org.glassfish.grizzly.http.server.HttpServer.start(HttpServer.java:255) at com.sun.jersey.api.container.grizzly2.GrizzlyServerFactory.createHttpServer(GrizzlyServerFactory.java:326) at com.sun.jersey.api.container.grizzly2.GrizzlyServerFactory.createHttpServer(GrizzlyServerFactory.java:343)
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Naganarasimha G R/Wangda Tan

          Could you please review the latest patch attached

          Show
          bibinchundatt Bibin A Chundatt added a comment - Naganarasimha G R / Wangda Tan Could you please review the latest patch attached
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Will look into it shortly...

          Show
          Naganarasimha Naganarasimha G R added a comment - Will look into it shortly...
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Hi Bibin A Chundatt, thanks for providing the addendum patch, was able to apply only the test case from the patch and reproduce the issue . I am fine with the approach taken. Just one small nit.

          • Instead of creating a subclass just to verify whether the method was called or not we can make use of the Mockito.spy on the actual object and then try to use Mockito.verify method
            lmgr = spy(new RMNodeLabelsManager());
            verify(lmgr,times(0)).checkRemoveFromClusterNodeLabelsOfQueue(any(Collection.class));
            
          Show
          Naganarasimha Naganarasimha G R added a comment - Hi Bibin A Chundatt , thanks for providing the addendum patch, was able to apply only the test case from the patch and reproduce the issue . I am fine with the approach taken. Just one small nit. Instead of creating a subclass just to verify whether the method was called or not we can make use of the Mockito.spy on the actual object and then try to use Mockito.verify method lmgr = spy( new RMNodeLabelsManager()); verify(lmgr,times(0)).checkRemoveFromClusterNodeLabelsOfQueue(any(Collection.class));
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Naganarasimha G R
          Thank you for review comment. Attached patch after handling the same

          Show
          bibinchundatt Bibin A Chundatt added a comment - Naganarasimha G R Thank you for review comment. Attached patch after handling the same
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 52s Maven dependency ordering for branch
          +1 mvninstall 6m 36s trunk passed
          +1 compile 1m 50s trunk passed with JDK v1.8.0_77
          +1 compile 2m 7s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 34s trunk passed
          +1 mvnsite 1m 6s trunk passed
          +1 mvneclipse 0m 27s trunk passed
          +1 findbugs 2m 13s trunk passed
          +1 javadoc 0m 47s trunk passed with JDK v1.8.0_77
          +1 javadoc 0m 59s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 10s Maven dependency ordering for patch
          +1 mvninstall 0m 57s the patch passed
          +1 compile 1m 42s the patch passed with JDK v1.8.0_77
          +1 javac 1m 42s the patch passed
          +1 compile 2m 5s the patch passed with JDK v1.7.0_95
          +1 javac 2m 5s the patch passed
          +1 checkstyle 0m 32s hadoop-yarn-project/hadoop-yarn: patch generated 0 new + 40 unchanged - 1 fixed = 40 total (was 41)
          +1 mvnsite 1m 1s the patch passed
          +1 mvneclipse 0m 25s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 39s the patch passed
          +1 javadoc 0m 43s the patch passed with JDK v1.8.0_77
          +1 javadoc 0m 56s the patch passed with JDK v1.7.0_95
          +1 unit 1m 53s hadoop-yarn-common in the patch passed with JDK v1.8.0_77.
          -1 unit 67m 59s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77.
          +1 unit 2m 11s hadoop-yarn-common in the patch passed with JDK v1.7.0_95.
          -1 unit 49m 58s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 18s Patch does not generate ASF License warnings.
          152m 24s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
            hadoop.yarn.webapp.TestRMWithCSRFFilter
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
            hadoop.yarn.webapp.TestRMWithCSRFFilter
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797968/0005-YARN-3971.001.addendum.patch
          JIRA Issue YARN-3971
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 1aa9e705ff02 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1ff27f9
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11020/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11020/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11020/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11020/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11020/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11020/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 52s Maven dependency ordering for branch +1 mvninstall 6m 36s trunk passed +1 compile 1m 50s trunk passed with JDK v1.8.0_77 +1 compile 2m 7s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 34s trunk passed +1 mvnsite 1m 6s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 2m 13s trunk passed +1 javadoc 0m 47s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 59s trunk passed with JDK v1.7.0_95 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 0m 57s the patch passed +1 compile 1m 42s the patch passed with JDK v1.8.0_77 +1 javac 1m 42s the patch passed +1 compile 2m 5s the patch passed with JDK v1.7.0_95 +1 javac 2m 5s the patch passed +1 checkstyle 0m 32s hadoop-yarn-project/hadoop-yarn: patch generated 0 new + 40 unchanged - 1 fixed = 40 total (was 41) +1 mvnsite 1m 1s the patch passed +1 mvneclipse 0m 25s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 39s the patch passed +1 javadoc 0m 43s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 56s the patch passed with JDK v1.7.0_95 +1 unit 1m 53s hadoop-yarn-common in the patch passed with JDK v1.8.0_77. -1 unit 67m 59s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77. +1 unit 2m 11s hadoop-yarn-common in the patch passed with JDK v1.7.0_95. -1 unit 49m 58s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 152m 24s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps   hadoop.yarn.webapp.TestRMWithCSRFFilter   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps   hadoop.yarn.webapp.TestRMWithCSRFFilter   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797968/0005-YARN-3971.001.addendum.patch JIRA Issue YARN-3971 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1aa9e705ff02 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1ff27f9 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/11020/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11020/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11020/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11020/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11020/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/11020/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Testcase failures are already tracked as part of YARN-4909

          Show
          bibinchundatt Bibin A Chundatt added a comment - Testcase failures are already tracked as part of YARN-4909
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks for the latest patch Bibin A Chundatt, patch LGTM.
          Tan, Wangda if you are ok with the approach taken then will go ahead and commit the addendum patch.

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks for the latest patch Bibin A Chundatt , patch LGTM. Tan, Wangda if you are ok with the approach taken then will go ahead and commit the addendum patch.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks Bibin A Chundatt/Naganarasimha G R looking at this issue.

          I think we need to fix AbstractService.state transition, only return started after serviceStart() returns.

          Do you know why we need to change service state to started before invoke serviceStart()? It doesn't make sense to me actually...

          + Vinod Kumar Vavilapalli, who made this change in YARN-530.

          Show
          leftnoteasy Wangda Tan added a comment - Thanks Bibin A Chundatt / Naganarasimha G R looking at this issue. I think we need to fix AbstractService.state transition, only return started after serviceStart() returns. Do you know why we need to change service state to started before invoke serviceStart()? It doesn't make sense to me actually... + Vinod Kumar Vavilapalli , who made this change in YARN-530 .
          Hide
          leftnoteasy Wangda Tan added a comment -

          Offline synced with Vinod Kumar Vavilapalli, it is a bug. Bibin A Chundatt, could you open a new bug to fix it? Thanks.

          Show
          leftnoteasy Wangda Tan added a comment - Offline synced with Vinod Kumar Vavilapalli , it is a bug. Bibin A Chundatt , could you open a new bug to fix it? Thanks.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Do you know why we need to change service state to started before invoke serviceStart()? It doesn't make sense to me actually...

          Thought about the same too. Since its a very old code didn't want to change the same. Thanks for the clarification.
          Will raise a new jira and close this jira.

          Show
          bibinchundatt Bibin A Chundatt added a comment - Do you know why we need to change service state to started before invoke serviceStart()? It doesn't make sense to me actually... Thought about the same too. Since its a very old code didn't want to change the same. Thanks for the clarification. Will raise a new jira and close this jira.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Hi Tan, Wangda & Bibin A Chundatt,
          Bibin has raised HADOOP-13035 has raised, we can evaluate whether its simple fix or it comes with lot of failures and risks and then decide whether to close this jira without using the addendum patch or proceed with it.

          Show
          Naganarasimha Naganarasimha G R added a comment - Hi Tan, Wangda & Bibin A Chundatt , Bibin has raised HADOOP-13035 has raised, we can evaluate whether its simple fix or it comes with lot of failures and risks and then decide whether to close this jira without using the addendum patch or proceed with it.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Hi Tan, Wangda, going through the discussions in HADOOP-13035, seems like waiting for change in the service layer would be riskier and hence not be immediate. Also this is a critical issue which can be possible in any installation. So IMHO i would vote for the approach in the latest patch and get it in for 2.8 release itself. thoughts ?

          Show
          Naganarasimha Naganarasimha G R added a comment - Hi Tan, Wangda , going through the discussions in HADOOP-13035 , seems like waiting for change in the service layer would be riskier and hence not be immediate. Also this is a critical issue which can be possible in any installation. So IMHO i would vote for the approach in the latest patch and get it in for 2.8 release itself. thoughts ?
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Had offline discussion with Tan, Wangda, & Bibin A Chundatt, as HADOOP-13035 is not expected to come in near future, going ahead with patch 0005-YARN-3971.001.addendum.patch . Will initiate the jenkins again and once the result is fine willl commit the patch !

          Show
          Naganarasimha Naganarasimha G R added a comment - Had offline discussion with Tan, Wangda , & Bibin A Chundatt , as HADOOP-13035 is not expected to come in near future, going ahead with patch 0005- YARN-3971 .001.addendum.patch . Will initiate the jenkins again and once the result is fine willl commit the patch !
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 22s Maven dependency ordering for branch
          +1 mvninstall 6m 25s trunk passed
          +1 compile 2m 3s trunk passed
          +1 checkstyle 0m 36s trunk passed
          +1 mvnsite 1m 4s trunk passed
          +1 mvneclipse 0m 25s trunk passed
          +1 findbugs 1m 47s trunk passed
          +1 javadoc 0m 49s trunk passed
          0 mvndep 0m 8s Maven dependency ordering for patch
          +1 mvninstall 0m 55s the patch passed
          +1 compile 1m 56s the patch passed
          +1 javac 1m 56s the patch passed
          -1 checkstyle 0m 36s hadoop-yarn-project/hadoop-yarn: patch generated 4 new + 56 unchanged - 3 fixed = 60 total (was 59)
          +1 mvnsite 1m 1s the patch passed
          +1 mvneclipse 0m 21s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 59s the patch passed
          +1 javadoc 0m 47s the patch passed
          +1 unit 2m 7s hadoop-yarn-common in the patch passed.
          -1 unit 34m 32s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 asflicense 0m 16s Patch does not generate ASF License warnings.
          59m 9s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797968/0005-YARN-3971.001.addendum.patch
          JIRA Issue YARN-3971
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 12ab41d99a4f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6161d9b
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11627/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11627/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11627/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11627/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11627/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 22s Maven dependency ordering for branch +1 mvninstall 6m 25s trunk passed +1 compile 2m 3s trunk passed +1 checkstyle 0m 36s trunk passed +1 mvnsite 1m 4s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 1m 47s trunk passed +1 javadoc 0m 49s trunk passed 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 0m 55s the patch passed +1 compile 1m 56s the patch passed +1 javac 1m 56s the patch passed -1 checkstyle 0m 36s hadoop-yarn-project/hadoop-yarn: patch generated 4 new + 56 unchanged - 3 fixed = 60 total (was 59) +1 mvnsite 1m 1s the patch passed +1 mvneclipse 0m 21s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 59s the patch passed +1 javadoc 0m 47s the patch passed +1 unit 2m 7s hadoop-yarn-common in the patch passed. -1 unit 34m 32s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 16s Patch does not generate ASF License warnings. 59m 9s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797968/0005-YARN-3971.001.addendum.patch JIRA Issue YARN-3971 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 12ab41d99a4f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6161d9b Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11627/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11627/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11627/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11627/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/11627/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Bibin A Chundatt, Checkstyle comments seems to be minor and can be corrected while committing the patch and test case failure is not related to the patch , hence patch LGTM and committing it shortly if no further comments !

          Show
          Naganarasimha Naganarasimha G R added a comment - Bibin A Chundatt , Checkstyle comments seems to be minor and can be corrected while committing the patch and test case failure is not related to the patch , hence patch LGTM and committing it shortly if no further comments !
          Hide
          Naganarasimha Naganarasimha G R added a comment - - edited

          Committed the patch to trunk, Branch-2 & Branch-2.8. Thanks for the contribution Bibin A Chundatt and review from Tan, Wangda

          Show
          Naganarasimha Naganarasimha G R added a comment - - edited Committed the patch to trunk, Branch-2 & Branch-2.8. Thanks for the contribution Bibin A Chundatt and review from Tan, Wangda
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #9846 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9846/)
          YARN-3971. Skip (naganarasimha_gr: rev b4078bd17b41fbfff5a5c0bca5bf903a327826a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9846 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9846/ ) YARN-3971 . Skip (naganarasimha_gr: rev b4078bd17b41fbfff5a5c0bca5bf903a327826a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestRMNodeLabelsManager.java

            People

            • Assignee:
              bibinchundatt Bibin A Chundatt
              Reporter:
              bibinchundatt Bibin A Chundatt
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development