Details

    • Hadoop Flags:
      Reviewed

      Description

      The Federation Membership State encapsulates the information about the active RM of each sub-cluster that is participating in Federation. The information includes addresses for ClientRM, ApplicationMaster and Admin services along with the sub_cluster capability which is currently defined by ClusterMetricsInfo. Please refer to the design doc in parent JIRA for further details.

      1. YARN-3662-YARN-2915-v7.patch
        171 kB
        Vinod Kumar Vavilapalli
      2. YARN-3662-YARN-2915-v6.patch
        171 kB
        Subru Krishnan
      3. YARN-3662-YARN-2915-v5.patch
        177 kB
        Subru Krishnan
      4. YARN-3662-YARN-2915-v4.patch
        128 kB
        Subru Krishnan
      5. YARN-3662-YARN-2915-v3.01.patch
        128 kB
        Subru Krishnan
      6. YARN-3662-YARN-2915-v3.patch
        128 kB
        Subru Krishnan
      7. YARN-3662-YARN-2915-v2.patch
        120 kB
        Subru Krishnan
      8. YARN-3662-YARN-2915-v1.1.patch
        189 kB
        Subru Krishnan
      9. YARN-3662-YARN-2915-v1.patch
        190 kB
        Subru Krishnan

        Issue Links

          Activity

          Hide
          subru Subru Krishnan added a comment - - edited

          Attaching v1 of the Federation StateStore APIs. The APIs include:

          • FederationMembershipState which encapsulates the YARN (sub)clusters that are participating in Federation.
          • FederationApplicationState which is mapping between the Application and it's home (sub)cluster.

          Please refer to the design doc in the parent JIRA (YARN-2915) for further details.

          Show
          subru Subru Krishnan added a comment - - edited Attaching v1 of the Federation StateStore APIs. The APIs include: FederationMembershipState which encapsulates the YARN (sub)clusters that are participating in Federation. FederationApplicationState which is mapping between the Application and it's home (sub)cluster. Please refer to the design doc in the parent JIRA ( YARN-2915 ) for further details.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 4s YARN-3662 does not apply to YARN-2915. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12814029/YARN-3662-YARN-2915-v1.patch
          JIRA Issue YARN-3662
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12152/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 4s YARN-3662 does not apply to YARN-2915 . Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12814029/YARN-3662-YARN-2915-v1.patch JIRA Issue YARN-3662 Console output https://builds.apache.org/job/PreCommit-YARN-Build/12152/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          subru Subru Krishnan added a comment -

          Attaching v1.1 after rebasing with YARN-5300 commit.

          Show
          subru Subru Krishnan added a comment - Attaching v1.1 after rebasing with YARN-5300 commit.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 20m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 2m 54s Maven dependency ordering for branch
          +1 mvninstall 8m 45s YARN-2915 passed
          +1 compile 1m 59s YARN-2915 passed
          +1 checkstyle 0m 36s YARN-2915 passed
          +1 mvnsite 0m 52s YARN-2915 passed
          +1 mvneclipse 0m 30s YARN-2915 passed
          +1 findbugs 1m 33s YARN-2915 passed
          +1 javadoc 0m 42s YARN-2915 passed
          0 mvndep 0m 8s Maven dependency ordering for patch
          +1 mvninstall 0m 42s the patch passed
          +1 compile 1m 56s the patch passed
          +1 cc 1m 56s the patch passed
          +1 javac 1m 56s the patch passed
          -1 checkstyle 0m 33s hadoop-yarn-project/hadoop-yarn: The patch generated 18 new + 26 unchanged - 2 fixed = 44 total (was 28)
          +1 mvnsite 0m 47s the patch passed
          +1 mvneclipse 0m 21s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 1m 42s the patch passed
          +1 javadoc 0m 38s the patch passed
          +1 unit 2m 8s hadoop-yarn-common in the patch passed.
          +1 unit 0m 24s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          48m 22s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:e2f6409
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12814044/YARN-3662-YARN-2915-v1.1.patch
          JIRA Issue YARN-3662
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc
          uname Linux b0dde95a2849 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 41b006e
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12153/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12153/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12153/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 20m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 2m 54s Maven dependency ordering for branch +1 mvninstall 8m 45s YARN-2915 passed +1 compile 1m 59s YARN-2915 passed +1 checkstyle 0m 36s YARN-2915 passed +1 mvnsite 0m 52s YARN-2915 passed +1 mvneclipse 0m 30s YARN-2915 passed +1 findbugs 1m 33s YARN-2915 passed +1 javadoc 0m 42s YARN-2915 passed 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 0m 42s the patch passed +1 compile 1m 56s the patch passed +1 cc 1m 56s the patch passed +1 javac 1m 56s the patch passed -1 checkstyle 0m 33s hadoop-yarn-project/hadoop-yarn: The patch generated 18 new + 26 unchanged - 2 fixed = 44 total (was 28) +1 mvnsite 0m 47s the patch passed +1 mvneclipse 0m 21s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 1m 42s the patch passed +1 javadoc 0m 38s the patch passed +1 unit 2m 8s hadoop-yarn-common in the patch passed. +1 unit 0m 24s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 48m 22s Subsystem Report/Notes Docker Image:yetus/hadoop:e2f6409 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12814044/YARN-3662-YARN-2915-v1.1.patch JIRA Issue YARN-3662 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc uname Linux b0dde95a2849 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 41b006e Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12153/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12153/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/12153/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          subru Subru Krishnan added a comment -

          Splitting the JIRA into 2 to allow for more manageable review/feedback cycle:

          • Current one which is for adding FederationMembershipState.
          • YARN-5307 for adding FederationApplicationState.
          Show
          subru Subru Krishnan added a comment - Splitting the JIRA into 2 to allow for more manageable review/feedback cycle: Current one which is for adding FederationMembershipState . YARN-5307 for adding FederationApplicationState .
          Hide
          subru Subru Krishnan added a comment -

          Uploading v2 patch that contains only the FederationMembershipState APIs as mentioned above.

          Show
          subru Subru Krishnan added a comment - Uploading v2 patch that contains only the FederationMembershipState APIs as mentioned above.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 1m 15s Maven dependency ordering for branch
          +1 mvninstall 7m 23s YARN-2915 passed
          +1 compile 2m 2s YARN-2915 passed
          +1 checkstyle 0m 36s YARN-2915 passed
          +1 mvnsite 0m 54s YARN-2915 passed
          +1 mvneclipse 0m 29s YARN-2915 passed
          +1 findbugs 1m 33s YARN-2915 passed
          +1 javadoc 0m 42s YARN-2915 passed
          0 mvndep 0m 7s Maven dependency ordering for patch
          +1 mvninstall 0m 41s the patch passed
          +1 compile 1m 57s the patch passed
          +1 cc 1m 57s the patch passed
          +1 javac 1m 57s the patch passed
          -1 checkstyle 0m 32s hadoop-yarn-project/hadoop-yarn: The patch generated 10 new + 26 unchanged - 2 fixed = 36 total (was 28)
          +1 mvnsite 0m 47s the patch passed
          +1 mvneclipse 0m 22s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 1m 43s the patch passed
          +1 javadoc 0m 38s the patch passed
          +1 unit 2m 9s hadoop-yarn-common in the patch passed.
          +1 unit 0m 24s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          25m 45s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:e2f6409
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12815848/YARN-3662-YARN-2915-v2.patch
          JIRA Issue YARN-3662
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc
          uname Linux 2dd04b7de53b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 41b006e
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12173/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12173/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12173/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 1m 15s Maven dependency ordering for branch +1 mvninstall 7m 23s YARN-2915 passed +1 compile 2m 2s YARN-2915 passed +1 checkstyle 0m 36s YARN-2915 passed +1 mvnsite 0m 54s YARN-2915 passed +1 mvneclipse 0m 29s YARN-2915 passed +1 findbugs 1m 33s YARN-2915 passed +1 javadoc 0m 42s YARN-2915 passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 0m 41s the patch passed +1 compile 1m 57s the patch passed +1 cc 1m 57s the patch passed +1 javac 1m 57s the patch passed -1 checkstyle 0m 32s hadoop-yarn-project/hadoop-yarn: The patch generated 10 new + 26 unchanged - 2 fixed = 36 total (was 28) +1 mvnsite 0m 47s the patch passed +1 mvneclipse 0m 22s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 43s the patch passed +1 javadoc 0m 38s the patch passed +1 unit 2m 9s hadoop-yarn-common in the patch passed. +1 unit 0m 24s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 25m 45s Subsystem Report/Notes Docker Image:yetus/hadoop:e2f6409 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12815848/YARN-3662-YARN-2915-v2.patch JIRA Issue YARN-3662 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc uname Linux 2dd04b7de53b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 41b006e Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12173/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12173/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/12173/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          subru Subru Krishnan added a comment -

          I have fixed all the checkstyle issues (ran test-patch locally apriori) and only reported ones are related to:

          • methods have public modifiers (this patch has API only, implementation will follow in YARN-3663...).
          • FederationSubClusterInfo::newInstance having more than 7 params
          • one API signature that exceed 80 characters.

          I feel it is safe to ignore these.

          Show
          subru Subru Krishnan added a comment - I have fixed all the checkstyle issues (ran test-patch locally apriori) and only reported ones are related to: methods have public modifiers (this patch has API only, implementation will follow in YARN-3663 ...). FederationSubClusterInfo::newInstance having more than 7 params one API signature that exceed 80 characters. I feel it is safe to ignore these.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Hi Subru Krishnan,

          I took a very quick look at this patch and also YARN-3664/YARN-5367, I put all questions and comments here:

          Questions:

          • Could not quite sure about what is FederationPolicy and how to use the class. Is it a state or a configuration? And why compressing parameters into a byte array instead more meaningful fields?
          • It could be better to add RPC service interface definitions of FederationPolicy storage API for easier review, now I cannot understand how these protocol definitions will be used.

          (Highlevel) Comments:

          • FederationMembershipState looks like a "state manager" since it supports operations to modify existing members. At the first glance, it's a sub-cluster-resource-tracker which is similar to existing RM resource tracker.
          • Similiarly, FederationApplicationState looks like a "federation-application-manager" instead of a "state".
          • FederationMembershipState has same parameter FederationSubClusterInfo for register/heartbeat – is it possible that we require different parameter for registration and heartbeat? (Just like NM registration request and NM update request).
          • FederationSubClusterInfo: fields like amRMAddress is actually a service endpoint, names of these fields are little confusing to me.

          Styles:

          • redundunt "public" in all interface definitions (considering switching to Intellij instead of Eclipse? :-p)

          Thanks,

          Show
          leftnoteasy Wangda Tan added a comment - Hi Subru Krishnan , I took a very quick look at this patch and also YARN-3664 / YARN-5367 , I put all questions and comments here: Questions: Could not quite sure about what is FederationPolicy and how to use the class. Is it a state or a configuration? And why compressing parameters into a byte array instead more meaningful fields? It could be better to add RPC service interface definitions of FederationPolicy storage API for easier review, now I cannot understand how these protocol definitions will be used. (Highlevel) Comments: FederationMembershipState looks like a "state manager" since it supports operations to modify existing members. At the first glance, it's a sub-cluster-resource-tracker which is similar to existing RM resource tracker. Similiarly, FederationApplicationState looks like a "federation-application-manager" instead of a "state". FederationMembershipState has same parameter FederationSubClusterInfo for register/heartbeat – is it possible that we require different parameter for registration and heartbeat? (Just like NM registration request and NM update request). FederationSubClusterInfo: fields like amRMAddress is actually a service endpoint, names of these fields are little confusing to me. Styles: redundunt "public" in all interface definitions (considering switching to Intellij instead of Eclipse? :-p) Thanks,
          Hide
          subru Subru Krishnan added a comment - - edited

          Thanks Wangda Tan for reviewing the APIs. I am attaching an updated patch (v3) that addresses your concerns on membership API, specifically:

          • Renaming FederationMembershipState --> FederationMembershipStateStore as this is the equivalent of RMStateStore for Federation.
          • Added an explicit FederationSubClusterHeartbeatRequest
          • Renamed all the address fields in FederationSubClusterInfo to serviceAddress

          I'll also update YARN-5307 by renaming FederationApplicationState --> FederationApplicationStateStore.

          Regarding your question about the Policy API, Carlo has uploaded the initial version of the Policies in YARN-5323 with a summary of how they will be used in Federation. We are updating the API Javadoc in YARN-3664 to increase clarity.

          Show
          subru Subru Krishnan added a comment - - edited Thanks Wangda Tan for reviewing the APIs. I am attaching an updated patch (v3) that addresses your concerns on membership API, specifically: Renaming FederationMembershipState --> FederationMembershipStateStore as this is the equivalent of RMStateStore for Federation. Added an explicit FederationSubClusterHeartbeatRequest Renamed all the address fields in FederationSubClusterInfo to serviceAddress I'll also update YARN-5307 by renaming FederationApplicationState --> FederationApplicationStateStore . Regarding your question about the Policy API, Carlo has uploaded the initial version of the Policies in YARN-5323 with a summary of how they will be used in Federation. We are updating the API Javadoc in YARN-3664 to increase clarity.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 docker 2m 19s Docker failed to build yetus/hadoop:e2f6409.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818722/YARN-3662-YARN-2915-v3.patch
          JIRA Issue YARN-3662
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12364/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 docker 2m 19s Docker failed to build yetus/hadoop:e2f6409. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818722/YARN-3662-YARN-2915-v3.patch JIRA Issue YARN-3662 Console output https://builds.apache.org/job/PreCommit-YARN-Build/12364/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 docker 2m 17s Docker failed to build yetus/hadoop:e2f6409.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818722/YARN-3662-YARN-2915-v3.patch
          JIRA Issue YARN-3662
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12365/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 docker 2m 17s Docker failed to build yetus/hadoop:e2f6409. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818722/YARN-3662-YARN-2915-v3.patch JIRA Issue YARN-3662 Console output https://builds.apache.org/job/PreCommit-YARN-Build/12365/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 docker 4m 37s Docker failed to build yetus/hadoop:e2f6409.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818722/YARN-3662-YARN-2915-v3.patch
          JIRA Issue YARN-3662
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12366/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 docker 4m 37s Docker failed to build yetus/hadoop:e2f6409. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818722/YARN-3662-YARN-2915-v3.patch JIRA Issue YARN-3662 Console output https://builds.apache.org/job/PreCommit-YARN-Build/12366/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 docker 0m 2s Docker failed to build yetus/hadoop:e2f6409.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818722/YARN-3662-YARN-2915-v3.patch
          JIRA Issue YARN-3662
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12373/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 docker 0m 2s Docker failed to build yetus/hadoop:e2f6409. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818722/YARN-3662-YARN-2915-v3.patch JIRA Issue YARN-3662 Console output https://builds.apache.org/job/PreCommit-YARN-Build/12373/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          subru Subru Krishnan added a comment -

          Reattaching patch after rebasing YARN-2915 branch to pull in HADOOP-13342

          Show
          subru Subru Krishnan added a comment - Reattaching patch after rebasing YARN-2915 branch to pull in HADOOP-13342
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 5m 15s Maven dependency ordering for branch
          +1 mvninstall 8m 32s YARN-2915 passed
          +1 compile 2m 28s YARN-2915 passed
          +1 checkstyle 0m 42s YARN-2915 passed
          +1 mvnsite 0m 55s YARN-2915 passed
          +1 mvneclipse 0m 30s YARN-2915 passed
          +1 findbugs 1m 42s YARN-2915 passed
          +1 javadoc 0m 45s YARN-2915 passed
          0 mvndep 0m 9s Maven dependency ordering for patch
          +1 mvninstall 0m 47s the patch passed
          +1 compile 2m 26s the patch passed
          +1 cc 2m 26s the patch passed
          +1 javac 2m 26s the patch passed
          -1 checkstyle 0m 37s hadoop-yarn-project/hadoop-yarn: The patch generated 15 new + 26 unchanged - 2 fixed = 41 total (was 28)
          +1 mvnsite 0m 50s the patch passed
          +1 mvneclipse 0m 22s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 1m 52s the patch passed
          +1 javadoc 0m 43s the patch passed
          +1 unit 2m 18s hadoop-yarn-common in the patch passed.
          +1 unit 0m 27s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          32m 45s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818947/YARN-3662-YARN-2915-v3.01.patch
          JIRA Issue YARN-3662
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc
          uname Linux 2d926d8146e8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 85eda58
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12427/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12427/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12427/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 5m 15s Maven dependency ordering for branch +1 mvninstall 8m 32s YARN-2915 passed +1 compile 2m 28s YARN-2915 passed +1 checkstyle 0m 42s YARN-2915 passed +1 mvnsite 0m 55s YARN-2915 passed +1 mvneclipse 0m 30s YARN-2915 passed +1 findbugs 1m 42s YARN-2915 passed +1 javadoc 0m 45s YARN-2915 passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 0m 47s the patch passed +1 compile 2m 26s the patch passed +1 cc 2m 26s the patch passed +1 javac 2m 26s the patch passed -1 checkstyle 0m 37s hadoop-yarn-project/hadoop-yarn: The patch generated 15 new + 26 unchanged - 2 fixed = 41 total (was 28) +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 22s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 52s the patch passed +1 javadoc 0m 43s the patch passed +1 unit 2m 18s hadoop-yarn-common in the patch passed. +1 unit 0m 27s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 32m 45s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818947/YARN-3662-YARN-2915-v3.01.patch JIRA Issue YARN-3662 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc uname Linux 2d926d8146e8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 85eda58 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12427/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12427/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/12427/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          subru Subru Krishnan added a comment -

          Minor update (v4) with fixes for checkstyle issues that can be addressed.

          Show
          subru Subru Krishnan added a comment - Minor update (v4) with fixes for checkstyle issues that can be addressed.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 40s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 3m 44s Maven dependency ordering for branch
          +1 mvninstall 8m 2s YARN-2915 passed
          +1 compile 2m 17s YARN-2915 passed
          +1 checkstyle 0m 40s YARN-2915 passed
          +1 mvnsite 0m 58s YARN-2915 passed
          +1 mvneclipse 0m 30s YARN-2915 passed
          +1 findbugs 1m 38s YARN-2915 passed
          +1 javadoc 0m 44s YARN-2915 passed
          0 mvndep 0m 10s Maven dependency ordering for patch
          +1 mvninstall 0m 46s the patch passed
          +1 compile 2m 13s the patch passed
          +1 cc 2m 13s the patch passed
          +1 javac 2m 13s the patch passed
          -1 checkstyle 0m 35s hadoop-yarn-project/hadoop-yarn: The patch generated 10 new + 26 unchanged - 2 fixed = 36 total (was 28)
          +1 mvnsite 0m 49s the patch passed
          +1 mvneclipse 0m 21s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 1m 41s the patch passed
          +1 javadoc 0m 40s the patch passed
          +1 unit 2m 17s hadoop-yarn-common in the patch passed.
          +1 unit 0m 28s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          30m 23s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819225/YARN-3662-YARN-2915-v4.patch
          JIRA Issue YARN-3662
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc
          uname Linux dd0caf86a4d3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 85eda58
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12429/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12429/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12429/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 40s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 3m 44s Maven dependency ordering for branch +1 mvninstall 8m 2s YARN-2915 passed +1 compile 2m 17s YARN-2915 passed +1 checkstyle 0m 40s YARN-2915 passed +1 mvnsite 0m 58s YARN-2915 passed +1 mvneclipse 0m 30s YARN-2915 passed +1 findbugs 1m 38s YARN-2915 passed +1 javadoc 0m 44s YARN-2915 passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 0m 46s the patch passed +1 compile 2m 13s the patch passed +1 cc 2m 13s the patch passed +1 javac 2m 13s the patch passed -1 checkstyle 0m 35s hadoop-yarn-project/hadoop-yarn: The patch generated 10 new + 26 unchanged - 2 fixed = 36 total (was 28) +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 21s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 1m 41s the patch passed +1 javadoc 0m 40s the patch passed +1 unit 2m 17s hadoop-yarn-common in the patch passed. +1 unit 0m 28s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 30m 23s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819225/YARN-3662-YARN-2915-v4.patch JIRA Issue YARN-3662 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc uname Linux dd0caf86a4d3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 85eda58 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12429/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12429/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/12429/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 29s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 3m 55s Maven dependency ordering for branch
          +1 mvninstall 9m 37s YARN-2915 passed
          +1 compile 2m 49s YARN-2915 passed
          +1 checkstyle 0m 42s YARN-2915 passed
          +1 mvnsite 0m 54s YARN-2915 passed
          +1 mvneclipse 0m 30s YARN-2915 passed
          +1 findbugs 1m 36s YARN-2915 passed
          +1 javadoc 0m 41s YARN-2915 passed
          0 mvndep 0m 9s Maven dependency ordering for patch
          +1 mvninstall 0m 42s the patch passed
          +1 compile 2m 19s the patch passed
          +1 cc 2m 19s the patch passed
          +1 javac 2m 19s the patch passed
          -1 checkstyle 0m 38s hadoop-yarn-project/hadoop-yarn: The patch generated 10 new + 26 unchanged - 2 fixed = 36 total (was 28)
          +1 mvnsite 0m 52s the patch passed
          +1 mvneclipse 0m 22s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 1m 47s the patch passed
          +1 javadoc 0m 39s the patch passed
          +1 unit 2m 16s hadoop-yarn-common in the patch passed.
          +1 unit 0m 25s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          32m 31s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819225/YARN-3662-YARN-2915-v4.patch
          JIRA Issue YARN-3662
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc
          uname Linux ddffb12c4be1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 85eda58
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12433/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12433/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12433/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 29s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 3m 55s Maven dependency ordering for branch +1 mvninstall 9m 37s YARN-2915 passed +1 compile 2m 49s YARN-2915 passed +1 checkstyle 0m 42s YARN-2915 passed +1 mvnsite 0m 54s YARN-2915 passed +1 mvneclipse 0m 30s YARN-2915 passed +1 findbugs 1m 36s YARN-2915 passed +1 javadoc 0m 41s YARN-2915 passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 0m 42s the patch passed +1 compile 2m 19s the patch passed +1 cc 2m 19s the patch passed +1 javac 2m 19s the patch passed -1 checkstyle 0m 38s hadoop-yarn-project/hadoop-yarn: The patch generated 10 new + 26 unchanged - 2 fixed = 36 total (was 28) +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 22s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 39s the patch passed +1 unit 2m 16s hadoop-yarn-common in the patch passed. +1 unit 0m 25s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 32m 31s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819225/YARN-3662-YARN-2915-v4.patch JIRA Issue YARN-3662 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc uname Linux ddffb12c4be1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 85eda58 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12433/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12433/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/12433/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          vvasudev Varun Vasudev added a comment -

          Thanks for the patches Subru Krishnan! My thoughts on the patch -
          1)

          +  public FederationSubClusterId registerSubCluster(
          +      FederationSubClusterInfo subClusterInfo) throws YarnException;
          
          +  public FederationSubClusterHeartbeatResponse subClusterHeartbeat(
          +      FederationSubClusterInfo subClusterInfo) throws YarnException;
          
          +  public FederationSubClusterInfo getSubClusterInfo(
          +      FederationSubClusterId subClusterId) throws YarnException;
          
          +  public Map<FederationSubClusterId, FederationSubClusterInfo> getAllSubClustersInfo()
          +      throws YarnException;
          

          These functions should follow the Request/Response structure. i.e take a request object and return a response object. e.g registerSubCluster should take a RegisterSubClusterRequest object and return a RegisterSubClusterResponse. Similarly with all the other cases.

          2)

          +  public FederationSubClusterInfo getSubClusterInfo(
          +      FederationSubClusterId subClusterId) throws YarnException;
          

          Rename to getSubCluster? No need for info

          3)

          +  public Map<FederationSubClusterId, FederationSubClusterInfo> getAllSubClustersInfo()
          +      throws YarnException;
          

          Rename to getSubClusters - no need for all or info. It would also be useful to take a GetSubClustersRequest object so that we can add support for filters in the future.

          4)
          Rename SC_DEREGISTERED to SC_UNREGISTERED

          5)
          In FederationSubClusterHeartbeatRequest#newInstance, rename

           FederationSubClusterHeartbeatRequest subClusterInfo 

          to request

          6)

          +  @Private
          +  @Unstable
          +  public abstract void setCapability(String capability);
          

          I'm not sure String is the right way to express capability but I can't think of a good alternative for now. Maybe we should use the Resource class to express capability?

          7)

          +  @Override
          +  public int hashCode() {
          +    return getSubClusterId().hashCode();
          +  }
          

          In HeartbeatRequest and FederationSubClusterInfo , the hashcode function should take into account everything equals takes into account.

          8)
          This is more a code organization comment. Maybe we should move api pieces into hadoop-yarn-api? Right now the API and the implementation are in hadoop-server-common but it looks like the API pieces will be widely used. Please correct me if I'm wrong.

          Show
          vvasudev Varun Vasudev added a comment - Thanks for the patches Subru Krishnan ! My thoughts on the patch - 1) + public FederationSubClusterId registerSubCluster( + FederationSubClusterInfo subClusterInfo) throws YarnException; + public FederationSubClusterHeartbeatResponse subClusterHeartbeat( + FederationSubClusterInfo subClusterInfo) throws YarnException; + public FederationSubClusterInfo getSubClusterInfo( + FederationSubClusterId subClusterId) throws YarnException; + public Map<FederationSubClusterId, FederationSubClusterInfo> getAllSubClustersInfo() + throws YarnException; These functions should follow the Request/Response structure. i.e take a request object and return a response object. e.g registerSubCluster should take a RegisterSubClusterRequest object and return a RegisterSubClusterResponse. Similarly with all the other cases. 2) + public FederationSubClusterInfo getSubClusterInfo( + FederationSubClusterId subClusterId) throws YarnException; Rename to getSubCluster? No need for info 3) + public Map<FederationSubClusterId, FederationSubClusterInfo> getAllSubClustersInfo() + throws YarnException; Rename to getSubClusters - no need for all or info. It would also be useful to take a GetSubClustersRequest object so that we can add support for filters in the future. 4) Rename SC_DEREGISTERED to SC_UNREGISTERED 5) In FederationSubClusterHeartbeatRequest#newInstance, rename FederationSubClusterHeartbeatRequest subClusterInfo to request 6) + @Private + @Unstable + public abstract void setCapability( String capability); I'm not sure String is the right way to express capability but I can't think of a good alternative for now. Maybe we should use the Resource class to express capability? 7) + @Override + public int hashCode() { + return getSubClusterId().hashCode(); + } In HeartbeatRequest and FederationSubClusterInfo , the hashcode function should take into account everything equals takes into account. 8) This is more a code organization comment. Maybe we should move api pieces into hadoop-yarn-api? Right now the API and the implementation are in hadoop-server-common but it looks like the API pieces will be widely used. Please correct me if I'm wrong.
          Hide
          vvasudev Varun Vasudev added a comment -

          Couple of more renames -
          9)
          Rename FederationSubClusterDeregisterRequest to DeregisterFederationSubClusterRequest

          10)
          Rename FederationSubClusterDeregisterResponse to DeregisterFederationSubClusterResponse

          Show
          vvasudev Varun Vasudev added a comment - Couple of more renames - 9) Rename FederationSubClusterDeregisterRequest to DeregisterFederationSubClusterRequest 10) Rename FederationSubClusterDeregisterResponse to DeregisterFederationSubClusterResponse
          Hide
          subru Subru Krishnan added a comment -

          Thanks Varun Vasudev for reviewing the patches.

          I think there is a misunderstanding on the scope of these APIs - Federation does not add any public APIs (except for an Admin API which will be done in a separate JIRA) as the design philosophy we have followed in Federation is to be transparent to the applications. So the public APIs will be continue to be the standard YARN APIs (ApplicationClient/Master protocols).

          The Federation Store APIs are for analogous to RMStateStore. Subsequently we decided:

          • To have them in yarn-server and not in yarn-api. I will add a limited audience of YARN to make it clear.
          • Minimize the wrapper request/response classes as they cause more overhead. I can add them if you still feel it's better to have them?

          Regarding your rename suggestions, I'll update the method names accordingly. I do have a question, I prefer to have Federation as the prefix for request/response objects rather than the op (Get.../Set...) as it makes it easier to filter and also align with the package hierarchy. Thoughts?

          The title of the JIRAs might have been misleading so I updated them to call out that these are internal APIs.

          Show
          subru Subru Krishnan added a comment - Thanks Varun Vasudev for reviewing the patches. I think there is a misunderstanding on the scope of these APIs - Federation does not add any public APIs (except for an Admin API which will be done in a separate JIRA) as the design philosophy we have followed in Federation is to be transparent to the applications. So the public APIs will be continue to be the standard YARN APIs (ApplicationClient/Master protocols). The Federation Store APIs are for analogous to RMStateStore . Subsequently we decided: To have them in yarn-server and not in yarn-api. I will add a limited audience of YARN to make it clear. Minimize the wrapper request/response classes as they cause more overhead. I can add them if you still feel it's better to have them? Regarding your rename suggestions, I'll update the method names accordingly. I do have a question, I prefer to have Federation as the prefix for request/response objects rather than the op (Get.../Set...) as it makes it easier to filter and also align with the package hierarchy. Thoughts? The title of the JIRAs might have been misleading so I updated them to call out that these are internal APIs.
          Hide
          vvasudev Varun Vasudev added a comment - - edited

          That makes sense. Then let's drop the api piece from the package names? The fact that FedereationStore is an interface should make it clear it's an api.

          With regards to the Get/Set - I suspect it's better to follow the convention we follow in the rest of YARN. I think your point about filtering and the package hierarchy are valid, but on the other hand it makes it confusing for folks who've been working with the rest of YARN.

          Minimize the wrapper request/response classes as they cause more overhead. I can add them if you still feel it's better to have them?

          I think we should add them, irrespective of the overhead. It's hugely helpful to have them, especially for future re-factoring work, if it comes up. Even though these are internal APIs, it will avoid a ton of rework every time someone wants to add a field or some return information. We suffered for not having the equivalent of these wrappers when we had to re-factor the ContainerExecutor classes.

          Show
          vvasudev Varun Vasudev added a comment - - edited That makes sense. Then let's drop the api piece from the package names? The fact that FedereationStore is an interface should make it clear it's an api. With regards to the Get/Set - I suspect it's better to follow the convention we follow in the rest of YARN. I think your point about filtering and the package hierarchy are valid, but on the other hand it makes it confusing for folks who've been working with the rest of YARN. Minimize the wrapper request/response classes as they cause more overhead. I can add them if you still feel it's better to have them? I think we should add them, irrespective of the overhead. It's hugely helpful to have them, especially for future re-factoring work, if it comes up. Even though these are internal APIs, it will avoid a ton of rework every time someone wants to add a field or some return information. We suffered for not having the equivalent of these wrappers when we had to re-factor the ContainerExecutor classes.
          Hide
          subru Subru Krishnan added a comment -

          Thanks Varun Vasudev for the clarifications. I have addressed your feedback in v5 of the patch.

          Quite a few changes () so I'll list only the important ones:

          • Moved the FederationStore interfaces from API to store sub-package.
          • Added request/response objects for all methods
          • Renamed request/response objects to Get/Set... to align with YARN convention.
          • Other renames which you suggested.

          I have left the capability as string as it is more than a resource - we need the nodes in the cluster and utilization, i.e. why we currently use the serialized string representation of ClusterMetricsInfo. We can update it later if we find a better option.

          I'll also update YARN-5307/YARN-3664 similarly and post the patches tomorrow. This JIRA is more important as blocks both.

          Show
          subru Subru Krishnan added a comment - Thanks Varun Vasudev for the clarifications. I have addressed your feedback in v5 of the patch. Quite a few changes ( ) so I'll list only the important ones: Moved the FederationStore interfaces from API to store sub-package. Added request/response objects for all methods Renamed request/response objects to Get/Set... to align with YARN convention. Other renames which you suggested. I have left the capability as string as it is more than a resource - we need the nodes in the cluster and utilization, i.e. why we currently use the serialized string representation of ClusterMetricsInfo . We can update it later if we find a better option. I'll also update YARN-5307 / YARN-3664 similarly and post the patches tomorrow. This JIRA is more important as blocks both.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 35s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 16m 50s Maven dependency ordering for branch
          +1 mvninstall 11m 40s YARN-2915 passed
          +1 compile 2m 33s YARN-2915 passed
          +1 checkstyle 0m 43s YARN-2915 passed
          +1 mvnsite 1m 33s YARN-2915 passed
          +1 mvneclipse 0m 31s YARN-2915 passed
          +1 findbugs 1m 44s YARN-2915 passed
          +1 javadoc 0m 45s YARN-2915 passed
          0 mvndep 0m 10s Maven dependency ordering for patch
          +1 mvninstall 0m 44s the patch passed
          +1 compile 2m 31s the patch passed
          +1 cc 2m 31s the patch passed
          +1 javac 2m 31s the patch passed
          -1 checkstyle 0m 39s hadoop-yarn-project/hadoop-yarn: The patch generated 9 new + 26 unchanged - 2 fixed = 35 total (was 28)
          +1 mvnsite 1m 14s the patch passed
          +1 mvneclipse 0m 51s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 3s The patch has no ill-formed XML file.
          +1 findbugs 4m 2s the patch passed
          +1 javadoc 1m 31s the patch passed
          +1 unit 4m 43s hadoop-yarn-common in the patch passed.
          +1 unit 1m 8s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 39s The patch does not generate ASF License warnings.
          56m 47s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820351/YARN-3662-YARN-2915-v5.patch
          JIRA Issue YARN-3662
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc
          uname Linux f49ebfa3af1c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 85eda58
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12519/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12519/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12519/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 35s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 16m 50s Maven dependency ordering for branch +1 mvninstall 11m 40s YARN-2915 passed +1 compile 2m 33s YARN-2915 passed +1 checkstyle 0m 43s YARN-2915 passed +1 mvnsite 1m 33s YARN-2915 passed +1 mvneclipse 0m 31s YARN-2915 passed +1 findbugs 1m 44s YARN-2915 passed +1 javadoc 0m 45s YARN-2915 passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 0m 44s the patch passed +1 compile 2m 31s the patch passed +1 cc 2m 31s the patch passed +1 javac 2m 31s the patch passed -1 checkstyle 0m 39s hadoop-yarn-project/hadoop-yarn: The patch generated 9 new + 26 unchanged - 2 fixed = 35 total (was 28) +1 mvnsite 1m 14s the patch passed +1 mvneclipse 0m 51s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. +1 findbugs 4m 2s the patch passed +1 javadoc 1m 31s the patch passed +1 unit 4m 43s hadoop-yarn-common in the patch passed. +1 unit 1m 8s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 39s The patch does not generate ASF License warnings. 56m 47s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820351/YARN-3662-YARN-2915-v5.patch JIRA Issue YARN-3662 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc uname Linux f49ebfa3af1c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 85eda58 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12519/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12519/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/12519/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          vvasudev Varun Vasudev added a comment -

          Thanks for the new patch Subru Krishnan. +1. I'll commit this tomorrow if no one objects. Just to make sure, I don't mess it up the patch needs to go into branch YARN-2915 correct?

          Show
          vvasudev Varun Vasudev added a comment - Thanks for the new patch Subru Krishnan . +1. I'll commit this tomorrow if no one objects. Just to make sure, I don't mess it up the patch needs to go into branch YARN-2915 correct?
          Hide
          subru Subru Krishnan added a comment -

          Thanks Varun Vasudev for the sign-off. You are right, this is for branch YARN-2915.

          I'll commit it, just waiting to hear from Wangda Tan as he had some feedback which I have also addressed.

          Show
          subru Subru Krishnan added a comment - Thanks Varun Vasudev for the sign-off. You are right, this is for branch YARN-2915 . I'll commit it, just waiting to hear from Wangda Tan as he had some feedback which I have also addressed.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Latest patch LGTM as well, thanks Subru Krishnan.

          Show
          leftnoteasy Wangda Tan added a comment - Latest patch LGTM as well, thanks Subru Krishnan .
          Hide
          kasha Karthik Kambatla added a comment -

          Quickly skimmed through. It looks good.

          Show
          kasha Karthik Kambatla added a comment - Quickly skimmed through. It looks good.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Apologies for chiming in very late. I have more suggestions, please cherry-pick things that you agree with are easy and feel free to defer the harder ones to later JIRAs

          • Drop Federation everywhere in the records? Instead, we can simply say SubClusterInfoProto, SubClusterIdProto etc.
          • None of these classes should be marked @Public.
          • GetFederationSubClusterRequest -> GetSubClusterInfoRequest, similarly response? Similarly the GetSubClustersRequest / response objects? Also, should they be returning the info objects?
          • HeartbeatFederationSubClusterRequest -> SubClusterHeartbeatRequest. Similarly the response? Trying to follow NodeHeartbeat* records.
          • Enums under FederationSubClusterState can afford not having the SC_ prefix - the enum class is enough to distinguish them.. Again trying to follow other enums in YARN.
          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Apologies for chiming in very late. I have more suggestions, please cherry-pick things that you agree with are easy and feel free to defer the harder ones to later JIRAs Drop Federation everywhere in the records? Instead, we can simply say SubClusterInfoProto, SubClusterIdProto etc. None of these classes should be marked @Public . GetFederationSubClusterRequest -> GetSubClusterInfoRequest, similarly response? Similarly the GetSubClustersRequest / response objects? Also, should they be returning the info objects? HeartbeatFederationSubClusterRequest -> SubClusterHeartbeatRequest. Similarly the response? Trying to follow NodeHeartbeat* records. Enums under FederationSubClusterState can afford not having the SC_ prefix - the enum class is enough to distinguish them.. Again trying to follow other enums in YARN.
          Hide
          subru Subru Krishnan added a comment -

          Thanks Vinod Kumar Vavilapalli for reviewing the patch. PFA v6 which addresses your feedback.

          I had to leave the SC_ prefix in the enum as otherwise I get the following error during protobuf compilation:

          yarn_server_federation_protos.proto:33:3: "hadoop.yarn.NEW" is already defined in file "yarn_protos.proto".

          I recall encountering this during YARN-1051 and also NodeStateProto uses a NS_ prefix.

          Show
          subru Subru Krishnan added a comment - Thanks Vinod Kumar Vavilapalli for reviewing the patch. PFA v6 which addresses your feedback. I had to leave the SC_ prefix in the enum as otherwise I get the following error during protobuf compilation: yarn_server_federation_protos.proto:33:3: "hadoop.yarn.NEW" is already defined in file "yarn_protos.proto". I recall encountering this during YARN-1051 and also NodeStateProto uses a NS_ prefix.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 32s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 3m 39s Maven dependency ordering for branch
          +1 mvninstall 8m 15s YARN-2915 passed
          +1 compile 2m 28s YARN-2915 passed
          +1 checkstyle 0m 42s YARN-2915 passed
          +1 mvnsite 0m 55s YARN-2915 passed
          +1 mvneclipse 0m 30s YARN-2915 passed
          +1 findbugs 1m 42s YARN-2915 passed
          +1 javadoc 0m 44s YARN-2915 passed
          0 mvndep 0m 9s Maven dependency ordering for patch
          +1 mvninstall 0m 46s the patch passed
          +1 compile 2m 29s the patch passed
          +1 cc 2m 29s the patch passed
          +1 javac 2m 29s the patch passed
          -1 checkstyle 0m 37s hadoop-yarn-project/hadoop-yarn: The patch generated 9 new + 26 unchanged - 2 fixed = 35 total (was 28)
          +1 mvnsite 0m 52s the patch passed
          +1 mvneclipse 0m 23s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 1m 54s the patch passed
          +1 javadoc 0m 41s the patch passed
          +1 unit 2m 19s hadoop-yarn-common in the patch passed.
          +1 unit 0m 27s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          31m 17s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820778/YARN-3662-YARN-2915-v6.patch
          JIRA Issue YARN-3662
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc
          uname Linux acbe8fef2af4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 85eda58
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12544/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12544/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12544/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 32s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 3m 39s Maven dependency ordering for branch +1 mvninstall 8m 15s YARN-2915 passed +1 compile 2m 28s YARN-2915 passed +1 checkstyle 0m 42s YARN-2915 passed +1 mvnsite 0m 55s YARN-2915 passed +1 mvneclipse 0m 30s YARN-2915 passed +1 findbugs 1m 42s YARN-2915 passed +1 javadoc 0m 44s YARN-2915 passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 0m 46s the patch passed +1 compile 2m 29s the patch passed +1 cc 2m 29s the patch passed +1 javac 2m 29s the patch passed -1 checkstyle 0m 37s hadoop-yarn-project/hadoop-yarn: The patch generated 9 new + 26 unchanged - 2 fixed = 35 total (was 28) +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 1m 54s the patch passed +1 javadoc 0m 41s the patch passed +1 unit 2m 19s hadoop-yarn-common in the patch passed. +1 unit 0m 27s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 31m 17s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820778/YARN-3662-YARN-2915-v6.patch JIRA Issue YARN-3662 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc uname Linux acbe8fef2af4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 85eda58 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12544/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12544/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/12544/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          This looks okay for now.

          I was updating the patch to fix some of the checkstyle issues. Attaching it right now and will commit if Jenkins says okay.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - This looks okay for now. I was updating the patch to fix some of the checkstyle issues. Attaching it right now and will commit if Jenkins says okay.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Same patch but with the public modifier related checkstyle warnings addressed. Others are not addressable.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Same patch but with the public modifier related checkstyle warnings addressed. Others are not addressable.
          Hide
          subru Subru Krishnan added a comment -

          Thanks Vinod Kumar Vavilapalli!

          FYI I deliberately kept the modifiers public as I mentioned earlier but I am fine with adding it when required downstream.

          Show
          subru Subru Krishnan added a comment - Thanks Vinod Kumar Vavilapalli ! FYI I deliberately kept the modifiers public as I mentioned earlier but I am fine with adding it when required downstream.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 8m 22s Maven dependency ordering for branch
          +1 mvninstall 7m 51s YARN-2915 passed
          +1 compile 2m 15s YARN-2915 passed
          +1 checkstyle 0m 40s YARN-2915 passed
          +1 mvnsite 0m 53s YARN-2915 passed
          +1 mvneclipse 0m 30s YARN-2915 passed
          +1 findbugs 1m 35s YARN-2915 passed
          +1 javadoc 0m 43s YARN-2915 passed
          0 mvndep 0m 9s Maven dependency ordering for patch
          +1 mvninstall 0m 43s the patch passed
          +1 compile 2m 13s the patch passed
          +1 cc 2m 13s the patch passed
          +1 javac 2m 13s the patch passed
          -1 checkstyle 0m 35s hadoop-yarn-project/hadoop-yarn: The patch generated 3 new + 26 unchanged - 2 fixed = 29 total (was 28)
          +1 mvnsite 0m 48s the patch passed
          +1 mvneclipse 0m 21s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 1m 42s the patch passed
          +1 javadoc 0m 39s the patch passed
          +1 unit 2m 14s hadoop-yarn-common in the patch passed.
          +1 unit 0m 26s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          34m 5s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821044/YARN-3662-YARN-2915-v7.patch
          JIRA Issue YARN-3662
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc
          uname Linux b0095c28649f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 85eda58
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12565/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12565/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12565/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 8m 22s Maven dependency ordering for branch +1 mvninstall 7m 51s YARN-2915 passed +1 compile 2m 15s YARN-2915 passed +1 checkstyle 0m 40s YARN-2915 passed +1 mvnsite 0m 53s YARN-2915 passed +1 mvneclipse 0m 30s YARN-2915 passed +1 findbugs 1m 35s YARN-2915 passed +1 javadoc 0m 43s YARN-2915 passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 0m 43s the patch passed +1 compile 2m 13s the patch passed +1 cc 2m 13s the patch passed +1 javac 2m 13s the patch passed -1 checkstyle 0m 35s hadoop-yarn-project/hadoop-yarn: The patch generated 3 new + 26 unchanged - 2 fixed = 29 total (was 28) +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 21s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 42s the patch passed +1 javadoc 0m 39s the patch passed +1 unit 2m 14s hadoop-yarn-common in the patch passed. +1 unit 0m 26s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 34m 5s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821044/YARN-3662-YARN-2915-v7.patch JIRA Issue YARN-3662 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc uname Linux b0095c28649f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 85eda58 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12565/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12565/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/12565/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          subru Subru Krishnan added a comment -

          I just committed this to branch YARN-2915.

          Thanks to Wangda Tan, Varun Vasudev, Vinod Kumar Vavilapalli and Karthik Kambatla for reviewing this thoroughly and come up with a robust API for Federation.

          Show
          subru Subru Krishnan added a comment - I just committed this to branch YARN-2915 . Thanks to Wangda Tan , Varun Vasudev , Vinod Kumar Vavilapalli and Karthik Kambatla for reviewing this thoroughly and come up with a robust API for Federation.

            People

            • Assignee:
              subru Subru Krishnan
              Reporter:
              subru Subru Krishnan
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development