Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      YARN-3662 defines the FederationMembershipStateStore API. This JIRA tracks an in-memory based implementation which is useful for both single-box testing and for future unit tests that depend on the state store.

      1. YARN-5406-YARN-2915.v0.patch
        18 kB
        Ellen Hui
      2. YARN-5406-YARN-2915.v1.patch
        21 kB
        Ellen Hui

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 8m 24s YARN-2915 passed
          +1 compile 0m 22s YARN-2915 passed
          +1 checkstyle 0m 12s YARN-2915 passed
          +1 mvnsite 0m 24s YARN-2915 passed
          +1 mvneclipse 0m 14s YARN-2915 passed
          +1 findbugs 0m 42s YARN-2915 passed
          +1 javadoc 0m 17s YARN-2915 passed
          +1 mvninstall 0m 21s the patch passed
          +1 compile 0m 21s the patch passed
          +1 javac 0m 21s the patch passed
          +1 checkstyle 0m 11s the patch passed
          +1 mvnsite 0m 26s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 54s the patch passed
          +1 javadoc 0m 16s the patch passed
          +1 unit 0m 32s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          15m 9s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821440/YARN-5406-YARN-2915.v0.patch
          JIRA Issue YARN-5406
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 17b03c597981 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 22db8fd
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12598/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12598/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 24s YARN-2915 passed +1 compile 0m 22s YARN-2915 passed +1 checkstyle 0m 12s YARN-2915 passed +1 mvnsite 0m 24s YARN-2915 passed +1 mvneclipse 0m 14s YARN-2915 passed +1 findbugs 0m 42s YARN-2915 passed +1 javadoc 0m 17s YARN-2915 passed +1 mvninstall 0m 21s the patch passed +1 compile 0m 21s the patch passed +1 javac 0m 21s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 26s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 54s the patch passed +1 javadoc 0m 16s the patch passed +1 unit 0m 32s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 15m 9s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821440/YARN-5406-YARN-2915.v0.patch JIRA Issue YARN-5406 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 17b03c597981 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 22db8fd Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12598/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/12598/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          jianhe Jian He added a comment -

          lgtm, not related to this patch.
          I think mentioning the default value in javadoc for FilterInactiveSubClusters is useful for user. I had to open up the proto definition file to figure out what the default value is.

          Show
          jianhe Jian He added a comment - lgtm, not related to this patch. I think mentioning the default value in javadoc for FilterInactiveSubClusters is useful for user. I had to open up the proto definition file to figure out what the default value is.
          Hide
          jianhe Jian He added a comment -

          Also, should the .impl package be under the .store package ?

          Show
          jianhe Jian He added a comment - Also, should the .impl package be under the .store package ?
          Hide
          subru Subru Krishnan added a comment -

          Thanks Ellen Hui for the patch and Jian He for the review.

          I have a few minor comments:

          • I agree with Jian He that the impl package should be a sub-package of the store package.
          • Rename FederationInMemoryMembershipStateStore --> MemoryFederationStateStore and the corresponding test.
          • We need to validate the inputs (like null checks). Since this is common across different store implementations, I have created YARN-5467 to track this.
          • All the tests are for positive cases, can we add a few for negative cases.
          • I think we should add a isSubClusterActive method to SubClusterState and use it.
          • Can you update the Javadoc for FilterInactiveSubClusters as requested by Jian He.
          Show
          subru Subru Krishnan added a comment - Thanks Ellen Hui for the patch and Jian He for the review. I have a few minor comments: I agree with Jian He that the impl package should be a sub-package of the store package. Rename FederationInMemoryMembershipStateStore --> MemoryFederationStateStore and the corresponding test. We need to validate the inputs (like null checks). Since this is common across different store implementations, I have created YARN-5467 to track this. All the tests are for positive cases, can we add a few for negative cases. I think we should add a isSubClusterActive method to SubClusterState and use it. Can you update the Javadoc for FilterInactiveSubClusters as requested by Jian He .
          Hide
          ellenfkh Ellen Hui added a comment -

          Addressed feedback from Subru Krishnan and Jian He.

          Show
          ellenfkh Ellen Hui added a comment - Addressed feedback from Subru Krishnan and Jian He .
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 10m 51s YARN-2915 passed
          +1 compile 0m 19s YARN-2915 passed
          +1 checkstyle 0m 15s YARN-2915 passed
          +1 mvnsite 0m 24s YARN-2915 passed
          +1 mvneclipse 0m 16s YARN-2915 passed
          +1 findbugs 0m 47s YARN-2915 passed
          +1 javadoc 0m 16s YARN-2915 passed
          +1 mvninstall 0m 18s the patch passed
          +1 compile 0m 17s the patch passed
          +1 javac 0m 17s the patch passed
          +1 checkstyle 0m 10s the patch passed
          +1 mvnsite 0m 20s the patch passed
          +1 mvneclipse 0m 9s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 43s the patch passed
          +1 javadoc 0m 12s the patch passed
          +1 unit 0m 27s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          16m 47s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821899/YARN-5406-YARN-2915.v1.patch
          JIRA Issue YARN-5406
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 7dccb0e8b201 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 22db8fd
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12631/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12631/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 10m 51s YARN-2915 passed +1 compile 0m 19s YARN-2915 passed +1 checkstyle 0m 15s YARN-2915 passed +1 mvnsite 0m 24s YARN-2915 passed +1 mvneclipse 0m 16s YARN-2915 passed +1 findbugs 0m 47s YARN-2915 passed +1 javadoc 0m 16s YARN-2915 passed +1 mvninstall 0m 18s the patch passed +1 compile 0m 17s the patch passed +1 javac 0m 17s the patch passed +1 checkstyle 0m 10s the patch passed +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 43s the patch passed +1 javadoc 0m 12s the patch passed +1 unit 0m 27s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 16m 47s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821899/YARN-5406-YARN-2915.v1.patch JIRA Issue YARN-5406 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7dccb0e8b201 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 22db8fd Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12631/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/12631/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          subru Subru Krishnan added a comment -

          +1 on the latest patch. Thanks Ellen Hui for addressing our comments, I'll commit it shortly.

          Show
          subru Subru Krishnan added a comment - +1 on the latest patch. Thanks Ellen Hui for addressing our comments, I'll commit it shortly.
          Hide
          subru Subru Krishnan added a comment -

          I just committed this to branch YARN-2915. Thanks Ellen Hui for the contribution and Jian He for the review.

          Show
          subru Subru Krishnan added a comment - I just committed this to branch YARN-2915 . Thanks Ellen Hui for the contribution and Jian He for the review.

            People

            • Assignee:
              ellenfkh Ellen Hui
              Reporter:
              subru Subru Krishnan
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development