Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-149 [Umbrella] ResourceManager (RM) Fail-over
  3. YARN-4243

Add retry on establishing Zookeeper conenction in EmbeddedElectorService#serviceInit

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: resourcemanager
    • Labels:
      None

      Description

      Right now, the RM would shut down if the zk connection is down when the RM do the initialization. We need to add retry on this part

      1. YARN-4243.1.patch
        4 kB
        Xuan Gong
      2. YARN-4243.2.1.patch
        6 kB
        Xuan Gong
      3. YARN-4243.2.patch
        6 kB
        Xuan Gong
      4. YARN-4243.3.patch
        6 kB
        Xuan Gong
      5. YARN-4243.4.patch
        7 kB
        Xuan Gong
      6. YARN-4243.5.patch
        7 kB
        Xuan Gong

        Issue Links

          Activity

          Hide
          xgong Xuan Gong added a comment -

          Override the createConnection() in EmbeddedElectorService to add some retry, and create a Yarn Configuration for the maxAttempts because we have shared code (ActiveStandbyElector)and related configuration with HDFS ZKFC

          Show
          xgong Xuan Gong added a comment - Override the createConnection() in EmbeddedElectorService to add some retry, and create a Yarn Configuration for the maxAttempts because we have shared code (ActiveStandbyElector)and related configuration with HDFS ZKFC
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 22m 44s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 8m 59s There were no new javac warning messages.
          +1 javadoc 11m 53s There were no new javadoc warning messages.
          -1 release audit 0m 22s The applied patch generated 1 release audit warnings.
          -1 checkstyle 3m 1s The applied patch generated 2 new checkstyle issues (total was 211, now 212).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 52s mvn install still works.
          +1 eclipse:eclipse 0m 41s The patch built with eclipse:eclipse.
          +1 findbugs 5m 36s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 19m 18s Tests failed in hadoop-common.
          -1 yarn tests 0m 24s Tests failed in hadoop-yarn-api.
          -1 yarn tests 62m 59s Tests failed in hadoop-yarn-server-resourcemanager.
              138m 8s  



          Reason Tests
          Failed unit tests hadoop.yarn.conf.TestYarnConfigurationFields
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
          Timed out tests org.apache.hadoop.http.TestHttpServerLifecycle



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12765723/YARN-4243.1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / e1bf8b3
          Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9386/artifact/patchprocess/patchReleaseAuditProblems.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9386/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9386/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9386/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9386/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9386/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9386/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 22m 44s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 59s There were no new javac warning messages. +1 javadoc 11m 53s There were no new javadoc warning messages. -1 release audit 0m 22s The applied patch generated 1 release audit warnings. -1 checkstyle 3m 1s The applied patch generated 2 new checkstyle issues (total was 211, now 212). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 52s mvn install still works. +1 eclipse:eclipse 0m 41s The patch built with eclipse:eclipse. +1 findbugs 5m 36s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 19m 18s Tests failed in hadoop-common. -1 yarn tests 0m 24s Tests failed in hadoop-yarn-api. -1 yarn tests 62m 59s Tests failed in hadoop-yarn-server-resourcemanager.     138m 8s   Reason Tests Failed unit tests hadoop.yarn.conf.TestYarnConfigurationFields   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler Timed out tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765723/YARN-4243.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / e1bf8b3 Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9386/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9386/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9386/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9386/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9386/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9386/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9386/console This message was automatically generated.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Thanks Xuan Gong for working on this.
          Some comments and suggestions

          1. While initializing Elector service createConnection will retry as per configured value i.e maxRetryNum say 10. But if session is closed and reestablished then number of retry count will be maxRetryNum * maxRetryNum i.e 10*10=100 times.
          2. And method reEstablishSession() can be reused rather duplicating same logic over embedded electors. Instead of overriding createConnection() method, reEstablishSession() method can be used in ActiveStandByElector constructor.I'd prefer to make change in hadoop-common rather in embedded elector service.
          Show
          rohithsharma Rohith Sharma K S added a comment - Thanks Xuan Gong for working on this. Some comments and suggestions While initializing Elector service createConnection will retry as per configured value i.e maxRetryNum say 10. But if session is closed and reestablished then number of retry count will be maxRetryNum * maxRetryNum i.e 10*10=100 times. And method reEstablishSession() can be reused rather duplicating same logic over embedded electors. Instead of overriding createConnection() method, reEstablishSession() method can be used in ActiveStandByElector constructor.I'd prefer to make change in hadoop-common rather in embedded elector service.
          Hide
          djp Junping Du added a comment -

          Thanks for reporting the issue and delivering the patch, Xuan Gong!
          The patch make sense in overall. Some minor comments:
          1. I think we are adding a new configuration here, and we may want to add it to yarn-default.xml as well. It is only for documentation purpose and we don't have to specify default value though.
          2. Do we need to add another configuration for sleep interval during retry? hard coded with 5 seconds sounds lack of flexibility.
          3. If connection still get failed after max retry times, shall we put retry times in error messages as well? like: "Can not establish Zookeeper Connection... after retry x times").

          Show
          djp Junping Du added a comment - Thanks for reporting the issue and delivering the patch, Xuan Gong ! The patch make sense in overall. Some minor comments: 1. I think we are adding a new configuration here, and we may want to add it to yarn-default.xml as well. It is only for documentation purpose and we don't have to specify default value though. 2. Do we need to add another configuration for sleep interval during retry? hard coded with 5 seconds sounds lack of flexibility. 3. If connection still get failed after max retry times, shall we put retry times in error messages as well? like: "Can not establish Zookeeper Connection... after retry x times").
          Hide
          kasha Karthik Kambatla added a comment -

          I would like to review the patch before commit.

          Show
          kasha Karthik Kambatla added a comment - I would like to review the patch before commit.
          Hide
          djp Junping Du added a comment -

          No worry. Nobody want to commit it right now as we all leave concrete review/improvement comments.

          Show
          djp Junping Du added a comment - No worry. Nobody want to commit it right now as we all leave concrete review/improvement comments.
          Hide
          xgong Xuan Gong added a comment -

          Rohith Sharma K S Thanks for the review.

          And method reEstablishSession() can be reused rather duplicating same logic over embedded electors. Instead of overriding createConnection() method, reEstablishSession() method can be used in ActiveStandByElector constructor.I'd prefer to make change in hadoop-common rather in embedded elector service.

          This will affect the HDFS ZKFS, and they do not want the retry on initialization.

          While initializing Elector service createConnection will retry as per configured value i.e maxRetryNum say 10. But if session is closed and reestablished then number of retry count will be maxRetryNum * maxRetryNum i.e 10*10=100 times.

          I am not sure if I understand correctly. If we set the maxRetryNum as 10, and zk connect itself would do some retries (10times), the total is 10*10.

          Show
          xgong Xuan Gong added a comment - Rohith Sharma K S Thanks for the review. And method reEstablishSession() can be reused rather duplicating same logic over embedded electors. Instead of overriding createConnection() method, reEstablishSession() method can be used in ActiveStandByElector constructor.I'd prefer to make change in hadoop-common rather in embedded elector service. This will affect the HDFS ZKFS, and they do not want the retry on initialization. While initializing Elector service createConnection will retry as per configured value i.e maxRetryNum say 10. But if session is closed and reestablished then number of retry count will be maxRetryNum * maxRetryNum i.e 10*10=100 times. I am not sure if I understand correctly. If we set the maxRetryNum as 10, and zk connect itself would do some retries (10times), the total is 10*10.
          Hide
          xgong Xuan Gong added a comment -

          Thanks for the review, Junping Du
          Will create a new patch to address your comments

          Show
          xgong Xuan Gong added a comment - Thanks for the review, Junping Du Will create a new patch to address your comments
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 22m 9s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 8m 16s There were no new javac warning messages.
          +1 javadoc 10m 33s There were no new javadoc warning messages.
          -1 release audit 0m 20s The applied patch generated 1 release audit warnings.
          -1 checkstyle 3m 13s The applied patch generated 2 new checkstyle issues (total was 211, now 212).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 38s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 6m 38s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 7m 25s Tests passed in hadoop-common.
          +1 yarn tests 0m 25s Tests passed in hadoop-yarn-api.
          +1 yarn tests 2m 2s Tests passed in hadoop-yarn-common.
          -1 yarn tests 62m 25s Tests failed in hadoop-yarn-server-resourcemanager.
              126m 21s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12766161/YARN-4243.2.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 9849c8b
          Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9412/artifact/patchprocess/patchReleaseAuditProblems.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9412/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9412/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9412/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9412/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9412/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9412/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9412/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 22m 9s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 16s There were no new javac warning messages. +1 javadoc 10m 33s There were no new javadoc warning messages. -1 release audit 0m 20s The applied patch generated 1 release audit warnings. -1 checkstyle 3m 13s The applied patch generated 2 new checkstyle issues (total was 211, now 212). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 38s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 6m 38s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 7m 25s Tests passed in hadoop-common. +1 yarn tests 0m 25s Tests passed in hadoop-yarn-api. +1 yarn tests 2m 2s Tests passed in hadoop-yarn-common. -1 yarn tests 62m 25s Tests failed in hadoop-yarn-server-resourcemanager.     126m 21s   Reason Tests Failed unit tests hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12766161/YARN-4243.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 9849c8b Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9412/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9412/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9412/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9412/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9412/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9412/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9412/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9412/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 24m 46s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 9m 11s There were no new javac warning messages.
          +1 javadoc 11m 36s There were no new javadoc warning messages.
          +1 release audit 0m 26s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 3m 33s The applied patch generated 4 new checkstyle issues (total was 211, now 214).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 46s mvn install still works.
          +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse.
          +1 findbugs 7m 26s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 8m 12s Tests failed in hadoop-common.
          +1 yarn tests 0m 28s Tests passed in hadoop-yarn-api.
          +1 yarn tests 2m 12s Tests passed in hadoop-yarn-common.
          -1 yarn tests 0m 21s Tests failed in hadoop-yarn-server-resourcemanager.
              71m 26s  



          Reason Tests
          Failed unit tests hadoop.metrics2.impl.TestMetricsSystemImpl
          Failed build hadoop-yarn-server-resourcemanager



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12766207/YARN-4243.2.1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / c60a16f
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9415/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9415/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9415/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9415/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9415/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9415/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9415/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 24m 46s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 9m 11s There were no new javac warning messages. +1 javadoc 11m 36s There were no new javadoc warning messages. +1 release audit 0m 26s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 3m 33s The applied patch generated 4 new checkstyle issues (total was 211, now 214). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 46s mvn install still works. +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse. +1 findbugs 7m 26s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 8m 12s Tests failed in hadoop-common. +1 yarn tests 0m 28s Tests passed in hadoop-yarn-api. +1 yarn tests 2m 12s Tests passed in hadoop-yarn-common. -1 yarn tests 0m 21s Tests failed in hadoop-yarn-server-resourcemanager.     71m 26s   Reason Tests Failed unit tests hadoop.metrics2.impl.TestMetricsSystemImpl Failed build hadoop-yarn-server-resourcemanager Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12766207/YARN-4243.2.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / c60a16f checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9415/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9415/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9415/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9415/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9415/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9415/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9415/console This message was automatically generated.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          This will affect the HDFS ZKFS, and they do not want the retry on initialization.

          Given default ability to retry on initialization do not require by ZKFS, then is fine.

          If we set the maxRetryNum as 10, and zk connect itself would do some retries (10times), the total is 10*10.

          Sorry I did not get it. Could you explain bit more.

          Show
          rohithsharma Rohith Sharma K S added a comment - This will affect the HDFS ZKFS, and they do not want the retry on initialization. Given default ability to retry on initialization do not require by ZKFS, then is fine. If we set the maxRetryNum as 10, and zk connect itself would do some retries (10times) , the total is 10*10. Sorry I did not get it. Could you explain bit more.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 21m 53s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 58s There were no new javac warning messages.
          +1 javadoc 10m 20s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 3m 8s The applied patch generated 2 new checkstyle issues (total was 211, now 212).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse.
          +1 findbugs 6m 39s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 8m 24s Tests passed in hadoop-common.
          +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api.
          +1 yarn tests 2m 3s Tests passed in hadoop-yarn-common.
          +1 yarn tests 57m 23s Tests passed in hadoop-yarn-server-resourcemanager.
              121m 33s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12766368/YARN-4243.3.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 0d77e85
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9445/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9445/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9445/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9445/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9445/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9445/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9445/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 21m 53s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 58s There were no new javac warning messages. +1 javadoc 10m 20s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 3m 8s The applied patch generated 2 new checkstyle issues (total was 211, now 212). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse. +1 findbugs 6m 39s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 8m 24s Tests passed in hadoop-common. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api. +1 yarn tests 2m 3s Tests passed in hadoop-yarn-common. +1 yarn tests 57m 23s Tests passed in hadoop-yarn-server-resourcemanager.     121m 33s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12766368/YARN-4243.3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0d77e85 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9445/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9445/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9445/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9445/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9445/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9445/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9445/console This message was automatically generated.
          Hide
          xgong Xuan Gong added a comment -

          Karthik Kambatla Could you review the latest patch, please ?

          Show
          xgong Xuan Gong added a comment - Karthik Kambatla Could you review the latest patch, please ?
          Hide
          kasha Karthik Kambatla added a comment -

          Looking.

          Show
          kasha Karthik Kambatla added a comment - Looking.
          Hide
          kasha Karthik Kambatla added a comment -

          Looks like we are addressing two issues here:

          1. Have createConnection() retry connecting to ZK.
            1. I am with Rohith on this one - I think changing ActiveStandbyElector constructor either to use reestablishConnection or otherwise seems like the right approach. Do we know why the HDFS devs don't want connections to be retried on init, but are fine with it on reestablishConnection?
          2. Add a config to be able to set a different number of retries for Yarn.
            1. Sounds reasonable. Code comments - can we do the following instead:
              int maxRetryNum = conf.getInt(YarnConfiguration.RM_HA_FC_ELECTOR_ZK_OP_RETRIES_KEY,
                                                           conf.getInt(CommonConfigurationKeys.HA_FC_ELECTOR_ZK_OP_RETRIES_KEY,
                                                                             CommonConfigurationKeys.HA_FC_ELECTOR_ZK_OP_RETRIES_DEFAULT));
              
          Show
          kasha Karthik Kambatla added a comment - Looks like we are addressing two issues here: Have createConnection() retry connecting to ZK. I am with Rohith on this one - I think changing ActiveStandbyElector constructor either to use reestablishConnection or otherwise seems like the right approach. Do we know why the HDFS devs don't want connections to be retried on init, but are fine with it on reestablishConnection? Add a config to be able to set a different number of retries for Yarn. Sounds reasonable. Code comments - can we do the following instead: int maxRetryNum = conf.getInt(YarnConfiguration.RM_HA_FC_ELECTOR_ZK_OP_RETRIES_KEY, conf.getInt(CommonConfigurationKeys.HA_FC_ELECTOR_ZK_OP_RETRIES_KEY, CommonConfigurationKeys.HA_FC_ELECTOR_ZK_OP_RETRIES_DEFAULT));
          Hide
          kasha Karthik Kambatla added a comment -

          Oh, and Rohith's comment about this patch leading to retrying (numRetries * numRetries) times on reestablishConnection is a concern too.

          Show
          kasha Karthik Kambatla added a comment - Oh, and Rohith's comment about this patch leading to retrying (numRetries * numRetries) times on reestablishConnection is a concern too.
          Hide
          xgong Xuan Gong added a comment -

          Thanks for Karthik Kambatla 's suggestion.
          Upload a new patch which define a new constructor in ActiveStandbyElector which add a new parameter: failfast.

          Show
          xgong Xuan Gong added a comment - Thanks for Karthik Kambatla 's suggestion. Upload a new patch which define a new constructor in ActiveStandbyElector which add a new parameter: failfast.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 22m 1s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 55s There were no new javac warning messages.
          +1 javadoc 10m 35s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 35s The applied patch generated 1 new checkstyle issues (total was 15, now 16).
          -1 checkstyle 3m 9s The applied patch generated 2 new checkstyle issues (total was 211, now 212).
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 32s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 6m 35s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 6m 52s Tests passed in hadoop-common.
          +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api.
          +1 yarn tests 1m 59s Tests passed in hadoop-yarn-common.
          +1 yarn tests 57m 5s Tests passed in hadoop-yarn-server-resourcemanager.
              119m 52s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12766837/YARN-4243.4.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 8d2d3eb
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9460/artifact/patchprocess/diffcheckstylehadoop-common.txt https://builds.apache.org/job/PreCommit-YARN-Build/9460/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9460/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9460/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9460/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9460/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9460/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9460/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9460/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 22m 1s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 55s There were no new javac warning messages. +1 javadoc 10m 35s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 35s The applied patch generated 1 new checkstyle issues (total was 15, now 16). -1 checkstyle 3m 9s The applied patch generated 2 new checkstyle issues (total was 211, now 212). -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 32s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 6m 35s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 6m 52s Tests passed in hadoop-common. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api. +1 yarn tests 1m 59s Tests passed in hadoop-yarn-common. +1 yarn tests 57m 5s Tests passed in hadoop-yarn-server-resourcemanager.     119m 52s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12766837/YARN-4243.4.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8d2d3eb checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9460/artifact/patchprocess/diffcheckstylehadoop-common.txt https://builds.apache.org/job/PreCommit-YARN-Build/9460/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9460/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9460/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9460/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9460/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9460/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9460/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9460/console This message was automatically generated.
          Hide
          xgong Xuan Gong added a comment -

          Karthik Kambatla Do we have any other comments ?

          Show
          xgong Xuan Gong added a comment - Karthik Kambatla Do we have any other comments ?
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          I am +1 for the latest patch. The patch also keeps old behavior and gives options for the users to fail fast.

          Show
          rohithsharma Rohith Sharma K S added a comment - I am +1 for the latest patch. The patch also keeps old behavior and gives options for the users to fail fast.
          Hide
          xgong Xuan Gong added a comment -

          Karthik Kambatla Any comments for the latest patch ?

          Show
          xgong Xuan Gong added a comment - Karthik Kambatla Any comments for the latest patch ?
          Hide
          djp Junping Du added a comment -

          +1. I will commit latest patch soon if no further comments from community.

          Show
          djp Junping Du added a comment - +1. I will commit latest patch soon if no further comments from community.
          Hide
          kasha Karthik Kambatla added a comment -

          Thanks for the update, Xuan. Sorry for the delay in getting to this.

          Just one nit: To be consistent with the other config, can we call use zk-retries instead of zk.op.retries? I am +1 otherwise.

          One other thing to consider - should we make the change to ActiveStandbyElector as a Common JIRA or at least create a Common JIRA and close it as part of this one, so the common and HDFS devs are aware of this change? They might want to update the way HDFS handles the retries situation as well.

          Show
          kasha Karthik Kambatla added a comment - Thanks for the update, Xuan. Sorry for the delay in getting to this. Just one nit: To be consistent with the other config, can we call use zk-retries instead of zk.op.retries? I am +1 otherwise. One other thing to consider - should we make the change to ActiveStandbyElector as a Common JIRA or at least create a Common JIRA and close it as part of this one, so the common and HDFS devs are aware of this change? They might want to update the way HDFS handles the retries situation as well.
          Hide
          xgong Xuan Gong added a comment -

          Thanks for the comments, Karthik Kambatla

          To be consistent with the other config, can we call use zk-retries instead of zk.op.retries

          Modified

          should we make the change to ActiveStandbyElector as a Common JIRA or at least create a Common JIRA and close it as part of this one, so the common and HDFS devs are aware of this change? They might want to update the way HDFS handles the retries situation as well.

          Created https://issues.apache.org/jira/browse/HADOOP-12503, and link the ticket

          Show
          xgong Xuan Gong added a comment - Thanks for the comments, Karthik Kambatla To be consistent with the other config, can we call use zk-retries instead of zk.op.retries Modified should we make the change to ActiveStandbyElector as a Common JIRA or at least create a Common JIRA and close it as part of this one, so the common and HDFS devs are aware of this change? They might want to update the way HDFS handles the retries situation as well. Created https://issues.apache.org/jira/browse/HADOOP-12503 , and link the ticket
          Hide
          djp Junping Du added a comment -

          v5 patch LGTM. +1 pending on Jenkins result.

          Show
          djp Junping Du added a comment - v5 patch LGTM. +1 pending on Jenkins result.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 22m 16s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 8m 12s There were no new javac warning messages.
          +1 javadoc 10m 42s There were no new javadoc warning messages.
          +1 release audit 0m 27s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 44s The applied patch generated 1 new checkstyle issues (total was 15, now 16).
          -1 checkstyle 3m 17s The applied patch generated 2 new checkstyle issues (total was 211, now 212).
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 43s mvn install still works.
          +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse.
          +1 findbugs 6m 35s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 7m 33s Tests passed in hadoop-common.
          +1 yarn tests 0m 25s Tests passed in hadoop-yarn-api.
          -1 yarn tests 2m 0s Tests failed in hadoop-yarn-common.
          -1 yarn tests 68m 50s Tests failed in hadoop-yarn-server-resourcemanager.
              133m 22s  



          Reason Tests
          Failed unit tests hadoop.yarn.logaggregation.TestAggregatedLogsBlock
            hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12768079/YARN-4243.5.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 4c0bae2
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/diffcheckstylehadoop-common.txt https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9526/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9526/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 22m 16s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 12s There were no new javac warning messages. +1 javadoc 10m 42s There were no new javadoc warning messages. +1 release audit 0m 27s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 44s The applied patch generated 1 new checkstyle issues (total was 15, now 16). -1 checkstyle 3m 17s The applied patch generated 2 new checkstyle issues (total was 211, now 212). -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 43s mvn install still works. +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse. +1 findbugs 6m 35s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 7m 33s Tests passed in hadoop-common. +1 yarn tests 0m 25s Tests passed in hadoop-yarn-api. -1 yarn tests 2m 0s Tests failed in hadoop-yarn-common. -1 yarn tests 68m 50s Tests failed in hadoop-yarn-server-resourcemanager.     133m 22s   Reason Tests Failed unit tests hadoop.yarn.logaggregation.TestAggregatedLogsBlock   hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768079/YARN-4243.5.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 4c0bae2 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/diffcheckstylehadoop-common.txt https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9526/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9526/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9526/console This message was automatically generated.
          Hide
          xgong Xuan Gong added a comment -

          Testcase failures and findbug warnings are not related

          Show
          xgong Xuan Gong added a comment - Testcase failures and findbug warnings are not related
          Hide
          djp Junping Du added a comment -

          Committing it now.

          Show
          djp Junping Du added a comment - Committing it now.
          Hide
          djp Junping Du added a comment -

          I have commit the v5 patch to trunk and branch-2. Thanks Xuan Gong for contributing the patch and review comments from Rohith Sharma K S and Karthik Kambatla!

          Show
          djp Junping Du added a comment - I have commit the v5 patch to trunk and branch-2. Thanks Xuan Gong for contributing the patch and review comments from Rohith Sharma K S and Karthik Kambatla !
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8692 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8692/)
          YARN-4243. Add retry on establishing Zookeeper conenction in (junping_du: rev 0fce5f9a496925f0d53ea6c14318c9b513de9882)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/EmbeddedElectorService.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8692 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8692/ ) YARN-4243 . Add retry on establishing Zookeeper conenction in (junping_du: rev 0fce5f9a496925f0d53ea6c14318c9b513de9882) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/EmbeddedElectorService.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #571 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/571/)
          YARN-4243. Add retry on establishing Zookeeper conenction in (junping_du: rev 0fce5f9a496925f0d53ea6c14318c9b513de9882)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/EmbeddedElectorService.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #571 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/571/ ) YARN-4243 . Add retry on establishing Zookeeper conenction in (junping_du: rev 0fce5f9a496925f0d53ea6c14318c9b513de9882) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/EmbeddedElectorService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #1307 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1307/)
          YARN-4243. Add retry on establishing Zookeeper conenction in (junping_du: rev 0fce5f9a496925f0d53ea6c14318c9b513de9882)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/EmbeddedElectorService.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1307 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1307/ ) YARN-4243 . Add retry on establishing Zookeeper conenction in (junping_du: rev 0fce5f9a496925f0d53ea6c14318c9b513de9882) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/EmbeddedElectorService.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2517 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2517/)
          YARN-4243. Add retry on establishing Zookeeper conenction in (junping_du: rev 0fce5f9a496925f0d53ea6c14318c9b513de9882)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/EmbeddedElectorService.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2517 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2517/ ) YARN-4243 . Add retry on establishing Zookeeper conenction in (junping_du: rev 0fce5f9a496925f0d53ea6c14318c9b513de9882) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/EmbeddedElectorService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #586 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/586/)
          YARN-4243. Add retry on establishing Zookeeper conenction in (junping_du: rev 0fce5f9a496925f0d53ea6c14318c9b513de9882)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/EmbeddedElectorService.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #586 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/586/ ) YARN-4243 . Add retry on establishing Zookeeper conenction in (junping_du: rev 0fce5f9a496925f0d53ea6c14318c9b513de9882) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/EmbeddedElectorService.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2465 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2465/)
          YARN-4243. Add retry on establishing Zookeeper conenction in (junping_du: rev 0fce5f9a496925f0d53ea6c14318c9b513de9882)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/EmbeddedElectorService.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2465 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2465/ ) YARN-4243 . Add retry on establishing Zookeeper conenction in (junping_du: rev 0fce5f9a496925f0d53ea6c14318c9b513de9882) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/EmbeddedElectorService.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #528 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/528/)
          YARN-4243. Add retry on establishing Zookeeper conenction in (junping_du: rev 0fce5f9a496925f0d53ea6c14318c9b513de9882)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/EmbeddedElectorService.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #528 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/528/ ) YARN-4243 . Add retry on establishing Zookeeper conenction in (junping_du: rev 0fce5f9a496925f0d53ea6c14318c9b513de9882) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/EmbeddedElectorService.java

            People

            • Assignee:
              xgong Xuan Gong
              Reporter:
              xgong Xuan Gong
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development