Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.3, 3.0.0-alpha2
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The webapp directory for the SharedCacheManager is missing and the SCM fails to start up with the following:

      2017-01-22 00:14:25,162 INFO org.apache.hadoop.service.AbstractService: Service SharedCacheManager failed in state STARTED; cause: org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
      org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
              at org.apache.hadoop.yarn.webapp.WebApps$Builder.build(WebApps.java:330)
              at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:377)
              at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:373)
              at org.apache.hadoop.yarn.server.sharedcachemanager.webapp.SCMWebServer.serviceStart(SCMWebServer.java:65)
              at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
              at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
              at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
              at org.apache.hadoop.yarn.server.sharedcachemanager.SharedCacheManager.main(SharedCacheManager.java:157)
      Caused by: java.io.FileNotFoundException: webapps/sharedcache not found in CLASSPATH
              at org.apache.hadoop.http.HttpServer2.getWebAppsPath(HttpServer2.java:972)
              at org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:478)
              at org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:117)
              at org.apache.hadoop.http.HttpServer2$Builder.build(HttpServer2.java:392)
              at org.apache.hadoop.yarn.webapp.WebApps$Builder.build(WebApps.java:291)
              ... 7 more
      

        Issue Links

          Activity

          Hide
          ctrezzo Chris Trezzo added a comment -

          Working on the patch now. I will post v1 shortly.

          Show
          ctrezzo Chris Trezzo added a comment - Working on the patch now. I will post v1 shortly.
          Hide
          ctrezzo Chris Trezzo added a comment -

          Attached is a v1 patch. This patch adds the webapps/sharedcache directory and a rat exclude for the .keep file.

          Show
          ctrezzo Chris Trezzo added a comment - Attached is a v1 patch. This patch adds the webapps/sharedcache directory and a rat exclude for the .keep file.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 17m 12s trunk passed
          +1 compile 0m 42s trunk passed
          +1 mvnsite 0m 45s trunk passed
          +1 mvneclipse 0m 18s trunk passed
          +1 javadoc 0m 41s trunk passed
          +1 mvninstall 0m 42s the patch passed
          +1 compile 0m 42s the patch passed
          +1 javac 0m 42s the patch passed
          +1 mvnsite 0m 43s the patch passed
          +1 mvneclipse 0m 16s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 javadoc 0m 38s the patch passed
          +1 unit 2m 44s hadoop-yarn-common in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          26m 42s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue YARN-6117
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12849008/YARN-6117-trunk.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 11e2e9b7e37e 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ce83205
          Default Java 1.8.0_121
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14737/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/14737/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 17m 12s trunk passed +1 compile 0m 42s trunk passed +1 mvnsite 0m 45s trunk passed +1 mvneclipse 0m 18s trunk passed +1 javadoc 0m 41s trunk passed +1 mvninstall 0m 42s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed +1 mvnsite 0m 43s the patch passed +1 mvneclipse 0m 16s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 javadoc 0m 38s the patch passed +1 unit 2m 44s hadoop-yarn-common in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 26m 42s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6117 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12849008/YARN-6117-trunk.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 11e2e9b7e37e 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ce83205 Default Java 1.8.0_121 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14737/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/14737/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          sjlee0 Sangjin Lee added a comment -

          +1. Committing shortly.

          Show
          sjlee0 Sangjin Lee added a comment - +1. Committing shortly.
          Hide
          sjlee0 Sangjin Lee added a comment -

          Committed it to trunk (3.0.0-alpha3). Thanks Chris Trezzo for a quick fix! Let me know if we want to port this to earlier versions.

          Show
          sjlee0 Sangjin Lee added a comment - Committed it to trunk (3.0.0-alpha3). Thanks Chris Trezzo for a quick fix! Let me know if we want to port this to earlier versions.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11164 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11164/)
          YARN-6117. SharedCacheManager does not start up. Contributed by Chris (sjlee: rev dc6ec9704829f180ce0e182c436fe1a435744c88)

          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/pom.xml
          • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/sharedcache/.keep
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11164 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11164/ ) YARN-6117 . SharedCacheManager does not start up. Contributed by Chris (sjlee: rev dc6ec9704829f180ce0e182c436fe1a435744c88) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/pom.xml (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/sharedcache/.keep
          Hide
          ctrezzo Chris Trezzo added a comment -

          Thanks Sangjin Lee! I think it would be good to backport to branch-2 and hopefully branch-2.8 so that we have a working shared cache in future 2.x releases. Let me know what you think.

          Show
          ctrezzo Chris Trezzo added a comment - Thanks Sangjin Lee ! I think it would be good to backport to branch-2 and hopefully branch-2.8 so that we have a working shared cache in future 2.x releases. Let me know what you think.
          Hide
          ctrezzo Chris Trezzo added a comment -

          branch-2.7 might make sense as well, since this is in the 2.7.x line.

          Show
          ctrezzo Chris Trezzo added a comment - branch-2.7 might make sense as well, since this is in the 2.7.x line.
          Hide
          sjlee0 Sangjin Lee added a comment - - edited

          Ported it to branch-2 as well (2.9.0). Since the shared cache is not fully implemented in previous versions, I don't think we need to backport to those versions.

          Show
          sjlee0 Sangjin Lee added a comment - - edited Ported it to branch-2 as well (2.9.0). Since the shared cache is not fully implemented in previous versions, I don't think we need to backport to those versions.
          Hide
          ctrezzo Chris Trezzo added a comment -

          Thanks Sangjin Lee for the review and commit!

          Show
          ctrezzo Chris Trezzo added a comment - Thanks Sangjin Lee for the review and commit!

            People

            • Assignee:
              ctrezzo Chris Trezzo
              Reporter:
              ctrezzo Chris Trezzo
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development