Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3368 [Umbrella] YARN web UI: Next generation
  3. YARN-5000

[YARN-3368] App attempt page is not loading when timeline server is not started

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None

      Description

      If timeline server is not started, app attempt page is not getting loaded.

      In new web-ui, yarnContainer route is tightly coupled with both RM and Timeline server. And if one of server is not up, page will not load. If timeline server is not up, container information from RM is to be displayed.

      1. YARN-5000-YARN-3368.5.patch
        16 kB
        Sunil G
      2. YARN-5000-YARN-3368.4.patch
        17 kB
        Sunil G
      3. YARN-5000-YARN-3368.3.patch
        17 kB
        Sunil G
      4. YARN-5000-YARN-3368.2.patch
        13 kB
        Sunil G
      5. YARN-5000-YARN-3368.1.patch
        13 kB
        Sunil G
      6. screenshot-1.png
        139 kB
        Wangda Tan
      7. AppRunningAndNoTimelineServer.png
        72 kB
        Sunil G
      8. AppRunningAndNoTimelineServer_v2.png
        73 kB
        Sunil G
      9. AppFinishedAndNoTimelineServer.png
        42 kB
        Sunil G
      10. 0001-YARN-5000.patch
        7 kB
        Sunil G

        Activity

        Hide
        sunilg Sunil G added a comment -

        Attaching an initial patch.
        While testing, I found some more issues in app attempt page. Few fields are not getting displayed and End Time is always shown 1970.
        I will fix that also along with this in next version of patch.

        Show
        sunilg Sunil G added a comment - Attaching an initial patch. While testing, I found some more issues in app attempt page. Few fields are not getting displayed and End Time is always shown 1970. I will fix that also along with this in next version of patch.
        Hide
        sunilg Sunil G added a comment -

        Attaching a clean version of the patch by fixing below mentioned issues:

        • If timeline server was not running, yarn app-attempt page was not loading. Now page will load and will show empty set of containers.
        • In app-attempt page, many fields were not getting displayed. This is due to 2 version of app-attempt dao objects from RM end. This scenario is now handled with ember property.
        • If there were no applications were running, apps page were not displaying anything. Now some useful messages were added.

        Wangda Tan and Varun Saxena, pls help to check the same. I will also attach screen shots.

        Show
        sunilg Sunil G added a comment - Attaching a clean version of the patch by fixing below mentioned issues: If timeline server was not running, yarn app-attempt page was not loading. Now page will load and will show empty set of containers. In app-attempt page, many fields were not getting displayed. This is due to 2 version of app-attempt dao objects from RM end. This scenario is now handled with ember property. If there were no applications were running, apps page were not displaying anything. Now some useful messages were added. Wangda Tan and Varun Saxena , pls help to check the same. I will also attach screen shots.
        Hide
        sunilg Sunil G added a comment -

        Updating patch after some more cleanup.

        Show
        sunilg Sunil G added a comment - Updating patch after some more cleanup.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 1m 34s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 whitespace 0m 0s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix.
        -1 whitespace 0m 0s The patch has 33 line(s) with tabs.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        2m 8s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:f38692c
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801465/YARN-5000-YARN-3368.2.patch
        JIRA Issue YARN-5000
        Optional Tests asflicense
        uname Linux 3f2bfbd025ef 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision YARN-3368 / 2d617a5
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/11283/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/11283/artifact/patchprocess/whitespace-tabs.txt
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11283/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 1m 34s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 whitespace 0m 0s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 whitespace 0m 0s The patch has 33 line(s) with tabs. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 2m 8s Subsystem Report/Notes Docker Image:yetus/hadoop:f38692c JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801465/YARN-5000-YARN-3368.2.patch JIRA Issue YARN-5000 Optional Tests asflicense uname Linux 3f2bfbd025ef 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-3368 / 2d617a5 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/11283/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/11283/artifact/patchprocess/whitespace-tabs.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui Console output https://builds.apache.org/job/PreCommit-YARN-Build/11283/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Thanks Sunil G for working on this fix, several comments while looking at screenshots.
        1) Instead of showing "containers are not found", we don't have to show containers table.
        2) Can we show application attempt status (running/finished/failed, etc.) and elapsed time on the app-attempt page?

        Thoughts?

        Show
        leftnoteasy Wangda Tan added a comment - Thanks Sunil G for working on this fix, several comments while looking at screenshots. 1) Instead of showing "containers are not found", we don't have to show containers table. 2) Can we show application attempt status (running/finished/failed, etc.) and elapsed time on the app-attempt page? Thoughts?
        Hide
        sunilg Sunil G added a comment -

        Yes Wangda Tan
        We could do these two changes. I will update a patch shortwhile.

        Show
        sunilg Sunil G added a comment - Yes Wangda Tan We could do these two changes. I will update a patch shortwhile.
        Hide
        sunilg Sunil G added a comment -

        Thanks Wangda Tan.
        Uploading a new patch addressing the comments.

        Show
        sunilg Sunil G added a comment - Thanks Wangda Tan . Uploading a new patch addressing the comments.
        Hide
        sunilg Sunil G added a comment -

        Also attached a v2 screen shot.

        Show
        sunilg Sunil G added a comment - Also attached a v2 screen shot.
        Hide
        sunilg Sunil G added a comment -

        Uploading a cleaner patch after changing LICENSE.txt and some error messages. Wangda Tan, kindly help to check the same.

        Show
        sunilg Sunil G added a comment - Uploading a cleaner patch after changing LICENSE.txt and some error messages. Wangda Tan , kindly help to check the same.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 1m 37s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        0 mvndep 4m 23s Maven dependency ordering for branch
        0 mvndep 0m 15s Maven dependency ordering for patch
        -1 whitespace 0m 0s The patch has 19 line(s) with tabs.
        +1 asflicense 0m 35s Patch does not generate ASF License warnings.
        7m 8s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:f38692c
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802435/YARN-5000-YARN-3368.4.patch
        JIRA Issue YARN-5000
        Optional Tests asflicense
        uname Linux 7750c6d0691d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision YARN-3368 / 2d617a5
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/11344/artifact/patchprocess/whitespace-tabs.txt
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui . U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11344/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 1m 37s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. 0 mvndep 4m 23s Maven dependency ordering for branch 0 mvndep 0m 15s Maven dependency ordering for patch -1 whitespace 0m 0s The patch has 19 line(s) with tabs. +1 asflicense 0m 35s Patch does not generate ASF License warnings. 7m 8s Subsystem Report/Notes Docker Image:yetus/hadoop:f38692c JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802435/YARN-5000-YARN-3368.4.patch JIRA Issue YARN-5000 Optional Tests asflicense uname Linux 7750c6d0691d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-3368 / 2d617a5 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/11344/artifact/patchprocess/whitespace-tabs.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui . U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/11344/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Sunil G,

        Tested the patch, haven't done deep tests, some comments so far

        1) When I try the patch, run RM without TS/NM. The application is in ACCEPTED state, but the application page has multiple entries of attempt: (see attachment), (This only happens when I access RM hosted page at localhost:8288, probably caused by auto/hash issue which is mentioned by you at: https://issues.apache.org/jira/browse/YARN-4515?focusedCommentId=15269292&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15269292).

        2) When an application is running (without enable TS, but NM is running). I get following intermittent exception when keep refreshing page at: http://localhost:4200/yarnAppAttempt/appattempt_1462494543748_0004_000001 (replace by your attempt ID

        http://localhost:1337/localhost:8088/ws/v1/cluster/apps/application_1462494543748_0004/appattempts/appattempt_1462494543748_0004_000001/containers
        yarn-container.js:31 http://localhost:1337/localhost:8188/ws/v1/applicationhistory/apps/applicat…494543748_0004/appattempts/appattempt_1462494543748_0004_000001/containers
        ember.debug.js:30877 TypeError: Cannot read property '0' of undefined
            at Class.error (application.js:11)
            at Object.triggerEvent (ember.debug.js:27476)
            at Object.trigger (ember.debug.js:51925)
            at Object.trigger (ember.debug.js:51739)
            at ember.debug.js:51559
            at tryCatch (ember.debug.js:52258)
            at invokeCallback (ember.debug.js:52273)
            at publish (ember.debug.js:52241)
            at publishRejection (ember.debug.js:52176)
            at ember.debug.js:30835onerrorDefault @ ember.debug.js:30877trigger @ ember.debug.js:52928(anonymous function) @ ember.debug.js:54177invoke @ ember.debug.js:320flush @ ember.debug.js:384flush @ ember.debug.js:185end @ ember.debug.js:563run @ ember.debug.js:685run @ ember.debug.js:20105hash.success @ rest-adapter.js:742fire @ jquery.js:3099fireWith @ jquery.js:3211done @ jquery.js:8264(anonymous function) @ jquery.js:8605
        jquery.js:8630 GET http://localhost:1337/localhost:8188/ws/v1/applicationhistory/apps/applicat…494543748_0004/appattempts/appattempt_1462494543748_0004_000001/containers 502 (Bad Gateway)
        

        3) (MINOR), I can get this error at console when TS is not enabled:
        jquery.js:8630 GET http://localhost:1337/localhost:8188/ws/v1/applicationhistory/apps/applicat…494543748_0004/appattempts/appattempt_1462494543748_0004_000001/containers 502 (Bad Gateway), probably we should not show such message

        For code implementation, some comments so far:
        1) Instead of returning object with id="dummy" and check "dummy" everywhere, is it possible to return an empty array? Which may be more clear.

        Show
        leftnoteasy Wangda Tan added a comment - Sunil G , Tested the patch, haven't done deep tests, some comments so far 1) When I try the patch, run RM without TS/NM. The application is in ACCEPTED state, but the application page has multiple entries of attempt: (see attachment ), (This only happens when I access RM hosted page at localhost:8288, probably caused by auto/hash issue which is mentioned by you at: https://issues.apache.org/jira/browse/YARN-4515?focusedCommentId=15269292&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15269292 ). 2) When an application is running (without enable TS, but NM is running). I get following intermittent exception when keep refreshing page at: http://localhost:4200/yarnAppAttempt/appattempt_1462494543748_0004_000001 (replace by your attempt ID http: //localhost:1337/localhost:8088/ws/v1/cluster/apps/application_1462494543748_0004/appattempts/appattempt_1462494543748_0004_000001/containers yarn-container.js:31 http: //localhost:1337/localhost:8188/ws/v1/applicationhistory/apps/applicat…494543748_0004/appattempts/appattempt_1462494543748_0004_000001/containers ember.debug.js:30877 TypeError: Cannot read property '0' of undefined at Class .error (application.js:11) at Object .triggerEvent (ember.debug.js:27476) at Object .trigger (ember.debug.js:51925) at Object .trigger (ember.debug.js:51739) at ember.debug.js:51559 at tryCatch (ember.debug.js:52258) at invokeCallback (ember.debug.js:52273) at publish (ember.debug.js:52241) at publishRejection (ember.debug.js:52176) at ember.debug.js:30835onerrorDefault @ ember.debug.js:30877trigger @ ember.debug.js:52928(anonymous function) @ ember.debug.js:54177invoke @ ember.debug.js:320flush @ ember.debug.js:384flush @ ember.debug.js:185end @ ember.debug.js:563run @ ember.debug.js:685run @ ember.debug.js:20105hash.success @ rest -adapter.js:742fire @ jquery.js:3099fireWith @ jquery.js:3211done @ jquery.js:8264(anonymous function) @ jquery.js:8605 jquery.js:8630 GET http: //localhost:1337/localhost:8188/ws/v1/applicationhistory/apps/applicat…494543748_0004/appattempts/appattempt_1462494543748_0004_000001/containers 502 (Bad Gateway) 3) (MINOR), I can get this error at console when TS is not enabled: jquery.js:8630 GET http://localhost:1337/localhost:8188/ws/v1/applicationhistory/apps/applicat …494543748_0004/appattempts/appattempt_1462494543748_0004_000001/containers 502 (Bad Gateway) , probably we should not show such message For code implementation, some comments so far: 1) Instead of returning object with id="dummy" and check "dummy" everywhere, is it possible to return an empty array? Which may be more clear.
        Hide
        sunilg Sunil G added a comment -

        Thank you Wangda Tan for the detailed comments.

        Issue mentioned in point 1) and 2) are mostly observed when cached data is used. With YARN-4514's locationType change (to hash), this issue will not happen.

        (MINOR), I can get this error at console when TS is not enabled:

        For Issue 3), its not been logged by us. Since timeline server is not running, ember adapter query is failing and its been logged as 502. Since we are catching that exception, we can continue and load the page now.

        For code implementation, .... Instead of returning object with..

        Yes. This is fixed. I have made use of empty array now. Also tested various cases.

        Show
        sunilg Sunil G added a comment - Thank you Wangda Tan for the detailed comments. Issue mentioned in point 1) and 2) are mostly observed when cached data is used. With YARN-4514 's locationType change (to hash), this issue will not happen. (MINOR), I can get this error at console when TS is not enabled: For Issue 3), its not been logged by us. Since timeline server is not running, ember adapter query is failing and its been logged as 502. Since we are catching that exception, we can continue and load the page now. For code implementation, .... Instead of returning object with.. Yes. This is fixed. I have made use of empty array now. Also tested various cases.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 1m 24s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        0 mvndep 2m 14s Maven dependency ordering for branch
        0 mvndep 0m 15s Maven dependency ordering for patch
        -1 whitespace 0m 0s The patch has 5 line(s) with tabs.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        4m 27s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:f38692c
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802882/YARN-5000-YARN-3368.5.patch
        JIRA Issue YARN-5000
        Optional Tests asflicense
        uname Linux a244d31526d8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision YARN-3368 / 2d617a5
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/11378/artifact/patchprocess/whitespace-tabs.txt
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui . U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11378/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 1m 24s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. 0 mvndep 2m 14s Maven dependency ordering for branch 0 mvndep 0m 15s Maven dependency ordering for patch -1 whitespace 0m 0s The patch has 5 line(s) with tabs. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 4m 27s Subsystem Report/Notes Docker Image:yetus/hadoop:f38692c JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802882/YARN-5000-YARN-3368.5.patch JIRA Issue YARN-5000 Optional Tests asflicense uname Linux a244d31526d8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-3368 / 2d617a5 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/11378/artifact/patchprocess/whitespace-tabs.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui . U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/11378/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Thanks,

        With YARN-4514's locationType change (to hash), this issue will not happen.

        Is this a typo? Did you mean YARN-4515?

        Show
        leftnoteasy Wangda Tan added a comment - Thanks, With YARN-4514 's locationType change (to hash), this issue will not happen. Is this a typo? Did you mean YARN-4515 ?
        Hide
        sunilg Sunil G added a comment -

        Sorry, yes. I meant YARN-4515.

        Show
        sunilg Sunil G added a comment - Sorry, yes. I meant YARN-4515 .
        Hide
        leftnoteasy Wangda Tan added a comment -

        A couple of comments about implementation:

        1) Instead of following logic:

        75      if (url == undefined) {
        76        url = "Not Available";
        77      }
        78      return url != "Not Available";
        

        Can we just return url != undefined ?

        2) yarn-app.js is still using dummy, could we change it to empty array

        3) A coupld of tabs found in your patch, could you double check it?

        Will try this patch again after YARN-4515 get in

        Show
        leftnoteasy Wangda Tan added a comment - A couple of comments about implementation: 1) Instead of following logic: 75 if (url == undefined) { 76 url = "Not Available" ; 77 } 78 return url != "Not Available" ; Can we just return url != undefined ? 2) yarn-app.js is still using dummy, could we change it to empty array 3) A coupld of tabs found in your patch, could you double check it? Will try this patch again after YARN-4515 get in
        Hide
        sunilg Sunil G added a comment -

        Thank you Wangda Tan for the comments. I ll update a patch here with these comments fixed. Now I am making some changes in pom.xml to copy the UI o/p folder under yarn/webapps. Will share a patch soon.

        Show
        sunilg Sunil G added a comment - Thank you Wangda Tan for the comments. I ll update a patch here with these comments fixed. Now I am making some changes in pom.xml to copy the UI o/p folder under yarn/webapps. Will share a patch soon.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Committed with YARN-4515. (72151c24452cfef250f7ce6f457dec6ada2cbd26). Thanks Sunil G!

        Show
        leftnoteasy Wangda Tan added a comment - Committed with YARN-4515 . (72151c24452cfef250f7ce6f457dec6ada2cbd26). Thanks Sunil G !

          People

          • Assignee:
            sunilg Sunil G
            Reporter:
            sunilg Sunil G
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development