Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-2221

Spark jobs UI from the paragraph is broken in YARN

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.0, 0.8.0
    • Fix Version/s: 0.8.0, 0.9.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Zeppelin from 3/1 master snapshot

      Description

      Discussed on PR-1663 for ZEPPELIN-1692
      https://github.com/apache/zeppelin/pull/1663#issuecomment-283477396

      When we click on any of Spark jobs UI from the paragraph links,
      link is leading to

      http://hostname.domain.com:8088/proxy/application_1488384993892_0001/jobs/job/
      
      and that error shows HTTP ERROR 400
      
      Page reads
      
      Problem accessing /jobs/job/.
      
      Reason: requirement failed: Missing id parameter
      Powered by Jetty://
      

      A little bit more information - the link on paragprah leads to

      http://10.20.32.57:28009/jobs/job?id=123
      
       Spark Driver UI.
      Spark Driver web server then redirects to a link like I sent in my previous post,
      and it misses job id in the redirected URL, i.e.
      
      http://host.domain.com:8088/proxy/application_1488384993892_0044/jobs/job/
      To your question on consistency - yes it happens in 100% cases, I never saw this new feature works for us.
      Is this Spark version dependent or something? We're running CDH Spark 2.
      
      

      According to https://github.com/apache/spark/pull/5947 URL format is different in YARN and non-YARN modes? Was PR-1663 for ZEPPELIN-1692 tested on both of these modes? Not sure what else might break those links.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jongyoul Jongyoul Lee
                Reporter:
                Tagar Ruslan Dautkhanov
              • Votes:
                1 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: