Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4313

"Thread Dump" link is broken in yarn-cluster mode

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.2.0
    • Fix Version/s: 1.2.0
    • Component/s: Web UI, YARN
    • Labels:
      None
    • Target Version/s:

      Description

      In yarn-cluster mode, the Web UI is running behind a yarn proxy server. Some features(or bugs?) of yarn proxy server will break the links for thread dump.

      1. Yarn proxy server will do http redirect internally, so if opening "http://example.com:8088/cluster/app/application_1415344371838_0012/executors", it will fetch "http://example.com:8088/cluster/app/application_1415344371838_0012/executors/" and return the content but won't change the link in the browser. Then when a user clicks "Thread Dump", it will jump to "http://example.com:8088/proxy/application_1415344371838_0012/threadDump/?executorId=2". This is a wrong link. The correct link should be "http://example.com:8088/proxy/application_1415344371838_0012/executors/threadDump/?executorId=2".

      2. Yarn proxy server has a bug about the URL encode/decode. When a user accesses "http://example.com:8088/proxy/application_1415344371838_0006/executors/threadDump/?executorId=%3Cdriver%3E", the yarn proxy server will require "http://example.com:36429/executors/threadDump/?executorId=%25253Cdriver%25253E". But Spark web server expects "http://example.com:36429/executors/threadDump/?executorId=%3Cdriver%3E". I will report this issue to Hadoop community later.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                zsxwing Shixiong Zhu
                Reporter:
                zsxwing Shixiong Zhu
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: