Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33611

Decode Query parameters of the redirect URL for reverse proxy

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0, 3.1.0
    • 3.0.2, 3.1.0
    • Web UI
    • None

    Description

      When running Spark with reverse proxy enabled, the query parameter of the request URL can be encoded twice: one from the browser and another one from the reverse proxy(e.g. Nginx).

      In Spark's stage page, the URL of "/taskTable" contains query parameter order[0][dir]. After encoding twice, the query parameter becomes `order%255B0%255D%255Bdir%255D` and it will be decoded as `order%5B0%5D%5Bdir%5D` instead of `order[0][dir]`. As a result, there will be NullPointerException from https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/status/api/v1/StagesResource.scala#L176

      Other than that, the other parameter may not work as expected after encoded twice.

      We should decode the query parameters and fix the problem

      Attachments

        Activity

          People

            Gengliang.Wang Gengliang Wang
            Gengliang.Wang Gengliang Wang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: