When an AM web request is redirected to the RM the query parameters are dropped from the web request.
This happens for Spark as described in
The repro steps are:
- Start up the spark-shell in yarn mode and run a job
- Try to access the job details through http://<fqdn>:4040/jobs/job?id=0
- A HTTP ERROR 400 is thrown (requirement failed: missing id parameter)
This works fine in local or standalone mode, but does not work on Yarn where the query parameter is dropped. If the UI filter org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter is removed from the config which shows that the problem is in the filter