Details
-
Bug
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
2.1.2, 2.2.1
-
None
Description
History server not showing incomplete/running applications when spark.history.ui.maxApplications property is set to a value that is smaller than the total number of applications.
I believe this is because any applications where completed=false wind up at the end of the list of apps returned by the /applications endpoint and when spark.history.ui.maxApplications is set, that list gets truncated and the running apps are never returned.
The fix I have in mind is to modify the history template to start passing the status parameter when calling the /applications endpoint (status=completed is the default).
I am running Spark in a Mesos environment but I don't think that is relevant to this issue
Attachments
Issue Links
- is duplicated by
-
SPARK-25975 Spark History does not display necessarily the incomplete applications when requested
- Resolved
- links to