Starting from Spark 2.3, the SHS REST API endpoint /applications/<app_id>/jobs/ is not including description in the JobData returned. This is not the case until Spark 2.2.
Steps to reproduce:
- Open spark-shell
- Access end REST API http://SHS-host:port/api/v1/applications/<app-id>/jobs/
- REST API of Spark 2.3 and above will not return description