[SPARK-25697] When zstd compression enabled in progress application is throwing Error in UI

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.2
• Fix Version/s: 2.4.0
• Component/s: Spark Core
• Assignee: shahid
• Reporter: ABHISHEK KUMAR GUPTA
• Created: 10/Oct/18 07:11
• Updated: 29/Oct/18 00:57
• Resolved: 12/Oct/18 18:00

Description

1. In spark-default.conf of Job History enable below parameter
   spark.eventLog.compress=true
   spark.io.compression.codec = org.apache.spark.io.ZStdCompressionCodec
2.  Restart Job History Services
3. Submit beeline jobs
4. Open Yarn Resource Page
5. Check for the running application in Yarn Resource Page it will list the application.
6. Open Job History Page 
7. Go and click Incomplete Application Link and click on the application

Actual Result:
UI display "Read error or truncated source" Error

Expected Result:
Job History should list the Jobs of the application on clicking the application ID.

Issue Links

• links to: [Github] Pull Request #22689 (shahidki31)
• links to: [Github] Pull Request #22689 (shahidki31)