Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25697

When zstd compression enabled in progress application is throwing Error in UI

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.2
    • Fix Version/s: 2.4.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      1. In spark-default.conf of Job History enable below parameter
        spark.eventLog.compress=true
        spark.io.compression.codec = org.apache.spark.io.ZStdCompressionCodec
      2.  Restart Job History Services
      3. Submit beeline jobs
      4. Open Yarn Resource Page
      5. Check for the running application in Yarn Resource Page it will list the application.
      6. Open Job History Page 
      7. Go and click Incomplete Application Link and click on the application

      Actual Result:
      UI display "Read error or truncated source" Error

      Expected Result:
      Job History should list the Jobs of the application on clicking the application ID.

        Attachments

          Activity

            People

            • Assignee:
              shahid shahid
              Reporter:
              abhishek.akg ABHISHEK KUMAR GUPTA
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: