Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-11058

failed spark job reports on YARN as successful

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: 1.3.0
    • Fix Version/s: None
    • Component/s: YARN
    • Labels:
      None
    • Environment:

      CDH 5.4

      Description

      I have a spark batch job running on CDH5.4 + Spark 1.3.0. Job is submitted in “yarn-client” mode. The job itself failed due to YARN kills several executor containers because the containers exceeded the memory limit posed by YARN. However, when I went to the YARN resource manager site, it displayed the job as successful. I found there was an issue reported in JIRA https://issues.apache.org/jira/browse/SPARK-3627, but it says it was fixed in Spark 1.2. On Spark history server, it shows the job as “Incomplete”.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              lanjiang Lan Jiang
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: