Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-7736

Exception not failing Python applications (in yarn cluster mode)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.5.1, 1.6.0
    • Component/s: YARN
    • Labels:
      None
    • Environment:

      Spark 1.3.1, Yarn 2.7.0, Ubuntu 14.04

      Description

      It seems that exceptions thrown in Python spark apps after the SparkContext is instantiated don't cause the application to fail, at least in Yarn: the application is marked as SUCCEEDED.

      Note that any exception right before the SparkContext correctly places the application in FAILED state.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vanzin Marcelo Masiero Vanzin
                Reporter:
                roji Shay Rojansky
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: