Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25022

Add spark.executor.pyspark.memory support to Mesos

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: 2.3.0
    • Fix Version/s: None
    • Component/s: Mesos
    • Labels:

      Description

      SPARK-25004 adds spark.executor.pyspark.memory to control the memory allocation for PySpark and updates YARN to add this memory to its container requests. Mesos should do something similar to account for the python memory allocation.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                rdblue Ryan Blue
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: