Uploaded image for project: 'Apache DataLab'
  1. Apache DataLab
  2. DATALAB-308

[Local spark]: Spark.executor.memory should be allocated depending on notebook instance shape

    XMLWordPrintableJSON

    Details

      Description

       This allocation is obsolete because of task https://issues.apache.org/jira/browse/DATALAB-1985

      Preconditions:
      Notebook is created

      Steps to reproduce:
      1. run playbok on local kernel

      Actual result:
      1. the value of spark.executor.memory is less than 75% from total (if instance shape is up to 8 GB)
      2. the value of spark.executor.memory is less than formula value: total value minus 3500MB (if instance shape is over than 8 GB)

      Expected result:
      1. the value of spark.executor.memory is equal 75% from total (if instance shape is up to 8 GB)
      2. the value of spark.executor.memory is equal formula value: total value minus 3500MB (if instance shape is over than 8 GB)

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              vira_vitanska@epam.com Vira Vitanska
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: