Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-34478

Ignore or reject wrong config when start sparksession

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: In Progress
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.2.0
    • Fix Version/s: None
    • Component/s: Spark Core, SQL
    • Labels:
      None

      Description

      When use 

      SparkSession.builder().config()
      

      In this method user may config `spark.driver.memory`. But when we run this code, jvm is started,  so this configuration won't work and in Spark UI, it will show as this configuration. 

      So we should ignore such as wrong way.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              angerszhu angerszhu

              Dates

              • Created:
                Updated:

                Issue deployment