Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-44050

Unable to Mount ConfigMap in Driver Pod - ConfigMap Creation Issue



    • Bug
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • 3.3.1
    • None
    • Kubernetes, Spark Submit
    • None


      Dear Spark community,

      I am facing an issue related to mounting a ConfigMap in the driver pod of my Spark application. Upon investigation, I realized that the problem is caused by the ConfigMap not being created successfully.

      Problem Description:
      When attempting to mount the ConfigMap in the driver pod, I encounter consistent failures and my pod stays in containerCreating state. Upon further investigation, I discovered that the ConfigMap does not exist in the Kubernetes cluster, which results in the driver pod's inability to access the required configuration data.

      Additional Information:

      I would like to highlight that this issue is not a frequent occurrence. It has been observed randomly, affecting the mounting of the ConfigMap in the driver pod only approximately 5% of the time. This intermittent behavior adds complexity to the troubleshooting process, as it is challenging to reproduce consistently.

      Error Message:

      when describing driver pod (kubectl describe pod pod_name)  get the below error.

      "ConfigMap '<configmap-name>' not found."

      To Reproduce:

      1. Download spark 3.3.1 from https://spark.apache.org/downloads.html

      2. create an image with "bin/docker-image-tool.sh"

      3. Submit on spark-client via bash command by passing all the details and configurations.

      4. Randomly in some of the driver pod we can observe this issue.




        1. image-2023-06-14-11-07-36-960.png
          478 kB
          Harshwardhan Singh Dodiya



            Unassigned Unassigned
            harshwardhan.dodiya Harshwardhan Singh Dodiya
            0 Vote for this issue
            3 Start watching this issue