Dear Spark community,
I am facing an issue related to mounting a ConfigMap in the driver pod of my Spark application. Upon investigation, I realized that the problem is caused by the ConfigMap not being created successfully.
When attempting to mount the ConfigMap in the driver pod, I encounter consistent failures and my pod stays in containerCreating state. Upon further investigation, I discovered that the ConfigMap does not exist in the Kubernetes cluster, which results in the driver pod's inability to access the required configuration data.
I would like to highlight that this issue is not a frequent occurrence. It has been observed randomly, affecting the mounting of the ConfigMap in the driver pod only approximately 5% of the time. This intermittent behavior adds complexity to the troubleshooting process, as it is challenging to reproduce consistently.
when describing driver pod (kubectl describe pod pod_name) get the below error.
"ConfigMap '<configmap-name>' not found."
1. Download spark 3.3.1 from https://spark.apache.org/downloads.html
2. create an image with "bin/docker-image-tool.sh"
3. Submit on spark-client via bash command by passing all the details and configurations.
4. Randomly in some of the driver pod we can observe this issue.