Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Incomplete
-
2.4.0
-
None
Description
Spark submit doesn't allow Xmx anywhere in the spark.driver.extraJavaOptions
For eg:
bin\spark-submit --class org.apache.spark.examples.SparkPi --master local[*] --conf "spark.driver.extraJavaOptions=-DmyKey=MyValueContainsXmx" examples\jars\spark-examples_2.11-2.4.4.jar Error: Not allowed to specify max heap(Xmx) memory settings through java options (was -DmyKey=MyValueContainsXmx). Use the corresponding --driver-memory or spark.driver.memory configuration instead.
Can we update the above condition to check more specific for eg -Xmx