[SPARK-27774] Avoid hardcoded configs

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.4.3
• Fix Version/s: 3.0.0
• Component/s: MLlib, Spark Core
• Assignee: wenxuanguan
• Reporter: wenxuanguan
• Created: 20/May/19 06:18
• Updated: 12/Dec/22 18:10
• Resolved: 22/May/19 01:46

Description

I found some hardcoded configs as below:

spark.executor.heartbeatInterval

spark.jars

spark.files

spark.kryoserializer.buffer.max

spark.kryoserializer.buffer

spark.shuffle.file.buffer

spark.submit.deployMode

spark.driver.supervise

spark.driver.cores

spark.cores.max

which should be replaced by declared variable, such as SHUFFLE_FILE_BUFFER_SIZE

 

Issue Links

• links to: [Github] Pull Request #24631 (wenxuanguan)
• links to: GitHub Pull Request #24631
• links to: Pull request #24631