[SPARK-16606] Misleading warning for SparkContext.getOrCreate "WARN SparkContext: Use an existing SparkContext, some configuration may not take effect."

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.1.0
• Fix Version/s: 2.1.0
• Component/s: Spark Core
• Assignee: Sean R. Owen
• Reporter: Jacek Laskowski
• Created: 18/Jul/16 12:42
• Updated: 21/Oct/16 14:33
• Resolved: 09/Aug/16 08:38

Description

SparkContext.getOrCreate should really be checking whether the code gets the already-created instance or creating a new one.

Just a nit-pick: the warning message should also be "Using..." not "Use"

scala> sc.version
res2: String = 2.1.0-SNAPSHOT

scala> sc
res3: org.apache.spark.SparkContext = org.apache.spark.SparkContext@1186374c

scala> SparkContext.getOrCreate
16/07/18 14:40:31 WARN SparkContext: Use an existing SparkContext, some configuration may not take effect.
res4: org.apache.spark.SparkContext = org.apache.spark.SparkContext@1186374c

Issue Links

• links to: [Github] Pull Request #14533 (srowen)
• links to: [Github] Pull Request #15586 (srowen)