[SPARK-12160] spark.mllib should use SQLContext.getOrCreate

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.1, 1.5.2, 1.6.0
• Fix Version/s: 1.5.3, 1.6.0
• Component/s: MLlib
• Target Version/s: 1.5.3, 1.6.1
• Assignee: Joseph K. Bradley
• Reporter: Joseph K. Bradley
• Created: 05/Dec/15 23:53
• Updated: 24/Dec/15 13:26
• Resolved: 08/Dec/15 07:37

Description

spark.mllib currently uses new SQLContext(...) in several places, especially the model save/load methods. It should use SQLContext.getOrCreate, which was introduced in Spark 1.4.

This can cause some problems, depending on how the user/system manages contexts.

Issue Links

• relates to: SPARK-8870 Use SQLContext.getOrCreate in model save/load
• links to: [Github] Pull Request #10161 (jkbradley)
• links to: [Github] Pull Request #10183 (jkbradley)