[SPARK-17456] Utility for parsing Spark versions

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.0.1, 2.1.0
• Component/s: Spark Core
• Target Version/s: 2.0.1, 2.1.0
• Assignee: Joseph K. Bradley
• Reporter: Joseph K. Bradley
• Created: 08/Sep/16 20:35
• Updated: 09/Sep/16 17:49
• Resolved: 09/Sep/16 12:37

Description

There are many hacks within Spark's codebase to identify and compare Spark versions. We should add a simple utility to standardize these code paths, especially since there have been mistakes made in the past. This will let us add unit tests as well. This initial patch will only add methods for extracting major and minor versions as Int types in Scala.

Issue Links

• blocks: SPARK-17462 Check for places within MLlib which should use VersionUtils to parse Spark version strings
• is required by: SPARK-16240 model loading backward compatibility for ml.clustering.LDA
• links to: [Github] Pull Request #15017 (jkbradley)