[SPARK-7751] Add @Since annotation to stable and experimental methods in MLlib

• Type: Umbrella
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: Documentation, MLlib
• Labels: starter
• Assignee: Xiangrui Meng
• Reporter: Xiangrui Meng
• Created: 20/May/15 01:04
• Updated: 21/Jun/16 07:42
• Resolved: 21/Jun/16 07:42

Description

This is useful to check whether a feature exists in some version of Spark. This is an umbrella JIRA to track the progress. We want to have @since tag @Since annotation for both stable (those without any Experimental/DeveloperApi/AlphaComponent annotations) and experimental methods in MLlib:

(Do NOT tag private or package private classes or methods, nor local variables and methods.)

• an example PR for Scala: https://github.com/apache/spark/pull/8309

We need to dig the history of git commit to figure out what was the Spark version when a method was first introduced. Take `NaiveBayes.setModelType` as an example. We can grep `def setModelType` at different version git tags.

meng@xm:~/src/spark
$ git show v1.3.0:mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala | grep "def setModelType"
meng@xm:~/src/spark
$ git show v1.4.0:mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala | grep "def setModelType"
  def setModelType(modelType: String): NaiveBayes = {

If there are better ways, please let us know.

We cannot add all @since tags @Since annotation in a single PR, which is hard to review. So we made some subtasks for each package, for example `org.apache.spark.classification`. Feel free to add more sub-tasks for Python and the `spark.ml` package.

Plan:

1. In 1.5, we try to add @Since annotation to all stable/experimental methods under `spark.mllib`.
2. Starting from 1.6, we require @Since annotation in all new PRs.
3. In 1.6, we try to add @SInce annotation to all stable/experimental methods under `spark.ml`, `pyspark.mllib`, and `pyspark.ml`.

Issue Links

• contains: SPARK-15790 Audit @Since annotations in ML
• is related to: SPARK-8967 Implement @since as an annotation
• relates to: SPARK-10373 Move @since annotator to pyspark to be shared by all components