Details
-
Umbrella
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
None
Description
This is useful to check whether a feature exists in some version of Spark. This is an umbrella JIRA to track the progress. We want to have @since tag @Since annotation for both stable (those without any Experimental/DeveloperApi/AlphaComponent annotations) and experimental methods in MLlib:
(Do NOT tag private or package private classes or methods, nor local variables and methods.)
- an example PR for Scala: https://github.com/apache/spark/pull/8309
We need to dig the history of git commit to figure out what was the Spark version when a method was first introduced. Take `NaiveBayes.setModelType` as an example. We can grep `def setModelType` at different version git tags.
meng@xm:~/src/spark $ git show v1.3.0:mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala | grep "def setModelType" meng@xm:~/src/spark $ git show v1.4.0:mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala | grep "def setModelType" def setModelType(modelType: String): NaiveBayes = {
If there are better ways, please let us know.
We cannot add all @since tags @Since annotation in a single PR, which is hard to review. So we made some subtasks for each package, for example `org.apache.spark.classification`. Feel free to add more sub-tasks for Python and the `spark.ml` package.
Plan:
1. In 1.5, we try to add @Since annotation to all stable/experimental methods under `spark.mllib`.
2. Starting from 1.6, we require @Since annotation in all new PRs.
3. In 1.6, we try to add @SInce annotation to all stable/experimental methods under `spark.ml`, `pyspark.mllib`, and `pyspark.ml`.
Attachments
Issue Links
- contains
-
SPARK-15790
Audit @Since annotations in ML
-
- Resolved
-
- is related to
-
SPARK-8967 Implement @since as an annotation
-
- Resolved
-
- relates to
-
SPARK-10373 Move @since annotator to pyspark to be shared by all components
-
- Resolved
-
