[SPARK-11605] ML 1.6 QA: API: Java compatibility, docs

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0, 2.0.0
• Component/s: Documentation, Java API, ML, MLlib
• Target Version/s: 1.6.0
• Assignee: yuhao yang
• Reporter: Joseph K. Bradley
• Shepherd: Joseph K. Bradley
• Created: 09/Nov/15 20:42
• Updated: 06/May/16 08:32
• Resolved: 08/Dec/15 19:47

Description

Check Java compatibility for MLlib for this release.

Checking compatibility means:

• comparing with the Scala doc
• verifying that Java docs are not messed up by Scala type incompatibilities. Some items to look out for are:
  • Check for generic "Object" types where Java cannot understand complex Scala types.
    • Note: The Java docs do not always match the bytecode. If you find a problem, please verify it using javap.
  • Check Scala objects (especially with nesting!) carefully.
  • Check for uses of Scala and Java enumerations, which can show up oddly in the other language's doc.
• If needed for complex issues, create small Java unit tests which execute each method. (The correctness can be checked in Scala.)

If you find issues, please comment here, or for larger items, create separate JIRAs and link here.
Note that we should not break APIs from previous releases. So if you find a problem, check if it was introduced in this Spark release (in which case we can fix it) or in a previous one (in which case we can create a java-friendly version of the API).

Issue Links

• is blocked by: SPARK-12000 `sbt publishLocal` hits a Scala compiler bug caused by `Since` annotation
• relates to: SPARK-14814 ML 2.0 QA: API: Java compatibility, docs
• links to: [Github] Pull Request #10102 (hhbyyh)