[SPARK-13920] MIMA checks should apply to @Experimental and @DeveloperAPI APIs

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: Project Infra
• Target Version/s: 2.0.0
• Assignee: Dongjoon Hyun
• Reporter: Josh Rosen
• Created: 16/Mar/16 00:34
• Updated: 16/Mar/16 06:25
• Resolved: 16/Mar/16 06:25

Description

Our MIMA binary compatibility checks currently ignore APIs which are marked as @Experimental or @DeveloperApi, but I don't think this makes sense. Even if those annotations reserve the right to break binary compatibility, we should still avoid compatibility breaks whenever possible and should be informed explicitly when compatibility breaks.

As a result, we should update GenerateMIMAIgnore to stop ignoring classes and methods which have those annotations. To remove the ignores, remove the checks from https://github.com/apache/spark/blob/643649dcbfabc5d6952c2ecfb98286324c887665/tools/src/main/scala/org/apache/spark/tools/GenerateMIMAIgnore.scala#L43

After removing the ignores, update project/MimaExcludes.scala to add exclusions for binary compatibility breaks introduced in 2.0.

Issue Links

• links to: [Github] Pull Request #11751 (dongjoon-hyun)