[SPARK-4628] Put external projects and examples behind a build flag

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Auto Closed
• Component/s: Examples
• Labels: bulk-closed
• Assignee: Josh Rosen
• Reporter: Patrick Wendell
• Created: 26/Nov/14 19:17
• Updated: 06/Jun/19 13:57
• Resolved: 06/Jun/19 13:57

Description

This is something we talked about doing for convenience, but I'm escalating this based on realizing today that some of our external projects depend on code that is not in maven central. I.e. if one of these dependencies is taken down (as happened recently with mqtt), all Spark builds will fail.

The proposal here is simple, have a profile -Pexternal-projects that enables these. This can follow the exact pattern of -Pkinesis-asl which was disabled by default due to a license issue.

Issue Links

• is duplicated by: SPARK-4376 Put external modules behind build profiles
• is related to: SPARK-4631 Add real unit test for MQTT
• links to: [Github] Pull Request #3485 (pwendell)
• links to: [Github] Pull Request #10659 (JoshRosen)
• links to: [Github] Pull Request #10688 (sarutak)