[SPARK-10383] Sync example code between API doc and user guide

• Type: Brainstorming
• Status: Resolved
• Priority: Major
• Resolution: Won't Do
• Component/s: Documentation, ML, MLlib
• Assignee: Xiangrui Meng
• Reporter: Xiangrui Meng
• Created: 01/Sep/15 00:28
• Updated: 05/May/18 03:17
• Resolved: 05/May/18 03:17

Description

It would be nice to provide example code in both user guide and API docs. However, it would become hard to keep the content in-sync. This JIRA is to collect approaches/processes to make it feasible.

This is related to SPARK-10382, where we discuss how to move example code from user guide markdown to `spark/examples/`. After that, we can look for solutions that can pick up example code from `spark/examples` and make them available in the API doc. Though I don't know any feasible solution right now, those are some relevant projects:

• https://github.com/tkawachi/sbt-doctest
• http://www.doctester.org/

It would be nice to hear more ideas.

Issue Links

• is related to: SPARK-12208 Abstract the examples into a common place
• is related to: SPARK-7924 Consolidate example code in MLlib
• relates to: SPARK-10382 Implement a Jekyll tag to include example code in user guide