[SPARK-16294] Labelling support for the include_example Jekyll plugin

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Component/s: Documentation
• Target Version/s: 2.0.0, 2.0.1, 2.1.0
• Assignee: Cheng Lian
• Reporter: Cheng Lian
• Created: 29/Jun/16 10:58
• Updated: 30/Jun/16 05:51
• Resolved: 30/Jun/16 05:51

Description

Part of the Spark programming guide pages are using the include_example Jekyll plugin to extract line blocks surrounded by example on and example off tag pairs (usually written as comments) from source files under the examples sub-project.

One limitation of the include_example plugin is that all line blocks within a single file must be included in a single example snippet block in the final HTML file. It would be nice to add labelling support to this plugin, so that we mark different line blocks with different labels in the source file:

// $example on:init_session$
val spark = SparkSession
  .builder()
  .appName("Spark Examples")
  .config("spark.some.config.option", "some-value")
  .getOrCreate()

// For implicit conversions like RDDs to DataFrames
import spark.implicits._
// $example off:init_session$

// $example on:create_df$
val df = spark.read.json("examples/src/main/resources/people.json")

// Displays the content of the DataFrame to stdout
df.show()
// age  name
// null Michael
// 30   Andy
// 19   Justin
// $example off:create_df$

and then, by referring different labels in the Liquid template, like this:

{% include_example init_session scala/org/apache/spark/examples/sql/SparkSessionExample.scala %}

{% include_example create_df scala/org/apache/spark/examples/sql/SparkSessionExample.scala %}

we may generate multiple example snippet blocks in the final HTML page from a single source file.

Issue Links

• is related to: SPARK-11399 Include_example should support labels to cut out different parts in one example code
• links to: [Github] Pull Request #13972 (liancheng)