[SPARK-1351] Documentation Improvements for Spark 1.0

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: Documentation
• Assignee: Matei Alexandru Zaharia
• Reporter: Patrick Wendell
• Created: 29/Mar/14 21:32
• Updated: 02/Mar/15 16:12
• Resolved: 02/Mar/15 16:12

Description

Umbrella to track necessary doc improvements. We can break these out into other JIRA's over time.

• Use grouping in the RDD and SparkContext scaladocs. See Schema RDD:
  http://people.apache.org/~pwendell/catalyst-docs/api/sql/core/index.html#org.apache.spark.sql.SchemaRDD
• Use spark-submit script wherever possible in docs.
• Have package-level documentation in Scaladoc. Also these can be grouped so that the o.a.s package doc looks nice.

Sub-Tasks

• Aggregate Scaladocs across projects: Resolved
• Generate JavaDoc instead of ScalaDoc for Java API: Resolved
• Update Java programming guide to explain Java 8 syntax: Resolved
• Add Naive Bayes to MLlib documentation: Resolved
• [streaming] Add deployment subsection to streaming: Resolved
• [streaming] Add 0.9 to 1.0 migration guide for streaming receiver: Resolved
• Update doc overview page to not mention building if you get a pre-built distro: Resolved
• [streaming] Update custom receiver guide with new Receiver API: Resolved
• Exclude internal catalyst classes from scaladoc, or make them package private: Resolved
• Add package-info.java and package.scala files for all packages that appear in docs: Resolved
• Consolidate the Spark Programming Guide with tabs for all languages: Resolved
• Add language tabs to quick start guide: Resolved
• Splash page should include correct syntax for launching examples: Resolved
• Update "third-party Hadoop distros" doc to list more distros: Resolved
• Add full Java examples in MLlib docs: Resolved