[SPARK-18267] Distribute PySpark via Python Package Index (pypi)

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.2.0
• Component/s: Build, Project Infra, PySpark
• Assignee: Holden Karau
• Reporter: Reynold Xin
• Created: 04/Nov/16 07:48
• Updated: 12/Sep/17 20:03
• Resolved: 12/Sep/17 20:03

Description

The goal is to distribute PySpark via pypi, so users can simply run Spark on a single node via "pip install pyspark" (or "pip install apache-spark").

Sub-Tasks

• Add a pip installer for PySpark: Resolved
• Sign pip artifacts: Resolved
• Add support for publishing to PyPI: Resolved
• Build only a single pip package: Resolved
• Include the example data and third-party licenses in pyspark package: Resolved
• Upload to PyPi: Resolved