[SPARK-18128] Add support for publishing to PyPI

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.2.0
• Component/s: PySpark
• Assignee: Holden Karau
• Reporter: Holden Karau
• Created: 27/Oct/16 00:50
• Updated: 12/Sep/17 20:03
• Resolved: 12/Sep/17 20:03

Description

After SPARK-1267 is done we should add support for publishing to PyPI similar to how we publish to maven central.

Note: one of the open questions is what to do about package name since someone has registered the package name PySpark on PyPI - we could use ApachePySpark or we could try and get find who registered PySpark and get them to transfer it to us (since they haven't published anything so maybe fine?)

Issue Links

• is blocked by: SPARK-1267 Add a pip installer for PySpark