[SPARK-12652] Upgrade py4j to the incoming version 0.9.1

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: PySpark
• Target Version/s: 2.0.0
• Assignee: Shixiong Zhu
• Reporter: Shixiong Zhu
• Created: 05/Jan/16 18:53
• Updated: 12/Jan/16 22:28
• Resolved: 12/Jan/16 22:28

Description

Upgrade py4j when py4j 0.9.1 is out. Mostly because it fixes two critical issues: SPARK-12511 and SPARK-12617

Issue Links

• blocks: SPARK-12511 streaming driver with checkpointing unable to finalize leading to OOM
• blocks: SPARK-12617 socket descriptor leak killing streaming app
• links to: [Github] Pull Request #10692 (zsxwing)

Sub-Tasks

• Revert SPARK-12617: Resolved
• Revert SPARK-12511: Resolved