[SPARK-11563] Use RpcEnv to transfer generated classes in spark-shell

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: Spark Shell
• Assignee: Marcelo Masiero Vanzin
• Reporter: Marcelo Masiero Vanzin
• Created: 06/Nov/15 23:35
• Updated: 10/Dec/15 21:26
• Resolved: 10/Dec/15 21:26

Description

Similar to SPARK-11140 (and building on top of it), spark-shell should transfer generated classes using the RpcEnv support for file transfers added in that change.

That would mean easy security configuration and, as a bonus, lower overhead (and one less HTTP server running inside Spark).

Issue Links

• is blocked by: SPARK-11140 Replace file server in driver with RPC-based alternative
• links to: [Github] Pull Request #9923 (vanzin)