Description
It's not possible to run Pyspark shell with cluster mode since the shell that is running in the cluster is not being able to interact with the client.
We can build a proxy that is transferring the inputs of the user and the output of the shell, and also be able to get connected and reconnected from the user.
Attachments
Issue Links
- is a clone of
-
SPARK-10160 Support Spark shell over Mesos Cluster Mode
- Resolved